Considering PITRtools 1.4

We quietly released PITRTool 1.3 last week. This version has been in development for a long time and over the past 6 months became a priority to complete. There is one known minor issue that may or may not be fixed as it doesn't affect production usage in a meaningful way. Release 1.3 contiues to support all the way back to 8.2 with warm standby but we also now support streaming replication and hot standby.

With 1.4 we will be making some changes, quite a few of them in fact. Over the years replication and and log shipping has matured in PostgreSQL. We want to take advantage of that maturity. With that here are some of the changes we plan to make:

  • Allow -B in cmd_standby to be multithreaded as well as CPU throttled.

    This will allow base backups to take advantage of multi-core machines as well as limit the amount of CPU resources that the process takes up. This will effectively eliminate any benefit to pg_basebackup when using PITRtools.

  • Allow for generic start/stop/restart cluster commands.

    This was an interesting problem we came across with one of our clients who still run Solaris. They are using zones as well SMF. This means that PITRTools out of the box didn't properly work on their environment because we directly call pg_ctl from PITRtools. The solution we came up is to allow the following in the cmd_standby.ini

    pg_start_cmd
    pg_stop_cmd
    pg_restart_cmd
    

    This will allow custom calls from SMF or any other management interface to stop and start PostgreSQL.

  • Make sane defaults for configuration options

    The configuration file has grown to a ridiculous number of options in 1.3. For 1.4 we are going to assume most commands are in $PATH. This will remove the need for many of our configuration options. We are considering still allowing specification of location if you don't use a standard path but that has yet to be decided.

  • Add better monitoring and logging options When PITRtools was originally written there was no reasonable way to know what the state of a warm standby was. That has changed with recent releases of PostgreSQL. Thus we will enable the ability to monitor and log in a more verbose manner.

  • Compress wal logs that are shipped/archived

    PITRtools already has a queuing mechanism for wal logs but they are stored uncompressed. This could benefit from compression on the disk.

  • Potentially adding non replication options

    PITRtools has all the mechanisms to properly handle postgresql backups. Why not add the option?

    Of course more things will come as we discuss further. Feel free to join the project and let us know your ideas!

    1. PITRtools