Ubuntu Manpage: walmgr - tools for managing WAL-based replication for PostgreSQL.

NAME

       walmgr - tools for managing WAL-based replication for PostgreSQL.

SYNOPSIS

       walmgr.py <config.ini> command

DESCRIPTION

       It is both admin and worker script for PostgreSQL PITR replication.

QUICK START

1. Set up passwordless ssh authentication from master to slave

master$ test -f ~/.ssh/id_dsa.pub || ssh-keygen -t dsa
master$ cat ~/.ssh/id_dsa.pub | ssh slave cat \>\> .ssh/authorized_keys

2. Configure paths

master$ edit master.ini
slave$ edit slave.ini

Make sure that walmgr.py executable has same pathname on slave and master.

3. Start archival process and create a base backup

master$ ./walmgr.py master.ini setup
master$ ./walmgr.py master.ini backup

Note: starting from PostgreSQL 8.3 the archiving is enabled by setting
archive_mode GUC to on. However changing this parameter requires the
server to be restarted.

4. Prepare postgresql.conf and pg_hba.conf on slave and start replay

master$ scp $PGDATA/*.conf slave:
slave$ ./walmgr.py slave.ini restore

For debian based distributions the standard configuration files are located
in /etc/postgresql/x.x/main directory. If another scheme is used the postgresql.conf
and pg_hba.conf should be copied to slave full_backup directory. Make sure to
disable archive_command in slave config.

'walmgr.py restore' moves data in place, creates recovery.conf and starts postmaster
in recovery mode.

5. In-progress WAL segments can be backup by command:

master$ ./walmgr.py master.ini sync

6. If need to stop replay on slave and boot into normal mode, do:

slave$ ./walmgr.py slave.ini boot

GENERAL OPTIONS

       Common options to all walmgr.py commands.

       -h, --help
           show this help message and exit

       -q, --quiet
           make program silent

       -v, --verbose
           make program more verbose

       -n, --not-really
           Show what would be done without actually doing anything.

MASTER COMMANDS

   setup
       Sets up postgres archiving, creates necessary directory structures on slave.

   sync
       Synchronizes in-progress WAL files to slave.

   syncdaemon
       Start WAL synchronization in daemon mode. This will start periodically synching the
       in-progress WAL files to slave.

       The following parameters are used to drive the syncdaemon: loop_delay - how long to sleep
       between the synchs. use_xlog_functions - use record based shipping to synchronize
       in-progress WAL segments.

   stop
       Deconfigures postgres archiving.

   periodic
       Runs periodic command, if configured. This enables to execute arbitrary commands on
       interval, useful for synchronizing scripts, config files, crontabs etc.

   listbackups
       List backup sets available on slave node.

   backup
       Creates a new base backup from master database. Will purge expired backups and WAL files
       on slave if keep_backups is specified. During a backup a lock file is created in slave
       completed_wals directory. This is to prevent simultaneous backups and resulting
       corruption. If running backup is terminated, the BACKUPLOCK file may have to be removed
       manually.

   restore <set> <dst>
       EXPERIMENTAL. Attempts to restore the backup from slave to master.

SLAVE COMMANDS

   boot
       Stop log playback and bring the database up.

   pause
       Pauses WAL playback.

   continue
       Continues previously paused WAL playback.

   listbackups
       Lists available backups.

   backup
       EXPERIMENTAL. Creates a new backup from slave data. Log replay is paused, slave data
       directory is backed up to full_backup directory and log replay resumed. Backups are
       rotated as needed. The idea is to move the backup load away from production node. Usable
       from postgres 8.2 and up.

   restore [src][dst]
       Restores the specified backup set to target directory. If specified without arguments the
       latest backup is moved to slave data directory (doesn’t obey retention rules). If src
       backup is specified the backup is copied (instead of moving). Alternative destination
       directory can be specified with dst.

CONFIGURATION

Common settings
job_name
Optional. Indentifies this script, used in logging. Keep unique if using central
logging.

logfile
Where to log.

use_skylog
Optional. If nonzero, skylog.ini is used for log configuration.

Master settings
pidfile
Pid file location for syncdaemon mode (if running with -d). Otherwise not required.

master_db
Database to connect to for pg_start_backup() etc. It is not a good idea to use
dbname=template if running syncdaemon in record shipping mode.

master_data
Master data directory location.

master_config
Master postgresql.conf file location. This is where archive_command gets updated.

master_restart_cmd
The command to restart master database, this used after changing archive_mode
parameter. Leave unset, if you cannot afford to restart the database at setup/stop.

slave
Slave host and base directory.

slave_config
Configuration file location for the slave walmgr.

completed_wals
Slave directory where archived WAL files are copied.

partial_wals
Slave directory where incomplete WAL files are stored.

full_backup
Slave directory where full backups are stored.

config_backup
Slave directory where configuration file backups are stored. Optional.

loop_delay
The frequency of syncdaemon updates. In record shipping mode only incremental updates
are sent, so smaller interval can be used.

use_xlog_functions
Use pg_xlog functions for record based shipping (available in 8.2 and up).

compression
If nonzero, a -z flag is added to rsync cmdline. Will reduce network traffic at the
cost of extra CPU time.

periodic_command
Shell script to be executed at specified time interval. Can be used for synchronizing
scripts, config files etc.

command_interval
How ofter to run periodic command script. In seconds, and only evaluated at log switch
times.

hot_standby ===
Boolean. If set to true, walmgr setup will set wal_level to hot_standby (9.0 and
newer).

Sample master.ini
[wal-master]
logfile = master.log
pidfile = master.pid
master_db = dbname=template1
master_data = /var/lib/postgresql/8.0/main
master_config = /etc/postgresql/8.0/main/postgresql.conf
slave = slave:/var/lib/postgresql/walshipping
completed_wals = %(slave)s/logs.complete
partial_wals = %(slave)s/logs.partial
full_backup = %(slave)s/data.master
loop_delay = 10.0
use_xlog_functions = 1
compression = 1

Slave settings
slave_data
Postgres data directory for the slave. This is where the restored backup is
copied/moved.

slave_config_dir
Directory for postgres configuration files. If specified, "walmgr restore" attempts to
restore configuration files from config_backup directory.

slave_stop_cmd
Script to stop postmaster on slave.

slave_start_cmd
Script to start postmaster on slave.

slave
Base directory for slave files (logs.complete, data.master etc)

slave_bin
Specifies the location of postgres binaries (pg_controldata, etc). Needed if they are
not already in the PATH.

completed_wals
Directory where complete WAL files are stored. Also miscellaneous control files are
created in this directory (BACKUPLOCK, STOP, PAUSE, etc.).

partial_wals
Directory where partial WAL files are stored.

full_backup
Directory where full backups are stored.

keep_backups
Number of backups to keep. Also all WAL files needed to bring earliest

backup up to date are kept. The backups are rotated before new backup is started, so
at one point there is actually one less backup available.

It probably doesn’t make sense to specify keep_backups if periodic backups are not
performed - the WAL files will pile up quickly.

Backups will be named data.master, data.master.0, data.master.1 etc.

archive_command
Script to execute before rotating away the oldest backup. If it fails backups will not
be rotated.

slave_pg_xlog
Set slave_pg_xlog to the directory on the slave where pg_xlog files get written to. On
a restore to the slave walmgr.py will create a symbolic link from data/pg_xlog to this
location.

backup_datadir
Set backup_datadir to no to prevent walmgr.py from making a backup of the data
directory when restoring to the slave. This defaults to yes

Sample slave.ini
[wal-slave]
logfile = slave.log
slave_data = /var/lib/postgresql/8.0/main
slave_stop_cmd = /etc/init.d/postgresql-8.0 stop
slave_start_cmd = /etc/init.d/postgresql-8.0 start
slave = /var/lib/postgresql/walshipping
completed_wals = %(slave)s/logs.complete
partial_wals = %(slave)s/logs.partial
full_backup = %(slave)s/data.master
keep_backups = 5
backup_datadir = yes

03/13/2012 WALMGR(1)