Physical Backups with pg_basebackup
Take a full streaming physical backup of the cluster, restore it to an alternate data directory, and start it as a second, independent, writable instance
Every backup so far in this block has been logical: pg_dump and pg_dumpall read data through SQL and write out instructions to recreate it. pg_basebackup is fundamentally different — it copies PGDATA itself, the real data files a live PostgreSQL server is reading and writing to right now, while the server keeps running.\n\nThat raises an obvious problem: files are being written while they are being copied. A file copied at the start of the backup might be mid-write by the time the backup finishes minutes later, and files copied later reflect a different moment than files copied first. pg_basebackup solves this by also streaming every WAL record generated during the copy, then replaying it into the copied files on startup — the copy is not consistent by itself, but copy plus WAL together always are.\n\nThis is also the mechanism that bootstraps every PostgreSQL streaming replica: a standby is, at its core, a pg_basebackup copy that keeps applying WAL forever instead of stopping once. In this lab you take a backup, restore it to an alternate directory, and start it as a genuinely independent second instance — the exact first step of standing up a replica, minus the ongoing streaming.
Taking a Streaming Physical Backup
pg_basebackup -U postgres -h /tmp -D /tmp/basebackup -Fp -Xs -c fast -P
| Flag | Meaning |
|------|---------|
| -D | Target directory to write the backup into |
| -Fp | Plain format — an actual copy of PGDATA's directory structure |
| -Xs / --wal-method=stream | Stream WAL generated during the backup over a second connection, so the backup is self-contained |
| -c fast / --checkpoint=fast | Force an immediate checkpoint at the start instead of waiting for the next scheduled one |
| -P | Show live progress |
Checkpoint: fast vs. spread
pg_basebackup needs the current checkpoint's redo point as its starting reference. --checkpoint=spread (the default) lets the checkpoint happen gradually, at the server's normal pace, to avoid an I/O spike on a busy production server. --checkpoint=fast forces it immediately, starting the backup sooner — the right choice when the backup itself is more time-sensitive than the brief I/O spike a forced checkpoint causes.
Without Streaming WAL — Why It Would Be Broken
# --wal-method=none (rarely what you want)
pg_basebackup -D /tmp/backup_incomplete --wal-method=none
A backup taken this way is not usable on its own — it needs every WAL segment generated during the copy, archived separately, to reach consistency. --wal-method=stream (the default in modern PostgreSQL) avoids this entirely by streaming those segments alongside the file copy, so the backup directory is self-contained the moment pg_basebackup finishes.
Starting the Backup as a Live Instance
pg_ctl start -D /tmp/basebackup -o "-p 5433" -l /tmp/basebackup/startup.log
psql -h /tmp -p 5433 -U postgres -c "SELECT pg_is_in_recovery();"
On startup, PostgreSQL detects the backup_label file pg_basebackup wrote, replays the streamed WAL to reach a consistent state, removes the label, and — because no standby.signal file is present — becomes an ordinary, independent, writable primary. (Passing -R to pg_basebackup instead writes standby.signal plus connection info automatically, turning this into the first step of a real streaming replica — outside the scope of this lab.)
pg_basebackup
Copies a running PostgreSQL cluster's actual data files directly, rather than reconstructing objects through SQL. It streams the WAL generated during the copy alongside the file copy itself, so the result reaches a consistent state on startup even though individual files were captured at different moments during the copy.
Streaming WAL During Backup (-Xs)
pg_basebackup opens a second connection specifically to stream WAL records as they are generated throughout the backup, writing them into the target directory's pg_wal/. On startup, those records are replayed against the copied files to bring them to a single consistent point — without this, a backup taken while the server is live would not be usable on its own.
backup_label
A file pg_basebackup writes at the root of the target directory, recording the WAL position where the backup began. PostgreSQL reads it automatically on startup to know it is recovering from a base backup rather than performing an ordinary crash recovery, then removes it once the replay reaches a consistent state.
🔎 Check — and Fix — wal_level and max_wal_senders
pg_basebackup needs wal_level at least replica and at least one available WAL sender slot. This cluster ships with neither — check first, then configure both and restart to apply.
psql -U postgres -c "SHOW wal_level;" -c "SHOW max_wal_senders;"echo "wal_level = replica" | as-postgres tee -a /var/lib/postgresql/18/data/postgresql.confecho "max_wal_senders = 10" | as-postgres tee -a /var/lib/postgresql/18/data/postgresql.confas-postgres pg_ctl restart -D /var/lib/postgresql/18/data -l /var/lib/postgresql/18/data/pg_log/startup.logstudent@lab:~$ psql -U postgres -c "SHOW wal_level;" -c "SHOW max_wal_senders;" SET wal_level ----------- minimal (1 row) max_wal_senders ------------------ 0 (1 row) student@lab:~$ echo "wal_level = replica" | as-postgres tee -a /var/lib/postgresql/18/data/postgresql.conf wal_level = replica student@lab:~$ echo "max_wal_senders = 10" | as-postgres tee -a /var/lib/postgresql/18/data/postgresql.conf max_wal_senders = 10 student@lab:~$ as-postgres pg_ctl restart -D /var/lib/postgresql/18/data -l /var/lib/postgresql/18/data/pg_log/startup.log waiting for server to shut down.... done server stopped waiting for server to start.... done server started
🗜️ Try tar Format First — a Quick, Throwaway Look
Take a small tar-format backup purely to see the difference in shape, then delete it immediately. This VM has just enough /tmp space for one full cluster copy at a time — the tar demo has to be cleaned up before the real, kept backup in the next step.
pg_basebackup -U postgres -h /tmp -D /tmp/basebackup_tar -Ft -Xs -c fastls -la /tmp/basebackup_tarrm -rf /tmp/basebackup_tarstudent@lab:~$ pg_basebackup -U postgres -h /tmp -D /tmp/basebackup_tar -Ft -Xs -c fast student@lab:~$ ls -la /tmp/basebackup_tar total 132072 drwx------ 2 student student 4096 Jun 26 16:02 . drwxrwxrwt 3 root root 4096 Jun 26 16:02 .. -rw------- 1 student student 355170 Jun 26 16:02 backup_manifest -rw------- 1 student student 118095360 Jun 26 16:02 base.tar -rw------- 1 student student 16778752 Jun 26 16:02 pg_wal.tar student@lab:~$ rm -rf /tmp/basebackup_tar
📸 Take the Streaming Physical Backup
Run pg_basebackup in plain format, streaming WAL, with an immediate checkpoint and live progress. This is the copy the rest of this lab uses.
pg_basebackup -U postgres -h /tmp -D /tmp/basebackup -Fp -Xs -c fast -Pstudent@lab:~$ pg_basebackup -U postgres -h /tmp -D /tmp/basebackup -Fp -Xs -c fast -P 115327/115327 kB (100%), 1/1 tablespace
📁 Inspect the Backup Directory
Confirm the backup directory has the structure of a real PGDATA — not a script, not an archive, an actual cluster copy.
ls -la /tmp/basebackupstudent@lab:~$ ls -la /tmp/basebackup total 488 drwx------ 20 student student 4096 Jun 26 16:02 . drwxrwxrwt 3 root root 4096 Jun 26 16:02 .. -rw------- 1 student student 3 Jun 26 16:02 PG_VERSION -rw------- 1 student student 225 Jun 26 16:02 backup_label -rw------- 1 student student 355029 Jun 26 16:02 backup_manifest drwx------ 9 student student 4096 Jun 26 16:02 base drwx------ 2 student student 4096 Jun 26 16:02 global drwx------ 2 student student 4096 Jun 26 16:02 pg_commit_ts drwx------ 2 student student 4096 Jun 26 16:02 pg_dynshmem -rw------- 1 student student 5721 Jun 26 16:02 pg_hba.conf -rw------- 1 student student 2681 Jun 26 16:02 pg_ident.conf drwxr-xr-x 2 student student 4096 Jun 26 16:02 pg_log drwx------ 4 student student 4096 Jun 26 16:02 pg_logical drwx------ 4 student student 4096 Jun 26 16:02 pg_multixact drwx------ 2 student student 4096 Jun 26 16:02 pg_notify drwx------ 2 student student 4096 Jun 26 16:02 pg_replslot drwx------ 2 student student 4096 Jun 26 16:02 pg_serial drwx------ 2 student student 4096 Jun 26 16:02 pg_snapshots drwx------ 2 student student 4096 Jun 26 16:02 pg_stat drwx------ 2 student student 4096 Jun 26 16:02 pg_stat_tmp drwx------ 2 student student 4096 Jun 26 16:02 pg_subtrans drwx------ 2 student student 4096 Jun 26 16:02 pg_tblspc drwx------ 2 student student 4096 Jun 26 16:02 pg_twophase drwx------ 4 student student 4096 Jun 26 16:02 pg_wal drwx------ 2 student student 4096 Jun 26 16:02 pg_xact -rw------- 1 student student 88 Jun 26 16:02 postgresql.auto.conf -rw------- 1 student student 33479 Jun 26 16:02 postgresql.conf
▶️ Start the Backup as a Second Instance
Start the plain-format backup directory as its own live PostgreSQL instance on port 5434.
pg_ctl start -D /tmp/basebackup -o "-p 5434" -l /tmp/basebackup/startup.logstudent@lab:~$ pg_ctl start -D /tmp/basebackup -o "-p 5434" -l /tmp/basebackup/startup.log waiting for server to start.... done server started
✅ Confirm It Reached Consistency
Query pg_is_in_recovery() on the new instance — it should return false, meaning WAL replay finished and it is now an ordinary writable primary.
psql -h /tmp -p 5434 -U postgres -c "SELECT pg_is_in_recovery();"student@lab:~$ psql -h /tmp -p 5434 -U postgres -c "SELECT pg_is_in_recovery();" SET pg_is_in_recovery -------------------- f (1 row)
🔢 Verify the Data Is Actually There
Confirm the restored instance has the same data as the primary — query beer_db's beers table on port 5434.
psql -h /tmp -p 5434 -U postgres -d beer_db -c "SELECT count(*) FROM beers;"student@lab:~$ psql -h /tmp -p 5434 -U postgres -d beer_db -c "SELECT count(*) FROM beers;" SET count ------- 400 (1 row)
🧹 Stop the Second Instance
Shut down the port-5434 instance cleanly now that it has been verified.
pg_ctl stop -D /tmp/basebackup -m faststudent@lab:~$ pg_ctl stop -D /tmp/basebackup -m fast waiting for server to shut down.... done server stopped
Lab 2.3.4 complete. You can now take, restore, and verify a full physical backup:\n\n\n pg_basebackup -Xs -c fast : ✅ streaming physical backup taken\n Plain + tar format : ✅ both demonstrated\n Restored as instance : ✅ started on an alternate port\n pg_is_in_recovery = f : ✅ consistency confirmed\n Data verified : ✅ full cluster, not one database\n
Enable JavaScript to run the live terminal and track your progress.