Backup and Restore

PicPeak’s built-in backup service runs on a cron schedule and writes to local disk, an rsync/SSH server or S3-compatible storage. Each run to a local or S3 destination is a complete, standalone restore point; every manifest is signed with a dedicated key that you must keep separately from the backups.

Manual backup (DB + files)

If you don’t want the built-in service, you can backup manually with one-shot commands:

Database

docker exec picpeak-postgres pg_dump -U picpeak picpeak_prod > backup/db_$(date +%Y%m%d_%H%M%S).sql

Files

tar -czf backup/photos_$(date +%Y%m%d_%H%M%S).tar.gz events/ thumbnails/

For SQLite:

cp data/picpeak.db backup/picpeak_$(date +%Y%m%d_%H%M%S).db

Built-in backup service

Configure under Admin → Backup & Restore.

Settings reference

Where the database dump is written

Every file backup first writes a database dump (the inline dump, on by default). Scheduled and manual database backups use the same directory. It is controlled by database_backup_destination_path, a setting the admin UI does not show; installs seed it as /backup/database.

In builds containing PicPeak PR #1502  the directory is chosen like this:

1. A customised database_backup_destination_path is used as set.
2. The seeded /backup/database (or an empty value) is kept when /backup/database or /backup is writable. The default docker-compose.yml (./backup:/backup) and the all-in-one image provide /backup, so nothing changes there.
3. Otherwise the dump goes to <backup_destination_path>/database, or to <storage>/backups/database when no backup destination is set, so it ends up inside the backup.

A publicly served directory is refused. If the directory cannot be created, the backup fails with Cannot create the database backup directory <path>: <code>. Set database_backup_destination_path to a directory the backend can write to, or mount a writable volume at that path. In production that error is written to logs/error.log; it only appears in docker compose logs when LOG_TO_CONSOLE=true.

The database backup test configuration check (POST /api/admin/database-backup/test) tests and reports this same directory in builds containing PicPeak PR #1504 . Older builds always reported the destination as not writable.

    environment:
      - BACKUP_DIR=/backup
    volumes:
      - ./backup:/backup

Local destination checks

The local destination is checked the same way by Test connection and by the backup run (beta channel from v3.158.2-beta.0 and v3.165.1-beta.0, stable from v3.133.10 and v3.134.1):

• Not writable. If the directory cannot be created, the test answers LOCAL_PATH_NOT_WRITABLE and names the nearest existing directory that blocks it. A path like /home/ubuntu/picpeak/backups that exists on the host but not in the container fails this way; use a container path such as /app/storage/backups or the /backup mount. A directory that does not exist yet but can be created passes.
• Inside a backed-up folder. A destination inside a folder the backup copies (for example <storage>/uploads/backups) is skipped by the file walk, so the backup does not copy itself. This also works when the same host folder is visible at two container paths.
• Equal to a backed-up folder. A destination that is a backed-up folder, or the storage folder itself, cannot be skipped. The test answers LOCAL_PATH_IS_BACKED_UP_FOLDER and backups fail before the dump is written, with a message naming the folder. Pick a directory outside the backed-up folders.

Saving the form trims the destination path.

S3 destination settings

Each S3 run is stored under {prefix}/YYYY/MM/DD/backup-<uuid>/, with its manifest in a manifests/ folder inside that prefix.

Private S3 endpoints

In production, an S3 endpoint that resolves to a private address (RFC 1918, loopback, IPv6 unique-local), such as a MinIO or RustFS container on the same Docker network, is refused with S3_PRIVATE_ENDPOINT until a Super Admin approves it. The Backup page then shows a panel with the endpoint’s origin and an approval checkbox; the next Test connection or save carries the approval. It covers only that scheme, host and port: an approval for another origin answers S3_APPROVAL_MISMATCH, and moving the endpoint to a public address clears it.

Link-local, cloud-metadata, carrier-grade NAT, reserved and multicast addresses answer S3_ENDPOINT_FORBIDDEN and can never be approved. Every connection re-checks the DNS answers, so a hostname that resolved publicly when it was saved cannot later point uploads at an internal address. Test connection runs a real bucket check, and reuses the saved secret only against the endpoint it was saved for. (Beta channel from v3.142.1-beta.0, stable from v3.133.5.)

Rsync destination settings

SSH host keys

Rsync connects with StrictHostKeyChecking=yes and never learns a host key on first use. Before the first test or backup, get the destination’s host key through a channel you trust (not unverified ssh-keyscan output) and provide a known_hosts entry for the hostname:

• set BACKUP_SSH_KNOWN_HOSTS to the absolute path of a readable known_hosts file, or
• put known_hosts in the same directory as the configured SSH key.

With neither, pre-provisioned OpenSSH default user/global trust is used. Read-only trust files work. For a port other than 22 the entry is named [host]:port, as ssh-keyscan -p writes it. An unknown or changed key fails with RSYNC_SSH_TRUST_REQUIRED; if the selected trust file is missing, the backend logs the expected path at startup. The SSH command ignores local SSH configuration (aliases, proxies, jump hosts, ports from config) and connects to the vetted address while checking the key against the configured hostname. Mount the key and trust files persistently, outside the backed-up data. (Beta channel from v3.167.2-beta.0, stable from v3.134.2.)

Restore points

On the beta channel from v3.167.3-beta.0 and stable from v3.134.2, every run to a local or S3 destination copies every selected file into a new, standalone restore point (backup-<uuid>/ locally, its own prefix on S3), together with the run’s database dump. A restore point does not depend on earlier runs, so any one of them can be restored on its own. Plan destination capacity for backup_retention_count complete copies of the selected files.

A run that cannot copy a required file, the dump or the manifest fails; it is not recorded as a completed partial backup.

Rsync keeps syncing into one mirror on the remote host, so only changed files are transferred and its manifests are recorded as incremental.

S3 as primary storage

With STORAGE_BACKEND=s3, standard backups read originals, archives, derived media, customer documents and transfer attachments through the storage adapter, with their object metadata; CRM PDFs, signatures and static uploads stay local and are included alongside. The backup destination (backup_s3_*, local or rsync) is independent of the live media bucket. The run stages the files in a private temporary directory, so provide temporary disk space for the whole selected file set. Before restoring onto a fresh host, configure the target STORAGE_BACKEND and STORAGE_S3_* bucket and credentials first. (Beta channel from v3.167.6-beta.0, stable from v3.134.3.)

Manifest format

Every backup writes a manifest alongside the photos. The manifest is the index — it tells the restore process what to expect.

manifest:
  version: '3.0'
backup:
  id: backup-20260428-022150-a1b2c3d4
  type: full
  parent_backup_id: null
  timestamp: 2026-04-28T02:21:50.000Z
files:
  count: 1245
  manifest:
    - path: events/active/wedding-smith/DSC_0001.jpg
      size: 4523894
      checksum: ...
    - ...
database:
  type: postgresql
  backup_file: ...
  checksum: ...                  # SHA-256 of the dump
verification:
  checksum_algorithm: hmac-sha256
  serialization: canonical-v1
  key_id: ...
  total_checksum: ...            # HMAC-SHA256 signature over the manifest

Format is json or yaml per backup_manifest_format. JSON is more compact; YAML is human-readable.

The manifest also records the backup settings that were in force. In builds containing PicPeak PR #1809  credentials are left out of that record: the S3 access and secret key, the rsync SSH key and any other setting whose name marks it as a secret. Manifests written by older builds still contain them — treat old manifests like the credentials themselves, and rotate the keys if a manifest may have been read by someone who should not have them.

Manifest signing key

On the beta channel from v3.167.2-beta.0 and stable from v3.134.2, every standard (JSON/YAML) manifest is signed with HMAC-SHA256 using a dedicated 32-byte key. The signature covers the whole restore description, including dump and file checksums. A restore refuses a manifest when the key is missing, the manifest is unsigned, or its signature does not match; force and the install trigger do not bypass this. Portable .picpeak files are a separate format and do not use this key.

Where the key lives:

Key status. System Health reports whether the key is ready and whether the latest manifest is authenticated, separately from whether the backup completed. A latest manifest that predates signing is shown as legacy; the next backup signs a new one. An invalid key (wrong encoding, a symlink, or a group/world-writable file) does not stop the server: it is logged at boot and shown in System Health, and backups fail until it is fixed.

On the beta channel from v3.174.0-beta.0 the Backup page also shows a notice with the reason, a newly created key is read back so an unusable one stops the backup before anything is copied, and a key file the PicPeak process owns is restricted to 0600 automatically. On a filesystem that does not keep file modes (some NAS, VM shared folders, bind mounts that report every file as 0777/0666) the backup fails with:

The backup manifest key file <file> is group- or world-writable (mode 0777), so another account could replace the key that signs backups. chmod 600 did not take effect: its filesystem does not keep file modes (some NAS, VM shared folders or bind mounts). Move the key to a filesystem that does and point BACKUP_MANIFEST_KEY_FILE at it, or set BACKUP_MANIFEST_KEY to its 64 hex digits.

Follow one of the two fixes it names. A key directory that other accounts can write to is logged as a warning.

Rotating the key. Set the new key and list earlier keys in BACKUP_MANIFEST_KEYS_OLD (comma-separated 64-hex values) for as long as you keep backups they signed. Manifests signed before v3 signing by an earlier keyed build can be verified with BACKUP_MANIFEST_LEGACY_KEY (the original key string); a passphrase-style BACKUP_MANIFEST_KEY left in place verifies them the same way but never signs new ones.

Unsigned or lost-key backups. A backup that is unsigned, was signed by the old serializer, or whose key is lost is unauthenticated. Restore it only on an isolated host after inspecting its contents yourself (a database dump can contain arbitrary SQL). Run node scripts/backup-manifest-recovery-digest.js /path/to/manifest.json from backend/, set BACKUP_MANIFEST_RECOVERY_SHA256 to the digest it prints and BACKUP_MANIFEST_RECOVERY_REASON to a reason, restart and restore that one manifest. The approval does not carry over to other contents, and the restore logs the unauthenticated outcome. Remove both variables afterwards, restart, and take a new signed backup.

All key variables are listed under Environment Variables.

Retention & rotation

After every successful backup the service keeps the newest backup_retention_count completed restore points of the current destination type (local or S3) and deletes older ones whole, together with their history rows. 0 keeps everything. Legacy backup trees and anything not shaped like a standalone restore point are never deleted, rsync mirrors are not pruned, and a failed cleanup never fails the backup that just succeeded. Scheduled database dumps have their own retention (database_backup_retention_days).

If you want to keep a permanent ground-truth backup, copy it to a separate prefix, bucket or folder — anything inside the rotated path is fair game.

Deleting a backup

On the beta channel (from v3.148.2-beta.0) the History table has a Delete backup action (DELETE /api/admin/backup/runs/:id, permission backup.delete). It removes the run’s record and its manifest; for an S3 run it also deletes the objects under the run’s prefix. A running backup cannot be deleted (BACKUP_RUNNING). If the run’s files lie outside the configured destination (ARTIFACT_OUT_OF_SCOPE) or the manifest cannot be removed, nothing is deleted and the record is kept. Each deletion is written to the activity log.

Restore

From the admin UI

The Backup & Restore page lists every available backup with a Restore button. Behavior:

1. Downloads the backup files from the destination.
2. Restores the database (overwrites the existing DB).
3. Restores files to events/active/, events/archived/, thumbnails/ etc.
4. Verifies checksums against the manifest.
5. Reports any files that failed verification.

Restores and .picpeak imports replace admin accounts, roles and permissions from the file, so in builds containing PicPeak PR #1493  they require a Super Admin. Other roles see an explanation on the Restore tab instead of the wizard.

Before the database is touched, the restore checks the dump file against the SHA-256 recorded in the manifest (database.checksum) and refuses a dump that was changed or truncated. From v3.167.2-beta.0 (beta) and v3.134.2 (stable) the manifest must also carry a valid signature (see Manifest signing key); a manifest without a dump checksum is refused instead of restoring with a warning, and BACKUP_MANIFEST_REQUIRE_KEYED no longer exists. The only way to restore an unsigned backup is the one-artifact recovery approval described there.

Local restore paths must lie inside backup_destination_path or backup_manifest_path, also after symbolic links are resolved. To restore from a folder elsewhere (for example a rescued disk mounted at another path), list it in RESTORE_ALLOWED_ROOTS (several paths separated by :, or ; on Windows).

A .picpeak import is checked before and while it is extracted: too many entries (PICPEAK_IMPORT_MAX_ENTRIES), more unpacked bytes than PICPEAK_IMPORT_MAX_EXPANDED_BYTES or than the free space of the extraction directory, or an oversized manifest.json (PICPEAK_IMPORT_MAX_MANIFEST_BYTES) stop the import before anything is restored. See Environment Variables.

The UI restore is a one-shot, full-rollback operation — run it on a fresh instance, not over a live one with newer data, unless you’ve thought through the consequences.

In builds containing PicPeak PR #1809  a restore is stricter about what it accepts and what it reports:

• Every file counts. A file that is missing, unreadable or fails its checksum fails the restore; it is no longer reported as complete with some files skipped. The check covers the whole manifest.
• Size limit. restore_max_file_size_mb (default 5000) is enforced for every file, on the size the manifest states and on the bytes actually read. An oversized or swapped object stops the restore, and the partial file is removed.
• No symlinks. A file in a local backup must be a regular file inside the backup folder. Symbolic links are refused.
• Everyone signs in again. After a full or database restore, after a rollback and after a .picpeak import, all admin, customer and gallery sessions issued before it are invalid.
• .picpeak imports only restore files under the folders an export writes: business-docs, uploads, events/active and events/archived. Anything else in the archive stops the import. Under the publicly served folders uploads/logos and uploads/favicons, HTML and JavaScript files are refused.
• S3 manifests larger than 16 MiB are refused.

.picpeak imports

A .picpeak import replaces the whole database and the managed files (beta channel from v3.168.1-beta.0, stable from v3.134.4; details in Portable restore ):

• Every table is cleared, including tables the archive does not list because they did not exist when it was made; those end up empty.
• Paths containing .., . or empty segments are refused.
• Host requirements. The import needs Linux (the Docker images qualify), the native restore components (cd backend && npm run build:native; Docker builds them), a process guard allowed to supervise the worker (Docker’s default profile allows it), and STORAGE_PATH on a local filesystem, not NFS or CIFS. Where one is missing, PicPeak runs normally, the import is shown as unavailable with the reason, and the import route answers 503.
• Maintenance. While the import runs, the instance answers ordinary requests with 503 and /health answers 200 with "maintenance": "restoring". Running work gets PICPEAK_RESTORE_DRAIN_GRACE_MS (default 60 s) to finish first.
• After a failure the files and database are rolled back and the instance reopens by itself. After a committed import, restart every backend process once, including background-only instances. All instances must share one host, database and STORAGE_PATH volume.
• Interrupted imports. An import interrupted by a crash or reboot keeps the instance in maintenance as recovery_required until the next start recovers it. Do not delete the maintenance files under <STORAGE_PATH>/.picpeak-maintenance to clear it; if it stays stuck, stop the backend and run node scripts/clear-portable-restore-fence.js (add --confirm to clear).

SQLite: characters in the dump path

On SQLite installs, database_backup_destination_path must not contain quotes, backslashes or control characters; saving such a path is refused (builds containing PicPeak PR #1809 ).

scripts/backup.sh

The helper script writes its archive readable by the calling user only (directory 0700, archive 0600), because the archive contains .env (builds containing PicPeak PR #1809 ).

Manual restore — DB

cat backup/db_20260428_022150.sql | docker exec -i picpeak-postgres psql -U picpeak picpeak_prod

Manual restore — files

tar -xzf backup/photos_20260428_022150.tar.gz -C /path/to/storage/

Coverage tab

Backup → Coverage answers “what will the next backup actually include?” without having to run it:

• Database — inline-dump mode + last dump timestamp + staleness check.
• Configured paths — one row per backup-path entry with its current coverage status (will-scan / skipped-by-toggle / skipped-by-feature-flag / missing-on-disk).
• Drift detection — flags top-level directories under STORAGE_PATH that exist on disk but have no matching backup-path row. This is the canary for “a feature shipped without a matching backup row” — the most common cause of silent data loss on older installs.

The Coverage tab auto-fetches on open. If everything is green, your next backup will capture what you’d expect.

Integrity tab

Backup → Integrity verifies that every *_path column on quotes / contracts / invoices / signatures actually resolves to a file on disk, and that files with a stored *_sha256 still hash to the same value. Read-only, on-demand. Useful for:

• Post-restore validation
• Detecting bit-rot
• Auditing legal-evidence artefacts before a tax review or dispute

Restoring an older backup on a newer image

PicPeak’s restore path is forward-compatible: a backup taken on an older version restores cleanly onto a newer image without any manual schema work. After loading the dump, the restore service runs the same npm run migrate:safe script that the container runs on every boot. Any migrations added between the backup’s snapshot and the current image are applied inline, against the freshly-restored DB, before the restore is reported as complete.

Net effect: even if a later release shipped a migration that adds a table, after restore your install has that table (empty), the right indexes, and any seed rows the migration emits — no “you’ll need to restart the container once” footnote. The same applies to the install-from-backup trigger below: migrations land inside the restore boundary, so the moment the server prints Server running on port 3000, the schema matches the running image.

Disaster recovery (install from a backup)

For full DR after docker compose down -v, host migration, drive replacement, or moving an install between hosts. PicPeak detects a trigger file on first boot and runs the restore before the admin UI surfaces. You open the browser, log in with your original credentials, and the install is fully populated — no onboarding wizard, no temporary admin step.

Prerequisites

• Your install’s backup files must already be present in the /backup mount. They survive docker compose down -v because /backup is a bind mount, not a Docker-managed volume.
• The original manifest signing key must be in place before the first boot (see Manifest signing key). With Compose, docker compose down -v deletes the picpeak-secrets volume and with it the generated key, so set BACKUP_MANIFEST_KEY to your off-host copy before starting.
• With STORAGE_BACKEND=s3, configure the target STORAGE_S3_* bucket before the restore runs.
• The backup must contain a database dump. The restore cannot reconstruct your CRM data, customers, quotes, invoices, or admin users from a files-only backup. Confirm by inspecting any backup-manifest-*.json and checking that database.backup_file is non-null.

How the trigger works

On every container start, PicPeak’s boot sequence checks for a trigger file in the root of the /backup mount. If found and the destination database is empty, the restore runs automatically. After a successful restore the trigger file is deleted so the next boot doesn’t redo the work. On failure the trigger file is preserved — fix the input and restart the container to retry.

The trigger file is named RESTORE_ON_INSTALL (no extension) or RESTORE_ON_INSTALL.txt — either is accepted.

Auto-pick the newest backup — create an empty trigger file:

touch /path/to/backup/RESTORE_ON_INSTALL

The boot hook looks for backup-manifest-*.json (or .yaml) in /backup/manifests/ and in the manifests/ folder of every standalone restore point directly under /backup (/backup/backup-<uuid>/manifests/), and picks the one with the most recent modification time. Symlinked restore points are ignored.

Use a specific backup — write the manifest path (relative to the /backup mount root, or absolute) into the trigger file:

echo "manifests/backup-manifest-backup-20260530-190617-e9be97b3.json" \
  > /path/to/backup/RESTORE_ON_INSTALL

Use this when you need an older backup — e.g. rolling back a data corruption that happened after the most recent backup ran.

Full DR walkthrough

# 1. Snapshot the backup outside the compose directory (belt + suspenders).
SNAPSHOT=~/picpeak-snapshots/$(date +%Y%m%d-%H%M%S)
mkdir -p "$SNAPSHOT" && cp -av /path/to/picpeak/backup/. "$SNAPSHOT/"
 
# 2. Drop the trigger file (auto-pick newest shown).
touch /path/to/picpeak/backup/RESTORE_ON_INSTALL
 
# 3. Boot.
docker compose down -v
docker compose up -d
docker compose logs -f backend --tail=100

When the boot log shows Install-from-backup: restore completed successfully followed by Server running on port 3000, the install is ready. Open the admin UI and log in with your original (pre-disaster) credentials.

Safety gates

Three layers prevent accidental data loss:

1. The trigger file must exist. No auto-magic — an admin explicitly drops the file to signal intent.
2. The destination database must be empty. If it contains any events, the hook refuses to run. The fresh-install default admin is treated as throwaway and replaced by the backup’s admin row, so a single admin user does not block the restore.
3. Failed restores roll back. If anything fails, the restore is rolled back and the install stays in its fresh-install state; the trigger file is kept for a retry. (The install trigger skips the pre-restore safety backup, since there is nothing to keep.)

Recommendations

• Run a daily backup and keep enough restore points (backup_retention_count) to cover the time it might take you to notice a problem.
• Keep a copy of the manifest signing key and your SSH key/known_hosts off-host, separate from the backups.
• Store backups off-site — S3 in a different region than your primary, or rsync to a remote server.
• Test the restore at least quarterly. A backup you’ve never restored isn’t a backup.
• Monitor backup_email_on_failure notifications — the only way you’ll find out a backup is failing silently.