From ae468d6cee78363ac22f5e5183fb966f19c8218f Mon Sep 17 00:00:00 2001 From: johnnyq Date: Fri, 31 Jul 2026 16:18:20 -0400 Subject: [PATCH] Encrypted backups with types, scheduling and CLI restore Backups are now AES-256 encrypted zips in three types (full, database only, master key), catalogued in a new backups table, built by cron rather than the web request, and kept under uploads/backups with retention in the nightly job. The encryption key is one value per install held in config.php, never in the database and never in the file name. Restore is shared by the setup wizard and the new scripts/restore_cli.php, which is the only path without an upload size limit. It verifies the key and unpacks the archive before dropping anything, and dumps the current database first so a failed import is rolled back. A backup dumps, zips and encrypts for minutes without issuing a query, so on a server with a short wait_timeout the connection is closed underneath it and the UPDATE marking the backup complete is what fails - long after the archive was written correctly. The connection is now held open for the job and re-established before any write that follows long file work, including the database phase of a restore. Retention recovers rows a dropped connection left behind: still Running after six hours becomes Complete if the archive is on disk, Failed if it is not. cron.php's own failure path is hardened to match. It recorded job failures through the same connection the failing job had just killed, so an uncaught exception ended the dispatch and no trace of the original error survived. Failures now also echo to stdout, so cron mails something useful when the database is unreachable. Security: the setup wizard's restore step is now closed on any install that has users, whatever config.php says. $config_enable_setup defaulted to enabled when the flag was absent, and the flag is only written at the end of a successful install, so an install abandoned partway left an unauthenticated endpoint that would drop every table, import an attacker-supplied archive, and overwrite uploads/ including the .htaccess that stops PHP running there. Affects 26.07 and earlier. Restoring over a live install is now CLI only. --- .gitignore | 3 + CHANGELOG.md | 17 + CONTRIBUTING.md | 21 +- admin/backup.php | 282 +++++- admin/backup_download.php | 61 ++ admin/database_updates/2.6.4.php | 38 + admin/post/backup.php | 370 ++------ cron/backup.php | 70 ++ cron/cron.php | 34 +- cron/nightly_tasks.php | 5 + db.sql | 30 +- functions.php | 1 + functions/backup.php | 1418 ++++++++++++++++++++++++++++++ includes/cron_jobs.php | 14 + scripts/restore_cli.php | 136 +++ setup/index.php | 355 +++----- uploads/backups/.htaccess | 3 + uploads/backups/index.php | 1 + 18 files changed, 2313 insertions(+), 546 deletions(-) create mode 100644 admin/backup_download.php create mode 100644 admin/database_updates/2.6.4.php create mode 100644 cron/backup.php create mode 100644 functions/backup.php create mode 100644 scripts/restore_cli.php create mode 100644 uploads/backups/.htaccess create mode 100644 uploads/backups/index.php diff --git a/.gitignore b/.gitignore index 74d875e4..1d73a564 100644 --- a/.gitignore +++ b/.gitignore @@ -20,6 +20,9 @@ uploads/users/* !uploads/users/index.php uploads/tmp/* !uploads/tmp/index.php +uploads/backups/* +!uploads/backups/index.php +!uploads/backups/.htaccess uploads/tickets/* !uploads/tickets/index.php uploads/ticket_templates/* diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a941c8c..69a6d89a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,23 @@ This file documents all notable changes made to ITFlow. ## [26.08] + +### Backups + +Backups are now encrypted, catalogued, schedulable, and restorable from the command line. + +- **Three types** — Full (database + uploads), Database Only, and Master Key. Every archive is an AES-256 encrypted zip. +- **One encryption key per install**, generated on first use and stored in `config.php` — never in the database and never in the file name. It is shown in Settings > Backup. **Write it down: without it a backup cannot be restored.** Open archives with 7-Zip, WinZip, PeaZip or Keka — `unzip`, Windows Explorer and the macOS Archive Utility do not support AES. +- **Backups are built by cron, not by your browser.** The button queues the work and the dispatcher picks it up within the minute, then notifies you. A dump of a real install takes longer than a web request is allowed to live, which is why the old Download Backup button timed out on large instances. +- **Scheduled backups** are a new `backup` cron job, off by default. Turn it on in Settings > Cron. Retention (by age and by count) runs in the nightly job and never deletes the newest backup. +- **Archives are stored outside the web-served path** under `uploads/backups/` with a deny-all rule, and downloaded through an admin-only handler. Set `$config_backup_path` in `config.php` to keep them off the web root entirely. +- **Restore from the command line** with `php scripts/restore_cli.php --file=/path/to/backup.zip`. This is the only restore path with no size limit — the setup wizard's restore is capped by PHP's upload limits, and a full backup is usually larger. Use `--inspect` to check an archive without changing anything. +- **Restores validate before they destroy.** The key is checked and the archive unpacked before any table is dropped, and the current database is dumped first and put back automatically if the import fails. + +### Security + +- **The setup wizard's restore step is now closed on any install that has users**, whatever `config.php` says. Previously `$config_enable_setup` defaulted to enabled when the flag was missing from `config.php` — and the flag is only written at the very end of a successful install, so an install abandoned partway (or one where that final write failed) left an unauthenticated endpoint that would drop every table, import an attacker-supplied archive, and overwrite `uploads/` including the `.htaccess` that stops PHP executing there. Restoring over a live install is now done from the command line. This affects 26.07 and earlier. +- A restore no longer takes ITFlow's `uploads/.htaccess` from the archive — the guards are rewritten afterwards regardless of what the backup contained, so restoring a backup taken before those guards existed no longer removes them. ### Upgrading to 26.08 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7ba93dd1..32c20f19 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -32,7 +32,7 @@ There is no `composer install` or `npm install` step. All third-party libraries | `js/`, `css/` (root) | Shared front-end assets (portals also have their own). | | `libs/` | Vendored third-party libraries. Never edit these; update them wholesale. | | `setup/` | First-run installer. | -| `scripts/` | Helper/utility scripts. | +| `scripts/` | Helper/utility scripts — `setup_cli.php`, `update_cli.php`, `restore_cli.php`. CLI only; the directory denies web access. | Rule of thumb: **root-level `includes/`, `post/`, `modals/`, `js/`, `css/` are shared code; everything inside a portal directory is scoped to that portal.** @@ -100,6 +100,25 @@ Because the jobs share one PHP process, job code has three rules: 3. **Be safe to run twice in one day.** The dispatcher's lock stops overlap, but nothing stops a repeat: an admin presses Run Now after the scheduled pass, or a schedule is misconfigured. Work selected by a date match (`... = CURDATE()`) fires again on every run of that day unless something records that it happened — nightly's late fees and overdue reminders guard on the history rows they write. A job whose work cannot be made repeat-safe declares `'interval_safe' => false` in `includes/cron_jobs.php`, which locks it to the daily schedule in Settings > Cron and in the dispatcher. 4. **Set what you read.** One global scope and one set of `require_once` includes are shared across the cycle — a job's own `require_once "../config.php"` is a no-op if an earlier job already loaded it, and any variable an earlier job left behind is still there. Do not rely on the state a fresh process would have given you. +A job can also ship switched off with `'enabled' => 0` in the registry. The row is seeded disabled and stays that way until somebody turns it on in Settings > Cron. Use it for work an install should opt into rather than inherit silently from an upgrade — `backup` ships this way, because a full backup can be gigabytes a night. + +## Backups + +`functions/backup.php` is the whole engine, and all three entry points go through it: Settings > Backup, `cron/backup.php`, and `scripts/restore_cli.php`. Nothing else should dump, zip, or import a database. + +Archives are AES-256 encrypted zips. The key is one value per install, generated on first use and appended to `config.php` — **never** the database and **never** the file name. That is the point: a backup that leaks cannot be opened with anything the backup itself contains, and a URL or an access log never carries the key. The 32 random characters in the file name are an unguessable path component, nothing more. Note that `unzip`, Windows Explorer and the macOS Archive Utility cannot read AES zips; 7-Zip, WinZip, PeaZip and Keka can. + +The web tier never builds an archive in the request. It writes a `Pending` row and `cron/backup.php` does the work, because a dump of a real install outlives `request_terminate_timeout` and `set_time_limit()` does not help. Same reasoning as Run Now. + +Two rules for anything touching restore: + +1. **Validate before you destroy.** The key is checked and the archive unpacked before a single table is dropped, and the live database is dumped to a rollback file first. If the import fails the rollback goes back in. `mysqli` throws rather than returning false under PHP 8.1's default report mode, so every statement in the import path is wrapped — an uncaught throw there leaves an install with no database at all. +2. **The archive does not get to decide what our guards say.** A restore wipes `uploads/`, and an archive is allowed to contain a `.htaccess`. `backupAssertUploadsGuards()` rewrites ours afterwards unconditionally, and the backup storage directory is preserved through the wipe so a restore cannot destroy every other archive on the box. + +Retention lives in `nightly_tasks.php`, never in the backup job, so a failed backup cannot delete the archive it was supposed to replace. It never removes the newest complete backup, and an archive on disk with no row is **adopted** rather than deleted — after a restore the `backups` table is the old one, so everything made since looks unknown. + +Setup's restore step closes itself once the `users` table has rows, whatever `config.php` says. It used to default `$config_enable_setup` to `1` when the flag was absent, which fails the wrong way: the flag is only appended at the end of a successful install, so an install abandoned in between left an unauthenticated endpoint that dropped every table, imported an arbitrary archive, and rewrote `uploads/` including the `.htaccess` that stops PHP running there. + Every script in `cron/` still runs standalone (`php cron/mail_queue.php`) and still takes its own lock when it does, so anything can be run by hand for testing. --- diff --git a/admin/backup.php b/admin/backup.php index f73c9476..51244262 100644 --- a/admin/backup.php +++ b/admin/backup.php @@ -1,32 +1,287 @@ Cron - archives are built by the dispatcher, so a dead +// crontab means the buttons below queue work that never runs +$cron_is_running = $cron_last_dispatch_at !== null && (time() - strtotime($cron_last_dispatch_at)) < 300; + +$backup_job = mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT cron_job_enabled, cron_job_daily_at FROM cron_jobs WHERE cron_job_name = 'backup'")); + +$backups = mysqli_query($mysqli, "SELECT * FROM backups ORDER BY backup_created_at DESC LIMIT 100"); + +$pending_count = intval(mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT COUNT(*) AS c FROM backups WHERE backup_status IN ('Pending','Running')"))['c']); + ?> + +
+
Master encryption key
+

Shown once. Refreshing this page will not show it again.

+ +
+ + + +
+
No backup encryption key
+ ITFlow could not write a backup encryption key to config.php, so it cannot produce an encrypted backup. + Make config.php writable by the web server user and reload this page, or add a line like + $config_backup_key = '<32 random characters>'; to it yourself. +
+ + + +
+
Cron is not running
+ Backups are built by the cron dispatcher, not by your browser. Until cron is running, anything you + start here will sit in the queue. See Settings > Cron. +
+ +
+ Cron is switched off in + Settings > Notifications. +
+ +
-

Download Database

+

Create a Backup

-
-
If you are unable to back up the entire VM, you'll need to back up the files & database individually. There is no built-in restore. See the docs here.
-

Download Backup +
+ + 0) { ?> +
+ backup + queued or building. You will get a notification when ready - this page does not refresh itself. +
+ + +
+
+
+ +
Full Backup
+

The database and everything in the uploads folder. This is the one to keep.

+ + Start + +
+
+
+
+ +
Database Only
+

Just the SQL dump. Much smaller and much quicker, but no attachments or documents.

+ + Start + +
+
+
+
+ +
Master Key
+

The credential vault key. Only needed if every user password is lost - a normal restore recovers the vault on its own.

+ +
+
+
+ +
+
+ +
+
+

Backups

+
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeCreatedSizeSourceStatusAction
No backups yet.
0 ? escapeHtml(backupFormatBytes($backup['backup_size'])) : '-' ?> + + +
+ + 		+ + + + + + + + +
+
+
+
+ +
+
+

Encryption Key

+
+
+
+ + Write this down and keep it somewhere other than this server. + Every backup is encrypted with it, and without it a backup cannot be restored - not by you, + not by anyone. It is stored in config.php and never in the database, which is what stops a + stolen backup from carrying its own key. +
+ + +
+ +
+ +
+
+ + +

+ Archives are AES-256 encrypted zips. 7-Zip, WinZip, PeaZip and Keka can open them with this key. + The unzip command, Windows Explorer and the macOS Archive Utility cannot - they do not support AES. +

+
+
+ +
+
+

Scheduled Backups & Retention

+
+
+
+ + +
+
+ + +
+
+ + + 0 disables age-based deletion. +
+
+ + + The newest is never deleted. +
+
+ + +
+ +
+ +

+ + Scheduled backups run daily at + . + + Scheduled backups are switched off. + + Turn them on or change the time in Settings > Cron. +

+

+ Old backups are removed by the nightly job, never by the backup itself, so a failed nightly + cannot delete an archive that was never replaced. +

-

Backup Master Encryption Key

+

Restoring

-
-
+

Restoring replaces the database and the uploads folder with what is in the archive. It cannot be done from here, on purpose - a running install is the wrong place to be dropping its own tables from a browser.

+

From the command line - the only option that works for large backups:

+
+

From a browser, on a fresh install only, the setup wizard has a restore step at /setup. Once an install has users, that step closes itself.

+
+
+ +