diff --git a/.gitignore b/.gitignore index 74d875e4..1d73a564 100644 --- a/.gitignore +++ b/.gitignore @@ -20,6 +20,9 @@ uploads/users/* !uploads/users/index.php uploads/tmp/* !uploads/tmp/index.php +uploads/backups/* +!uploads/backups/index.php +!uploads/backups/.htaccess uploads/tickets/* !uploads/tickets/index.php uploads/ticket_templates/* diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a941c8c..69a6d89a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,23 @@ This file documents all notable changes made to ITFlow. ## [26.08] + +### Backups + +Backups are now encrypted, catalogued, schedulable, and restorable from the command line. + +- **Three types** — Full (database + uploads), Database Only, and Master Key. Every archive is an AES-256 encrypted zip. +- **One encryption key per install**, generated on first use and stored in `config.php` — never in the database and never in the file name. It is shown in Settings > Backup. **Write it down: without it a backup cannot be restored.** Open archives with 7-Zip, WinZip, PeaZip or Keka — `unzip`, Windows Explorer and the macOS Archive Utility do not support AES. +- **Backups are built by cron, not by your browser.** The button queues the work and the dispatcher picks it up within the minute, then notifies you. A dump of a real install takes longer than a web request is allowed to live, which is why the old Download Backup button timed out on large instances. +- **Scheduled backups** are a new `backup` cron job, off by default. Turn it on in Settings > Cron. Retention (by age and by count) runs in the nightly job and never deletes the newest backup. +- **Archives are stored outside the web-served path** under `uploads/backups/` with a deny-all rule, and downloaded through an admin-only handler. Set `$config_backup_path` in `config.php` to keep them off the web root entirely. +- **Restore from the command line** with `php scripts/restore_cli.php --file=/path/to/backup.zip`. This is the only restore path with no size limit — the setup wizard's restore is capped by PHP's upload limits, and a full backup is usually larger. Use `--inspect` to check an archive without changing anything. +- **Restores validate before they destroy.** The key is checked and the archive unpacked before any table is dropped, and the current database is dumped first and put back automatically if the import fails. + +### Security + +- **The setup wizard's restore step is now closed on any install that has users**, whatever `config.php` says. Previously `$config_enable_setup` defaulted to enabled when the flag was missing from `config.php` — and the flag is only written at the very end of a successful install, so an install abandoned partway (or one where that final write failed) left an unauthenticated endpoint that would drop every table, import an attacker-supplied archive, and overwrite `uploads/` including the `.htaccess` that stops PHP executing there. Restoring over a live install is now done from the command line. This affects 26.07 and earlier. +- A restore no longer takes ITFlow's `uploads/.htaccess` from the archive — the guards are rewritten afterwards regardless of what the backup contained, so restoring a backup taken before those guards existed no longer removes them. ### Upgrading to 26.08 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7ba93dd1..32c20f19 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -32,7 +32,7 @@ There is no `composer install` or `npm install` step. All third-party libraries | `js/`, `css/` (root) | Shared front-end assets (portals also have their own). | | `libs/` | Vendored third-party libraries. Never edit these; update them wholesale. | | `setup/` | First-run installer. | -| `scripts/` | Helper/utility scripts. | +| `scripts/` | Helper/utility scripts — `setup_cli.php`, `update_cli.php`, `restore_cli.php`. CLI only; the directory denies web access. | Rule of thumb: **root-level `includes/`, `post/`, `modals/`, `js/`, `css/` are shared code; everything inside a portal directory is scoped to that portal.** @@ -100,6 +100,25 @@ Because the jobs share one PHP process, job code has three rules: 3. **Be safe to run twice in one day.** The dispatcher's lock stops overlap, but nothing stops a repeat: an admin presses Run Now after the scheduled pass, or a schedule is misconfigured. Work selected by a date match (`... = CURDATE()`) fires again on every run of that day unless something records that it happened — nightly's late fees and overdue reminders guard on the history rows they write. A job whose work cannot be made repeat-safe declares `'interval_safe' => false` in `includes/cron_jobs.php`, which locks it to the daily schedule in Settings > Cron and in the dispatcher. 4. **Set what you read.** One global scope and one set of `require_once` includes are shared across the cycle — a job's own `require_once "../config.php"` is a no-op if an earlier job already loaded it, and any variable an earlier job left behind is still there. Do not rely on the state a fresh process would have given you. +A job can also ship switched off with `'enabled' => 0` in the registry. The row is seeded disabled and stays that way until somebody turns it on in Settings > Cron. Use it for work an install should opt into rather than inherit silently from an upgrade — `backup` ships this way, because a full backup can be gigabytes a night. + +## Backups + +`functions/backup.php` is the whole engine, and all three entry points go through it: Settings > Backup, `cron/backup.php`, and `scripts/restore_cli.php`. Nothing else should dump, zip, or import a database. + +Archives are AES-256 encrypted zips. The key is one value per install, generated on first use and appended to `config.php` — **never** the database and **never** the file name. That is the point: a backup that leaks cannot be opened with anything the backup itself contains, and a URL or an access log never carries the key. The 32 random characters in the file name are an unguessable path component, nothing more. Note that `unzip`, Windows Explorer and the macOS Archive Utility cannot read AES zips; 7-Zip, WinZip, PeaZip and Keka can. + +The web tier never builds an archive in the request. It writes a `Pending` row and `cron/backup.php` does the work, because a dump of a real install outlives `request_terminate_timeout` and `set_time_limit()` does not help. Same reasoning as Run Now. + +Two rules for anything touching restore: + +1. **Validate before you destroy.** The key is checked and the archive unpacked before a single table is dropped, and the live database is dumped to a rollback file first. If the import fails the rollback goes back in. `mysqli` throws rather than returning false under PHP 8.1's default report mode, so every statement in the import path is wrapped — an uncaught throw there leaves an install with no database at all. +2. **The archive does not get to decide what our guards say.** A restore wipes `uploads/`, and an archive is allowed to contain a `.htaccess`. `backupAssertUploadsGuards()` rewrites ours afterwards unconditionally, and the backup storage directory is preserved through the wipe so a restore cannot destroy every other archive on the box. + +Retention lives in `nightly_tasks.php`, never in the backup job, so a failed backup cannot delete the archive it was supposed to replace. It never removes the newest complete backup, and an archive on disk with no row is **adopted** rather than deleted — after a restore the `backups` table is the old one, so everything made since looks unknown. + +Setup's restore step closes itself once the `users` table has rows, whatever `config.php` says. It used to default `$config_enable_setup` to `1` when the flag was absent, which fails the wrong way: the flag is only appended at the end of a successful install, so an install abandoned in between left an unauthenticated endpoint that dropped every table, imported an arbitrary archive, and rewrote `uploads/` including the `.htaccess` that stops PHP running there. + Every script in `cron/` still runs standalone (`php cron/mail_queue.php`) and still takes its own lock when it does, so anything can be run by hand for testing. --- diff --git a/admin/backup.php b/admin/backup.php index f73c9476..51244262 100644 --- a/admin/backup.php +++ b/admin/backup.php @@ -1,32 +1,287 @@ Cron - archives are built by the dispatcher, so a dead +// crontab means the buttons below queue work that never runs +$cron_is_running = $cron_last_dispatch_at !== null && (time() - strtotime($cron_last_dispatch_at)) < 300; + +$backup_job = mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT cron_job_enabled, cron_job_daily_at FROM cron_jobs WHERE cron_job_name = 'backup'")); + +$backups = mysqli_query($mysqli, "SELECT * FROM backups ORDER BY backup_created_at DESC LIMIT 100"); + +$pending_count = intval(mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT COUNT(*) AS c FROM backups WHERE backup_status IN ('Pending','Running')"))['c']); + ?> + +
+
Master encryption key
+

Shown once. Refreshing this page will not show it again.

+ +
+ + + +
+
No backup encryption key
+ ITFlow could not write a backup encryption key to config.php, so it cannot produce an encrypted backup. + Make config.php writable by the web server user and reload this page, or add a line like + $config_backup_key = '<32 random characters>'; to it yourself. +
+ + + +
+
Cron is not running
+ Backups are built by the cron dispatcher, not by your browser. Until cron is running, anything you + start here will sit in the queue. See Settings > Cron. +
+ +
+ Cron is switched off in + Settings > Notifications. +
+ +
-

Download Database

+

Create a Backup

-
-
If you are unable to back up the entire VM, you'll need to back up the files & database individually. There is no built-in restore. See the docs here.
-

Download Backup
+
+ + 0) { ?> +
+ backup + queued or building. You will get a notification when ready - this page does not refresh itself. +
+ + +
+
+
+ +
Full Backup
+

The database and everything in the uploads folder. This is the one to keep.

+ + Start + +
+
+
+
+ +
Database Only
+

Just the SQL dump. Much smaller and much quicker, but no attachments or documents.

+ + Start + +
+
+
+
+ +
Master Key
+

The credential vault key. Only needed if every user password is lost - a normal restore recovers the vault on its own.

+ +
+
+
+ +
+
+ +
+
+

Backups

+
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeCreatedSizeSourceStatusAction
No backups yet.
0 ? escapeHtml(backupFormatBytes($backup['backup_size'])) : '-' ?> + + +
+ +
+ + + + + + + + +
+
+
+
+ +
+
+

Encryption Key

+
+
+
+ + Write this down and keep it somewhere other than this server. + Every backup is encrypted with it, and without it a backup cannot be restored - not by you, + not by anyone. It is stored in config.php and never in the database, which is what stops a + stolen backup from carrying its own key. +
+ + +
+ +
+ +
+
+ + +

+ Archives are AES-256 encrypted zips. 7-Zip, WinZip, PeaZip and Keka can open them with this key. + The unzip command, Windows Explorer and the macOS Archive Utility cannot - they do not support AES. +

+
+
+ +
+
+

Scheduled Backups & Retention

+
+
+
+ + +
+
+ + +
+
+ + + 0 disables age-based deletion. +
+
+ + + The newest is never deleted. +
+
+ + +
+ +
+ +

+ + Scheduled backups run daily at + . + + Scheduled backups are switched off. + + Turn them on or change the time in Settings > Cron. +

+

+ Old backups are removed by the nightly job, never by the backup itself, so a failed nightly + cannot delete an archive that was never replaced. +

-

Backup Master Encryption Key

+

Restoring

-
-
+

Restoring replaces the database and the uploads folder with what is in the archive. It cannot be done from here, on purpose - a running install is the wrong place to be dropping its own tables from a browser.

+

From the command line - the only option that works for large backups:

+
+

From a browser, on a fresh install only, the setup wizard has a restore step at /setup. Once an install has users, that step closes itself.

+
+
+ +