diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 000000000..735551921 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,20 @@ +# Editor defaults for ITFlow - see CONTRIBUTING.md ("Style") +root = true + +[*] +charset = utf-8 +end_of_line = lf +insert_final_newline = true +trim_trailing_whitespace = true +indent_style = space +indent_size = 4 + +[*.md] +trim_trailing_whitespace = false + +# Vendored - do not reformat +[libs/**] +indent_style = unset +indent_size = unset +trim_trailing_whitespace = false +insert_final_newline = false diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 000000000..2cc86d975 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,44 @@ +# ITFlow line-ending policy +# +# Everything ITFlow ships is LF in the repository and LF in the working tree. +# Contributors on Windows get LF too - this is deliberate. ITFlow is deployed +# to Linux/Apache and edited over sftp/ssh as often as it is cloned, so a +# checkout must be byte-identical everywhere. + +* text=auto eol=lf + +# Explicit for the file types we author, so nothing depends on git's guess. +*.php text eol=lf +*.js text eol=lf +*.css text eol=lf +*.html text eol=lf +*.sql text eol=lf +*.md text eol=lf +*.json text eol=lf +*.yml text eol=lf +*.xsd text eol=lf +*.svg text eol=lf +*.txt text eol=lf +*.ini text eol=lf +.htaccess text eol=lf + +# Binary assets: never touched, never diffed as text. +*.png binary +*.gif binary +*.jpg binary +*.jpeg binary +*.webp binary +*.ico binary +*.icc binary +*.woff binary +*.woff2 binary +*.ttf binary +*.eot binary +*.crt binary +*.ser binary +*.z binary + +# Vendored third-party code is preserved byte-for-byte as shipped upstream. +# Per CONTRIBUTING.md libs/ is never edited in place - it is replaced wholesale - +# so normalizing it here would create spurious diffs on the next library update. +libs/** -text diff --git a/.gitignore b/.gitignore index dd0bf48d7..f18711d1f 100644 --- a/.gitignore +++ b/.gitignore @@ -20,6 +20,10 @@ uploads/users/* !uploads/users/index.php uploads/tmp/* !uploads/tmp/index.php +!uploads/tmp/.htaccess +uploads/backups/* +!uploads/backups/index.php +!uploads/backups/.htaccess uploads/tickets/* !uploads/tickets/index.php uploads/ticket_templates/* @@ -49,3 +53,5 @@ setup/custom/* api/v1/custom/* !api/v1/custom/readme.php .zed + +*.patch diff --git a/.htaccess b/.htaccess index f38dbabcd..826afe9a4 100644 --- a/.htaccess +++ b/.htaccess @@ -1,2 +1,2 @@ # Prevent access to .git, .github, and config.php -RedirectMatch 404 ^/(\.git|\.github|config\.php) \ No newline at end of file +RedirectMatch 401 ^/(\.git|\.github|config\.php) diff --git a/CHANGELOG.md b/CHANGELOG.md index 91081e891..426ff3f7c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,166 @@ This file documents all notable changes made to ITFlow. +## [26.08] + +### Upgrading to 26.08 + +> **Read this before you start.** Done out of order this update will break your instance. The database structure changes, every API key is deleted, and the whole cron setup is replaced. + +1. **Back everything up.** Take a full VM backup or snapshot before you start. +2. **Remove every ITFlow line from your crontab** (or delete `/etc/cron.d/itflow`). The old per-minute jobs must not keep firing against a half-updated install. You put the new one in at step 5. +3. **Update the files with the normal web updater**, from Settings > Update. It will pull the new files, report the update as successful, and then error out as it drops you back into the app. That is normal — the new code is now running against the old database. +4. **Run the database update from the command line.** Just this once it cannot be done from the web interface. Run it as the user that owns the ITFlow files — the script tells you which user if you get it wrong: +```bash +sudo -u www-data php /path/to/itflow/scripts/update_cli.php --update_db +``` +It applies every pending version in order and reports each one as it goes. On an install with a lot of ticket history it can take a minute or more, so let it finish. If a step fails it stops there without advancing the recorded version, so you can fix the problem and run it again. The 500s stop as soon as it completes. + +5. **Add the new cron entry.** One line runs everything now, and the schedules are managed in ITFlow under Settings > Cron: +``` +* * * * * www-data php /path/to/itflow/cron/cron.php >/dev/null +``` +Drop the `www-data` column if this goes in a user crontab rather than `/etc/cron.d`. + +6. **Recreate your API keys.** Every existing key is deleted by this update. Issue new ones and update anything that talks to the ITFlow API. +7. **Check it took.** Open Settings > Cron — the green "Cron last checked in" banner should appear within a couple of minutes and every job should pick up a schedule. +Only this release needs the command line for the database update. Normal updates go back to running from Settings > Update as usual. + +### Breaking Changes and Notes + +- Cron: the crontab collapses to a single entry. `cron/cron.php` is now a dispatcher that runs every minute and works out which jobs in `cron/` are due, and the old nightly work has moved to `cron/nightly_tasks.php` which it runs at 03:00. +- Cron: an existing crontab keeps working — the per-minute scripts still run and still lock correctly, and a single daily `cron.php` entry still runs the daily jobs — but any job added in this or a future release only runs once the dispatcher is scheduled. +- Cron: ticket SLAs need no entry of their own. `cron/ticket_sla.php` is in the dispatcher's job list and runs every minute once the new entry is in place. Without it SLA targets are still worked out and displayed, but warnings and breaches never fire. +- Backups: write down your backup encryption key. It is generated on first use and stored in `config.php`, never in the database, and without it a backup cannot be restored. +- API: every existing key is deleted by this update and must be recreated. Keys are now owned by a user and inherit that user's role, module and client permissions instead of carrying their own client scope, and existing keys cannot be safely mapped to a user. +- API: credential decrypt passwords are now read from the request body instead of the query string. Any caller passing that value in the URL needs updating. +- Client access permissions now support deny rules as well as allow, and the permissions UI will not load until the database update has run. +- Business hours are new and default to Monday to Friday, 09:00 to 17:00 in your configured timezone. SLA targets are measured against them, so set them before assigning SLAs. +- The `plugins` directory is now `libs`. Anything pointing at `plugins/` directly — custom scripts, reverse proxy rules, web server config — needs updating. +- Several pages dropped the `_details` suffix and moved to consistent singular and plural filenames, so old bookmarks and external links will 404. +- Credential passwords moved from `varbinary` to `varchar(500)` and now have a length guard. Existing credentials are migrated by the database update. +### New Features & Updates + +- Backups are now encrypted, catalogued, schedulable and restorable from the command line. Three types — Full (database and uploads), Database Only, and Master Key — and every archive is an AES-256 encrypted zip. Open them with 7-Zip, WinZip, PeaZip or Keka; `unzip`, Windows Explorer and the macOS Archive Utility do not support AES. +- Backups: one encryption key per install, generated on first use and stored in `config.php`, never in the database and never in the file name. It is shown in Maintenance > Backup. +- Backups are built by cron rather than by your browser. The button queues the work and the dispatcher picks it up within the minute, then notifies you — a dump of a real install takes longer than a web request is allowed to live, which is why the old Download Backup button timed out on large instances. +- Backups: scheduled backups are a new `backup` cron job, off by default, turned on in Maintenance > Cron. Retention by age and by count runs in the nightly job and never deletes the newest backup. +- Backups are stored outside the web-served path under `uploads/backups/` with a deny-all rule and downloaded through an admin-only handler. Set `$config_backup_path` in `config.php` to keep them off the web root entirely. +- Backups: restore from the command line with `php scripts/restore_cli.php --file=/path/to/backup.zip`. This is the only restore path with no size limit — the setup wizard's restore is capped by PHP's upload limits and a full backup is usually larger. `--inspect` checks an archive without changing anything. +- Backups: restores validate before they destroy. The key is checked and the archive unpacked before any table is dropped, and the current database is dumped first and put back automatically if the import fails. +- Cron: one entry instead of five, and a page to manage it. Jobs are tracked in a new `cron_jobs` table, so a job whose slot was missed runs at the next opportunity rather than waiting a day, and each job locks for its own run so a slow mailbox or a long nightly pass no longer holds anything else up. +- Cron: new Settings > Cron page listing every job with its schedule, last run, duration, outcome and next due time. Jobs can be disabled, rescheduled, or run on demand — Run Now hands the job to the next dispatch so it starts within a minute and still runs on the command line. The last error is kept until dismissed rather than vanishing behind the next success, and the page says plainly when the crontab entry itself is missing. +- Cron: the nightly run is safe to repeat. Late fees, overdue invoice reminders and autopay retries now apply at most once per invoice per day, so a Run Now after the scheduled pass no longer stacks fees or re-emails clients. Nightly Tasks only accepts the daily schedule. +- Cron: the master enable switch moved out of Notifications and into the Cron settings where it belongs, and the unused overdue invoice setting has been removed. +- Ticket: major UI overhaul of the ticket list, the kanban board and the ticket detail page. +- Ticket: redesigned the task bar on ticket details and removed the redundant task count. +- Ticket SLAs, optional throughout. An SLA sets a response target and an optional resolution target, assigned per client and priority with a global default and an explicit "no SLA" override. Targets are measured against your business hours. Tickets show time remaining and turn yellow at a configurable warning threshold and red on breach, on both the ticket list and the kanban board, and can be filtered by SLA state. Nominated statuses pause the resolution clock for "waiting on customer", preserving the remaining budget. Two new reports, SLA Summary and SLA by Client. With no assignments defined nothing behaves any differently. +- Ticket: added an Urgent priority. +- Ticket: agents can attach files to tickets from inside the app, both when raising a ticket and on a reply, and attachments are emailed to the contact through the mail queue. A 10 MB ceiling applies per message; anything that does not fit stays on the ticket to download. +- Ticket: tasks can be added and edited inline in the add ticket and add recurring ticket modals. +- Ticket: the older add ticket modal has been retired, there is one add ticket modal now. +- Ticket: watchers and attachments have moved into the assignment section of the add ticket modal. +- Ticket: recurring tickets can be assigned a ticket template. Picking one fills in the subject and details and stamps the template's task list onto every ticket the schedule raises, from the nightly run and a forced run alike. The recurring ticket list shows which schedules carry a template and how many tasks it adds. +- Ticket: recurring tickets now own their task list. The template fills it in when picked but it can then be edited per schedule, and it is those edits the run reads. Existing schedules are backfilled from their template by the database update. +- API: added ticket reply endpoints for creating and reading replies. +- API: added an invoice_items endpoint for adding line items to an invoice. +- Calendar: calendars can be published as a read-only iCalendar (ICS) subscription feed and read by Google Calendar, Nextcloud, Apple Calendar, Thunderbird or anything else that takes a feed URL. The link carries a secret key and needs no login, can be regenerated or revoked at any time, and a busy only option publishes time blocks without titles, descriptions or locations. Refresh timing belongs to the subscribing client — Google refreshes on its own schedule and cannot be forced, and Nextcloud defaults to weekly and refuses feed URLs resolving to private IPs. +- Calendar: events can be marked all day, and the date and time are now separate fields. Previously all day was inferred from a midnight start, which made a genuine midnight appointment indistinguishable from an all-day event. Existing events are backfilled by the database update using the old rule, so nothing changes appearance. +- Calendar: repeating events now work. The Repeat field was present but disabled and the stored value was never drawn. It is now selectable daily, weekly, monthly or yearly, and monthly and yearly series skip dates that do not exist in a period rather than sliding into the next month. Recurrence is series-wide — editing any occurrence edits the whole series, and individual occurrences cannot yet be moved or cancelled. Repeating events are marked with an icon and a hover note, and the delete action reads Delete series and asks for confirmation. +- Calendar: clicking empty space creates an event there. Clicking a day or a time slot, or dragging across several, opens the New Event modal with the start and end already filled in and the All day switch set to match. A range dragged out or lengthened by hand is no longer overwritten by the end-time-follows-start behaviour. +- Exports: every export modal now has a Filter tab and a Selectable Columns tab with sensible defaults, and can export to PDF as well as CSV. +- Combined Payments and Revenues into a single Income page with CSV export. Revenue not tied to an invoice is still added there and payments are still added from invoices. The standalone Payments and Revenues pages are gone. +- Income: added a Category column and filter, carried through to the exports. +- Income: added bulk actions for account, payment method and category. +- New Transactions page — a per-account ledger of transfers, revenues, payments and expenses with filtering by type, category, client, payment method, amount range and date, a running balance, summary cards, account balances in the account picker, and CSV export. +- Products: added a basic product import via CSV. +- Added user based RBAC for API keys, so a key runs as a user and inherits that user's permissions. +- Added deny rules to client access permissions, so access can be granted broadly and revoked for specific clients. +- New secure file download handler for files and ticket attachments, with client and contact permission isolation on the client portal. +- Invoices: clicking the Paid or Partial status badge opens a read-only breakdown of the payments recorded against that invoice. +- Assets: multiple notes per asset, same as contact notes, with categorized note types (Maintenance, Repair, Configuration, Upgrade, Inspection, Note). +- Reworked the Maintenance > Update page, and fixed the branch handling on it. +- Categories and tag types moved from a top button nav to a left side nav. +- Dashboard: added expiring asset warranties and licenses, along with an "Expiring in" filter for assets, licenses, domains and certificates. +- Added bulk and single refresh actions for domains and certificates. +- Stripe gateway fees now come from the actual Stripe balance transaction rather than a static percentage and flat fee configured in ITFlow, with a nightly pass to backfill fees that were not available at payment time. The static fee fields are gone from payment provider settings. +- Database updates are now split into per-version files under `admin/database_updates/`, the latest version is derived from the directory listing, and one run applies everything pending. Migration history before 2.0.0 has been pruned. +- Reorganized the main, client, admin and reports side navigation menus. +- Mail settings tabs are now URL addressable and stay on the active tab after saving. +- Removed the legacy vendor contacts feature. +- `dig` and `whois` are no longer required, domain lookups use native DNS and RDAP. +- Tightened the `.htaccess` rules, and added one for `uploads/tmp`. +- Bumped the minimum supported PHP version. +### Security + +- Rate limited 2FA code attempts and narrowed the TOTP acceptance window. +- Rotate the session ID on login to prevent session fixation. +- Stopped parallel login attempts from bypassing the login rate limits. +- Tightened validation on the recurring invoice frequency used by the billing run. +- Admin UI modals are now gated to admins. Previously any logged-in user could open them directly and read stored payment provider and AI provider API keys — rotate those keys when you update, as there is no record of who may have viewed them. +- Global search returned credentials to users without credential module access. It is now gated like every other credential surface. +- Credential password reveals are now written to the audit log, on both the reveal endpoint and the TOTP code. +- Swept module and client permission enforcement across modals and ajax endpoints to match the post handlers, closing a number of cases where a user restricted to certain clients could read another client's records by ID. +- Products: the CSV export now requires sales module read access. +- Neutralized CSV formula injection in generated exports. +- Fixed weak random number generation in TOTP secret generation. +- Shared item views are now claimed atomically so the view limit cannot be exceeded by simultaneous requests, and guest audit IPs are logged. +- Hardened CSRF handling and session cookies, and set `SameSite=Lax` on the session cookie. +- Hardened file upload handling to use random storage names. +- Client Portal: contacts can no longer edit their own contact record. +- The setup wizard's restore step is now closed on any install that has users, whatever `config.php` says. Previously, if `$config_enable_setup` was missing from `config.php` — a state an install can be left in when setup does not reach its final step — the restore step stayed reachable. Restoring over a live install is now done from the command line. +- A restore no longer takes ITFlow's `uploads/.htaccess` from the archive. The guards are rewritten afterwards regardless of what the backup contained, so restoring a backup taken before those guards existed no longer removes them. +- Tightened the directory guards under `uploads`. +### Bug Fixes + +- Deleting a payment now correctly recalculates and sets the invoice status. +- Fixed contact notes, and several broken modal links in contacts, assets and file linking. +- Client Portal: fixed adding saved payment methods and cards following a Stripe API change. +- Fixed sending invoices and quotes over OAUTH2, which was reading the SMTP host instead of the SMTP provider — the host is not filled in when OAUTH2 is selected. +- Mail Parser: correctly work out whether the ITFlow folder belongs under the `INBOX` namespace or the root directory, fixing folder creation on cPanel Dovecot Maildir++ setups. +- Fixed possible duplicate emails caused by a race condition in the mail queue. +- Added a shared lock guard across every cron entry point, scoped per script and per install, replacing the mail queue's non-atomic lock file. Rows left in a sending state by a run that died are now recovered. +- Prevented duplicate Stripe payment bookings and overlapping cron runs. +- Mail bodies are cleared after successful delivery. +- Reworked `getFieldById` to stop escaping its return value, and reworked every caller to escape at the point of use — it was causing double escaping in a lot of places. +- Added missing `maxlength` attributes to forms backed by length-limited columns, so an overlong value no longer throws a 500. +- Fixed undefined variables in the audit log and flash messages for expenses, assets, contacts and several other handlers, which were logging blanks in place of the record name. +- Fixed the spelling of the expense description in audit logging. +- Fixed autofill on invoices, quotes and recurring invoices, where the tax field was not updating and a dash was being placed in front of the product. +- Fixed gaps in ticket history. +- Recurring Expense: fixed editing not keeping the client. +- Fixed client name truncation in the side navigation being applied after escaping. +- Side navigation counts are only shown to users with permission to see them. +- Invoice statistics now only reflect clients the user has permission to see. +- The agent category handler no longer drops the category description. +- Expenses: allowed negative amounts, and the current date is now prefilled. +- Certificates can now be searched by description. +- Fixed cents calculation rounding. +- Fixed guest view credential TOTP display, and removed the legacy OTP code path. +- Gated the SLA option in ticket details, which was gated everywhere else. +- Deleting a ticket template task or a payment provider recorded the wrong name in the audit log and the confirmation message, reading an unrelated record's id in place of the name. +- Bulk-creating tickets from a template against multiple assets only added the template's tasks to the first ticket, and dropped each task's completion estimate. +- Deleting a ticket template now unlinks it from any recurring ticket that referenced it, instead of leaving the schedule pointing at a template that no longer exists. +- Fixed the asset section in the recurring ticket modal when opened outside a client, and project selection when raising a ticket. +- Tickets raised by the nightly recurring schedule were created without a guest URL key, so the "View ticket" link in reply and task approval emails could not be opened. Cron now generates a key like every other path that raises a ticket, and existing tickets missing one are backfilled by the database update. +- Calendar: fixed the last day of a multi-day all-day event not being drawn or published to subscribed feeds. `event_end` holds the last day the event covers, which is what the event modal asks for, but FullCalendar and iCalendar both treat an all-day end as exclusive. +- Fixed `confirm-link` doing nothing inside an ajax modal, where the handler was only bound to links present at page load. +### Developer Updates + +- Line endings normalized to LF across the codebase, with `.gitattributes` and `.editorconfig` added. Vendored code under `libs/` is marked so it stays byte identical to upstream. +- Converted to the short echo tag `/dev/null +``` + +`cron/cron.php` is a dispatcher. It wakes every minute, works out which scripts in `cron/` are due, and requires them into its own process. Adding a job is a new script in `cron/` plus an entry in `includes/cron_jobs.php`. The crontab never changes again. + +That registry is the only thing that decides **which** scripts can run, and the schedule in it is only a default: it seeds the job's `cron_jobs` row the first time the dispatcher meets the job, and from then on the row is what runs, because Maintenance > Cron writes to it. The database therefore holds **when and whether**, never **what** — a row naming a script that is not in the registry is ignored, so nothing that reaches the database can point the dispatcher at an arbitrary file. Keep it that way. + +Run Now in the admin UI does not execute anything in the web request: these scripts are CLI-only and some take minutes, so the button sets `cron_job_run_now` and the next dispatch picks it up, through the same lock and claim as a scheduled run. + +Due-ness is recorded in the `cron_jobs` table rather than matched against the clock, so a job whose minute was missed — machine down, previous run still going — runs at the next opportunity instead of being skipped for the day. A job is claimed *before* it runs, not after: a run that dies half way through is not repeated, which matters because `nightly_tasks.php` generates invoices and charges cards. Each job is also locked individually for the length of its own run (`cron/includes/cron_lock.php`), so a long or hung job holds up only itself — the next minute's dispatch picks up everything else in a second process. + +Because the jobs share one PHP process, job code has three rules: + +1. **Never `exit()` or `die()`.** It ends the whole cycle and every job after it. Use `cronJobStop($message, $exit_code)` instead: it exits when the script was run directly and unwinds back to the dispatcher when it wasn't, so both paths behave as they always have. +2. **Never declare a function or class another job might declare.** Two jobs each declaring the same helper is a fatal `Cannot redeclare` the moment they share a process. Shared helpers belong in `functions/`. +3. **Be safe to run twice in one day.** The dispatcher's lock stops overlap, but nothing stops a repeat: an admin presses Run Now after the scheduled pass, or a schedule is misconfigured. Work selected by a date match (`... = CURDATE()`) fires again on every run of that day unless something records that it happened — nightly's late fees and overdue reminders guard on the history rows they write. A job whose work cannot be made repeat-safe declares `'interval_safe' => false` in `includes/cron_jobs.php`, which locks it to the daily schedule in Maintenance > Cron and in the dispatcher. +4. **Set what you read.** One global scope and one set of `require_once` includes are shared across the cycle — a job's own `require_once "../config.php"` is a no-op if an earlier job already loaded it, and any variable an earlier job left behind is still there. Do not rely on the state a fresh process would have given you. + +A job can also ship switched off with `'enabled' => 0` in the registry. The row is seeded disabled and stays that way until somebody turns it on in Maintenance > Cron. Use it for work an install should opt into rather than inherit silently from an upgrade — `backup` ships this way, because a full backup can be gigabytes a night. + +### The master switch + +`config_enable_cron` is a second, coarser switch that sits above the per-job ones. It is **not** enforced by the dispatcher — every job checks it in its own header and stops itself with `cronJobStop()`. A new job has to make that check too; one that skips it keeps running on an install that believes cron is off, which is exactly the trap `ticket_email_parser` and `ticket_sla` sat in until 26.08. + +Be precise about what it does, because it is easy to oversell. It is **not** a guard on restored data: a full backup dumps the `settings` table, so a restored copy comes back with `config_enable_cron = 1` alongside every enabled `cron_jobs` row, exactly as production had them. What it gives you is one reversible bit — the fastest way to stop an install acting on live data once you have noticed, and the only way to stop everything without editing seven rows. + +That last part is the reason it is not redundant with `cron_job_enabled`. Turning the switch off and back on returns you to exactly the configuration you had. Sweeping all seven rows off and back on does not: `backup` ships `'enabled' => 0`, so the sweep quietly turns on nightly backups nobody asked for, along with anything else that was deliberately disabled. + +It defaults to `0`. That is a weaker guard than it looks — an install with no crontab entry runs nothing whatever the switch says, so the entry is the real gate — but it does mean adding the crontab line to a half-configured install is not enough on its own to start emailing. Both setup paths name the step on the way out. + +## Backups + +`functions/backup.php` is the whole engine, and all three entry points go through it: Maintenance > Backup, `cron/backup.php`, and `scripts/restore_cli.php`. Nothing else should dump, zip, or import a database. + +Archives are AES-256 encrypted zips. The key is one value per install, generated on first use and appended to `config.php` — **never** the database and **never** the file name. That is the point: a backup that leaks cannot be opened with anything the backup itself contains, and a URL or an access log never carries the key. The 32 random characters in the file name are an unguessable path component, nothing more. Note that `unzip`, Windows Explorer and the macOS Archive Utility cannot read AES zips; 7-Zip, WinZip, PeaZip and Keka can. + +The web tier never builds an archive in the request. It writes a `Pending` row and `cron/backup.php` does the work, because a dump of a real install outlives `request_terminate_timeout` and `set_time_limit()` does not help. Same reasoning as Run Now. + +Two rules for anything touching restore: + +1. **Validate before you destroy.** The key is checked and the archive unpacked before a single table is dropped, and the live database is dumped to a rollback file first. If the import fails the rollback goes back in. `mysqli` throws rather than returning false under PHP 8.1's default report mode, so every statement in the import path is wrapped — an uncaught throw there leaves an install with no database at all. +2. **The archive does not get to decide what our guards say.** A restore wipes `uploads/`, and an archive is allowed to contain a `.htaccess`. `backupAssertUploadsGuards()` rewrites ours afterwards unconditionally, and the backup storage directory is preserved through the wipe so a restore cannot destroy every other archive on the box. + +Retention lives in `nightly_tasks.php`, never in the backup job, so a failed backup cannot delete the archive it was supposed to replace. It never removes the newest complete backup, and an archive on disk with no row is **adopted** rather than deleted — after a restore the `backups` table is the old one, so everything made since looks unknown. + +Setup's restore step closes itself once the `users` table has rows, whatever `config.php` says. It used to default `$config_enable_setup` to `1` when the flag was absent, which fails the wrong way: the flag is only appended at the end of a successful install, so an install abandoned in between left an unauthenticated endpoint that dropped every table, imported an arbitrary archive, and rewrote `uploads/` including the `.htaccess` that stops PHP running there. + +Every script in `cron/` still runs standalone (`php cron/mail_queue.php`) and still takes its own lock when it does, so anything can be run by hand for testing. + +--- + +## Security rules (non-negotiable) + +ITFlow does not use prepared statements or an ORM; queries are built as strings. That works **only** if every value is neutralized before interpolation. The rules: + +### 1. Every value interpolated into SQL is cast or sanitized. No exceptions. + +- **Integers** (IDs, flags, counts): `intval($_POST['ticket_id'])`. Interpolate unquoted. +- **Strings**: `escapeSql($_POST['subject'])`. This normalizes encoding to UTF-8, then runs `strip_tags()`, `trim()`, and `mysqli_real_escape_string()`. Because it relies on SQL escaping, the value **must be placed inside quotes in the query** (`'$subject'`). An escaped string interpolated without quotes is still injectable. +- **Values read back from the database** get the same treatment before reuse in another query (you will see `escapeSql($row['ticket_prefix'])` throughout — this is why). +If you write a query and even one variable in it skipped these, that is a SQL injection. This is the single most common review rejection. + +**Fetch helpers return raw values — you escape them.** `getFieldById()` and `getTicketStatusName()` hand back exactly what is in the column. Escaping is the call site's job, the same as any other row you read: + +```php +$client_name = escapeSql(getFieldById('clients', $client_id, 'client_name')); // into a query +$client_name = escapeHtml(getFieldById('clients', $client_id, 'client_name')); // into a page +$client_id = intval(getFieldById('tickets', $ticket_id, 'ticket_client_id')); // an id +``` + +Both helpers used to escape internally, via an `$escape_method` argument. It went badly: most call sites wrapped them in `escapeSql()` anyway and got a double-escaped value, so a client named `O'Brien` came back as `O\'Brien` and the backslash reached export filenames, PDF headings, flash messages and — on the user restore path — the database itself. A value that is already safe cannot be made safer, only wrong. + +### 2. Every state-changing action validates CSRF. + +`validateCSRFToken()` is the first line of every action block. It takes no argument — it reads `csrf_token` from `$_POST`, then `$_GET`, itself, so the same call covers form posts and link-style actions. (The signature still accepts an explicit token for callers that need one, but no call site in the tree passes one; use the bare form.) Forms and action links must include the token; copy how existing modals do it. + +### 3. Every action enforces permissions. + +`enforceUserPermission('module_x', level)` where level is `1` = read, `2` = write, `3` = full/delete. Current modules: `module_client`, `module_support`, `module_sales`, `module_financial`, `module_credential`, `module_reporting`. Read pages enforce level 1; create/edit enforce 2; destructive actions enforce 3. CSV/PDF exports are reads — gate them with the bare one-argument form, e.g. `enforceUserPermission('module_sales')`. + +Two portals are gated differently, which is why their handlers look like they are missing the call: + +- **Admin.** `admin/post.php` only loads anything in `admin/post/` when `$session_is_admin` is set, so admin handlers inherit the gate from the dispatcher and do not call `enforceUserPermission()` themselves. +- **Client portal.** `client/post.php` is a single file of action blocks rather than a dispatcher, and gates on the contact's own capabilities with `enforceContactCan('accounting'|'contacts'|'itdoc')`. + +Everywhere else — anything under `agent/post/` — the call belongs in the block. + +### 4. Client scoping is enforced, not assumed. + +After loading a record, call `enforceClientAccess()` (optionally with the record's client ID) so technicians restricted to specific clients cannot touch other clients' data by editing an ID in the URL. Look at how `resolve_ticket` does it — including the "skip if the record has no client" case. + +### 5. Escape on output. + +Anything rendered into HTML goes through `escapeHtml()`. `escapeSql()` on the way in is **not** output escaping — data can enter the DB through other paths (API, email parser, older versions). + +In practice the escaping happens **where the row is read, not where it is echoed**. A page or modal fetches its row and assigns each field through `escapeHtml()` once, then echoes the resulting variable raw: + +```php +$row = mysqli_fetch_assoc($sql); +$asset_id = intval($row['asset_id']); // ints: intval, not escapeHtml +$asset_name = escapeHtml($row['asset_name']); +... + +``` + +Follow that pattern. Escaping at the echo instead would double-escape a value that is already safe, and mixing the two is how fields get missed. If you introduce a view variable that does not come from a row, escape it at assignment so the rule still holds at the top of the file. + +A value that arrives through a fetch helper rather than a row read follows the same rule — `escapeHtml(getFieldById(...))` at the assignment, not at the echo. See rule 1. + +Rich-text fields (TinyMCE content) are the exception and have their own handling; follow the existing pattern for the specific field rather than inventing one. + +### 6. No shell-outs. No `eval`. + +The project has deliberately moved off `shell_exec`/`exec` in favor of native PHP — `dns_get_record()` instead of `dig`, RDAP instead of `whois`, and so on. **Do not add new shell execution or `eval`.** PRs introducing either will be declined. + +A handful of legacy call sites survive, all of them wrapping `git` or `which` in the self-update and diagnostics paths: `admin/debug.php`, `admin/update.php`, `admin/post/update.php`, `admin/post/backup.php`, `cron/cron.php`, `functions/app.php`, `scripts/update_cli.php`, `setup/index.php`. They are on the list to be replaced with direct `.git` file reads; treat them as debt, not as precedent. + +### 7. Report vulnerabilities privately. + +Per [SECURITY.md](SECURITY.md) — never in a public issue. + +--- + +## Conventions + +**Database naming.** Every column is prefixed with the singular name of the entity it belongs to: `tickets.ticket_id`, `tickets.ticket_subject`, `clients.client_name`. This makes JOIN results unambiguous and is why queries can `SELECT *` across joins safely. New tables must follow it. + +The prefix is the entity name, which is usually but not always the singular of the table name. Where a table is named for its container rather than its row, the prefix follows the row: `calendar_events` → `event_*`, `asset_interfaces` → `interface_*`, `invoice_items` / `quote_items` → `item_*`, `rack_units` → `unit_*`, `user_roles` → `role_*`, `product_stock` → `stock_*`. Pick the prefix your columns will read best as and use it for every column in the table. + +Two standing exceptions: junction tables (`client_tags`, `service_assets`, …) carry the two parent FK names unprefixed, and `settings` / `user_settings` use `config_*` / `user_config_*`. + +**Schema changes require two edits in one PR:** + +1. `db.sql` — so fresh installs get the new schema. +2. `admin/database_updates/.php` — a new file named for the version it upgrades **to**, containing only the queries that apply the change. Migrations are sequential and rolling-release; never edit a historical file. + +That is the whole job. `LATEST_DATABASE_VERSION` is derived from the highest-numbered filename in `admin/database_updates/`, and the runner (`admin/database_updates.php`) steps `config_current_database_version` after each file succeeds — so there is no constant to bump and no version-bump query to write. Each migration file needs the standard `defined('FROM_DB_UPDATER') || die(...)` guard at the top; copy an existing file's header. + +A single update run applies every pending migration in order, stopping at the first failure with the version left at the last file that completed, so a re-run resumes at the one that broke. +**After acting, log and notify.** State changes call `logAudit($type, $action, $description, $client_id, $entity_id)` for the audit trail. User-facing events may also call `appNotify()`. Fire `triggerCustomAction()` where a site might reasonably want a hook. Then call `flashAlert($message, $type)` and `redirect()` (defaults to the referer) rather than setting session keys or `header()` manually. + +**Function names (post-rename).** Helpers were renamed for clarity in 2026; the old names **no longer exist** — code calling them fatals. If you're rebasing an old PR or following an old tutorial, translate: `sanitizeInput` → `escapeSql`, `nullable_htmlentities` → `escapeHtml`, `logAction` → `logAudit`, `flash_alert` → `flashAlert`, `customAction` → `triggerCustomAction`, `encryptLoginEntry`/`decryptLoginEntry` → `encryptCredentialEntry`/`decryptCredentialEntry`, `strtoAZaz09` → `toAlphanumeric`, `fetchUpdates` → `checkForUpdates`, `sanitize_url` → `escapeUrl`. + +**Helpers that fetch data return it raw.** If you add a `getXById()`-style helper, return the column value untouched and let callers escape it (security rule 1). Validating what the helper interpolates into its *own* query — table and column names, the id — is still the helper's job; that is query construction, not output escaping, and the two are not the same thing. + +**Bulk vs. single actions.** If you change the behavior of a single action (e.g. resolving a ticket), check whether a `bulk_*` counterpart exists and update it too. They are currently parallel implementations and drift between them is a known bug source. + +**UI.** Bootstrap 4 / AdminLTE, modals per-module under `/modals//`, DataTables for lists, monospace styling for technical data (IPs, serials, keys) and proportional for human text. Match the page you're standing in. + +**Modals post to the portal you are standing in, not the one they live in.** Modal forms use `action="post.php"`, which the browser resolves against the *page* URL, not the modal's own path. A modal under `admin/modals/` that an agent page opens by relative path therefore submits to `agent/post.php` and is handled by `agent/post/`, not `admin/post/`. If you reuse a modal across portals, every portal that can open it needs a handler that accepts the same field set — otherwise fields are silently dropped on one side. + +**Style.** Procedural PHP, 4-space indentation, LF line endings, code and comments in English. Match the surrounding code rather than importing a personal style. Don't reformat code you aren't changing — it buries the real diff. + +Line endings and indentation are enforced by `.gitattributes` and `.editorconfig` at the repo root, so an editor that respects EditorConfig needs no configuration. `.gitattributes` marks `libs/` as `-text`: vendored code is preserved byte-for-byte as shipped upstream and must never be normalized, or the next wholesale library update turns into an unreviewable diff. + +--- + +## Pull requests + +- **Small, focused diffs.** One feature or one fix per PR. Never mix relocation/reformatting with logic changes — split them into separate commits or PRs so each is reviewable on its own. +- Describe **what** and **why**, and note any schema changes prominently. +- CI runs PHP lint and db.sql lint; SonarCloud scans for security issues. Green checks are required but not sufficient — the conventions above are checked by human review. +- Test your change against a real install: fresh setup from `db.sql` **and** an upgrade via `database_updates.php` if you touched schema. +- For anything larger than a bug fix, **open an issue first** and discuss the approach. ITFlow's roadmap favors incremental modernization of the existing PHP codebase; large rewrites, framework introductions, and new runtime dependencies are out of scope. +## Getting help + +Open a GitHub issue using the templates, or ask in the community forum linked from the README. When in doubt about a convention, find the closest existing example in the codebase and follow it — consistency beats novelty here. \ No newline at end of file diff --git a/SECURITY.md b/SECURITY.md index aea5b2479..3479ef2ed 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -13,7 +13,7 @@ We operate a rolling release model. Any bug fixes will be released into latest v | Version | Supported | |---------| ------------------ | -| 25.12 | :white_check_mark: | +| 26.08 | :white_check_mark: | ## Reporting a Vulnerability via GitHub Security Advisories diff --git a/admin/ai_model.php b/admin/ai_models.php similarity index 76% rename from admin/ai_model.php rename to admin/ai_models.php index 2eab6bdad..e30dc6db3 100644 --- a/admin/ai_model.php +++ b/admin/ai_models.php @@ -17,7 +17,7 @@ $num_rows = mysqli_num_rows($sql); Admin @@ -35,17 +35,17 @@ $num_rows = mysqli_num_rows($sql); "> - + Model - + Provider - + Use Case @@ -60,23 +60,23 @@ $num_rows = mysqli_num_rows($sql); while ($row = mysqli_fetch_assoc($sql)) { $provider_id = intval($row['ai_provider_id']); - $provider_name = nullable_htmlentities($row['ai_provider_name']); + $provider_name = escapeHtml($row['ai_provider_name']); $model_id = intval($row['ai_model_id']); - $model_name = nullable_htmlentities($row['ai_model_name']); - $use_case = nullable_htmlentities($row['ai_model_use_case']); - $prompt = nl2br(nullable_htmlentities($row['ai_model_prompt'])); + $model_name = escapeHtml($row['ai_model_name']); + $use_case = escapeHtml($row['ai_model_use_case']); + $prompt = nl2br(escapeHtml($row['ai_model_prompt'])); ?> - + - - - + + + diff --git a/admin/ai_provider.php b/admin/ai_providers.php similarity index 79% rename from admin/ai_provider.php rename to admin/ai_providers.php index 1f8eb398a..3bb6b5729 100644 --- a/admin/ai_provider.php +++ b/admin/ai_providers.php @@ -25,17 +25,17 @@ $num_rows = mysqli_num_rows($sql); "> - + Provider - + URL - + Key @@ -50,9 +50,9 @@ $num_rows = mysqli_num_rows($sql); while ($row = mysqli_fetch_assoc($sql)) { $provider_id = intval($row['ai_provider_id']); - $provider_name = nullable_htmlentities($row['ai_provider_name']); - $url = nullable_htmlentities($row['ai_provider_api_url']); - $key = nullable_htmlentities($row['ai_provider_api_key']); + $provider_name = escapeHtml($row['ai_provider_name']); + $url = escapeHtml($row['ai_provider_api_url']); + $key = escapeHtml($row['ai_provider_api_key']); $row = mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT COUNT('ai_model_id') AS ai_model_count FROM ai_models WHERE ai_model_ai_provider_id = $provider_id")); $ai_model_count = intval($row['ai_model_count']); @@ -62,13 +62,13 @@ $num_rows = mysqli_num_rows($sql); - + - - + + - + diff --git a/admin/api_keys.php b/admin/api_keys.php index eb21404ca..1d6173b81 100644 --- a/admin/api_keys.php +++ b/admin/api_keys.php @@ -9,7 +9,7 @@ require_once "includes/inc_all_admin.php"; $sql = mysqli_query( $mysqli, "SELECT SQL_CALC_FOUND_ROWS * FROM api_keys - LEFT JOIN clients on api_keys.api_key_client_id = clients.client_id + LEFT JOIN users on api_key_user_id = user_id WHERE (api_key_name LIKE '%$q%') ORDER BY $sort $order LIMIT $record_from, $record_to" ); @@ -33,7 +33,7 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()"));
- +
@@ -64,7 +64,7 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()"));
- + "> @@ -75,27 +75,27 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); @@ -107,45 +107,44 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); while ($row = mysqli_fetch_assoc($sql)) { $api_key_id = intval($row['api_key_id']); - $api_key_name = nullable_htmlentities($row['api_key_name']); - $api_key_secret = nullable_htmlentities("************" . substr($row['api_key_secret'], -4)); - $api_key_created_at = nullable_htmlentities($row['api_key_created_at']); - $api_key_expire = nullable_htmlentities($row['api_key_expire']); + $api_key_name = escapeHtml($row['api_key_name']); + $api_key_secret = escapeHtml("************" . substr($row['api_key_secret'], -4)); + $api_key_created_at = escapeHtml($row['api_key_created_at']); + $api_key_expire = escapeHtml($row['api_key_expire']); if ($api_key_expire < date("Y-m-d H:i:s")) { $api_key_expire = $api_key_expire . " (Expired)"; } - if ($row['api_key_client_id'] == 0) { - $api_key_client = "All Clients"; - } else { - $api_key_client = nullable_htmlentities($row['client_name']); - } + $api_key_user = !empty($row['user_name']) ? escapeHtml($row['user_name']) : "None"; ?> - - - - - + + + + + "> @@ -143,18 +143,18 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); while ($row = mysqli_fetch_assoc($sql)) { $log_id = intval($row['app_log_id']); - $log_type = nullable_htmlentities($row['app_log_type']); - $log_category = nullable_htmlentities($row['app_log_category']); - $log_details = nullable_htmlentities($row['app_log_details']); - $log_created_at = nullable_htmlentities($row['app_log_created_at']); + $log_type = escapeHtml($row['app_log_type']); + $log_category = escapeHtml($row['app_log_category']); + $log_details = escapeHtml($row['app_log_details']); + $log_created_at = escapeHtml($row['app_log_created_at']); ?> - - - - + + + +
- +
@@ -90,9 +90,9 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); $sql_clients_filter = mysqli_query($mysqli, "SELECT * FROM clients ORDER BY client_name ASC"); while ($row = mysqli_fetch_assoc($sql_clients_filter)) { $client_id = intval($row['client_id']); - $client_name = nullable_htmlentities($row['client_name']); + $client_name = escapeHtml($row['client_name']); ?> - + @@ -110,9 +110,9 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); $sql_users_filter = mysqli_query($mysqli, "SELECT * FROM users ORDER BY user_name ASC"); while ($row = mysqli_fetch_assoc($sql_users_filter)) { $user_id = intval($row['user_id']); - $user_name = nullable_htmlentities($row['user_name']); + $user_name = escapeHtml($row['user_name']); ?> - + @@ -129,9 +129,9 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); - + @@ -148,9 +148,9 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); - + @@ -165,9 +165,9 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()"));
- - - + + +
@@ -179,44 +179,44 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()"));
text-nowrap"> @@ -227,22 +227,22 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); while ($row = mysqli_fetch_assoc($sql)) { $log_id = intval($row['log_id']); - $log_type = nullable_htmlentities($row['log_type']); - $log_action = nullable_htmlentities($row['log_action']); - $log_description = nullable_htmlentities($row['log_description']); - $log_ip = nullable_htmlentities($row['log_ip']); - $log_user_agent = nullable_htmlentities($row['log_user_agent']); + $log_type = escapeHtml($row['log_type']); + $log_action = escapeHtml($row['log_action']); + $log_description = escapeHtml($row['log_description']); + $log_ip = escapeHtml($row['log_ip']); + $log_user_agent = escapeHtml($row['log_user_agent']); $log_user_os = getOS($log_user_agent); $log_user_browser = getWebBrowser($log_user_agent); - $log_created_at = nullable_htmlentities($row['log_created_at']); + $log_created_at = escapeHtml($row['log_created_at']); $user_id = intval($row['user_id']); - $user_name = nullable_htmlentities($row['user_name']); + $user_name = escapeHtml($row['user_name']); if (empty($user_name)) { $user_name_display = "-"; } else { $user_name_display = $user_name; } - $client_name = nullable_htmlentities($row['client_name']); + $client_name = escapeHtml($row['client_name']); $client_id = intval($row['client_id']); if (empty($client_name)) { $client_name_display = "-"; @@ -254,16 +254,16 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); ?> - - + + - + - - - - - + + + + + Cron - archives are built by the dispatcher, so a dead +// crontab means the buttons below queue work that never runs +$cron_is_running = $cron_last_dispatch_at !== null && (time() - strtotime($cron_last_dispatch_at)) < 300; + +$backup_job = mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT cron_job_enabled, cron_job_daily_at FROM cron_jobs WHERE cron_job_name = 'backup'")); + +$backups = mysqli_query($mysqli, "SELECT * FROM backups ORDER BY backup_created_at DESC LIMIT 100"); + +$pending_count = intval(mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT COUNT(*) AS c FROM backups WHERE backup_status IN ('Pending','Running')"))['c']); + ?> + +
+
Master encryption key
+

Shown once. Refreshing this page will not show it again.

+ +
+ + + +
+
No backup encryption key
+ ITFlow could not write a backup encryption key to config.php, so it cannot produce an encrypted backup. + Make config.php writable by the web server user and reload this page, or add a line like + $config_backup_key = '<32 random characters>'; to it yourself. +
+ + + +
+
Cron is not running
+ Backups are built by the cron dispatcher, not by your browser. Until cron is running, anything you + start here will sit in the queue. See Maintenance > Cron. +
+ +
+ Cron is switched off in + Maintenance > Cron. +
+ +
-

Download Database

+

Create a Backup

-
-
If you are unable to back up the entire VM, you'll need to back up the files & database individually. There is no built-in restore. See the docs here.
-

Download Backup
+
+ + 0) { ?> +
+ backup + queued or building. You will get a notification when ready - this page does not refresh itself. +
+ + +
+
+
+ +
Full Backup
+

The database and everything in the uploads folder. This is the one to keep.

+ + Start + +
+
+
+
+ +
Database Only
+

Just the SQL dump. Much smaller and much quicker, but no attachments or documents.

+ + Start + +
+
+
+
+ +
Master Key
+

The credential vault key. Only needed if every user password is lost - a normal restore recovers the vault on its own.

+ +
+
+
+ +
+
+ +
+
+

Backups

+
+
+
+
- + Name - - Client + + User - + Secret - + Created - + Expires
- +
- + Timestamp - + Type - + Category - + Details
- + Timestamp - + User - + Client - + Type - + Action - + Description - + IP Address - + User Agent
$log_user_browser"; ?>$log_user_browser" ?>
+ + + + + + + + + + + + + + + + + + + + + + + + + +
TypeCreatedSizeSourceStatusAction
No backups yet.
0 ? escapeHtml(backupFormatBytes($backup['backup_size'])) : '-' ?> + + +
+ +
+ + + + + + + + +
+
+
+
+ +
+
+

Encryption Key

+
+
+
+ + Write this down and keep it somewhere other than this server. + Every backup is encrypted with it, and without it a backup cannot be restored - not by you, + not by anyone. It is stored in config.php and never in the database, which is what stops a + stolen backup from carrying its own key. +
+ + +
+ +
+ +
+
+ + +

+ Archives are AES-256 encrypted zips. 7-Zip, WinZip, PeaZip and Keka can open them with this key. + The unzip command, Windows Explorer and the macOS Archive Utility cannot - they do not support AES. +

+
+
+ +
+
+

Scheduled Backups & Retention

+
+
+ + + +
+
+ + +
+
+ + + 0 disables age-based deletion. +
+
+ + + Counted separately for each type. The newest of each is never deleted. +
+
+ + + + +
+ +

+ + Scheduled backups run daily at + . + + Scheduled backups are switched off. + + Turn them on or change the time in Maintenance > Cron. +

+

+ Old backups are removed by the nightly job, never by the backup itself, so a failed nightly + cannot delete an archive that was never replaced. +

-

Backup Master Encryption Key

+

Restoring

-
-
- -
-
- -
- -
+

Restoring replaces the database and the uploads folder with what is in the archive. It cannot be done from here, on purpose - a running install is the wrong place to be dropping its own tables from a browser.

+

From the command line - the only option that works for large backups:

+
+

From a browser, on a fresh install only, the setup wizard has a restore step at /setup. Once an install has users, that step closes itself.

+
+
+ +