diff --git a/CHANGELOG.md b/CHANGELOG.md index eb476c621..2f38060cf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -108,6 +108,18 @@ Update from Maintenance > Update — Queue Update hands the job to cron, which u - Online payments: the payment page an invoice's link opens never showed why a card was declined, or any other problem, and its button gave no sign of working while a payment went through. The reason now shows under the button until the next try, and the button shows a spinner while it works. - Emails: with mail sent through a Microsoft 365 or Google sign-in and no SMTP server filled in, a client paying from an invoice's link got no receipt and the paid notification address wasn't told, and the quote notification address wasn't told when a client accepted or declined a quote. Found by Mountaineer IT. - Recurring invoices: Make Recurring on an invoice with a discount kept the discounted total but not the discount, so the recurring invoice showed no discount and lost it the next time a line was changed. The discount goes with it now. Found by Mountaineer IT. +- Lists: a page number of 0 or less in a list's address - an old bookmark, a link typed by hand - stopped the page part way, with no list. It shows the first page now. A custom date range with a start and no end matched nothing; it now runs on from the start. Found by Mountaineer IT. +- Client portal: a contact signed in to the portal who went to ITFlow's main address was signed out, rather than taken to the portal. Found by Mountaineer IT. +- Custom pages: every form on a page of your own under agent/custom stopped with an error, as the handler loaded a file that is no longer there. Found by Mountaineer IT. +- Maintenance > Cron: a job that stopped on an error - the email parser not reaching the mail server, an RMM sync or alert check that failed - showed as Skipped, as if it had nothing to do, and wasn't counted among the failures at the top of the page. It shows as Failed now, and the email parser's failure says what the mail server answered. Found by Mountaineer IT. +- Mail: with Microsoft 365 or Google mail, the mail queue fetched a new sign-in token for every email it sent once the old one ran out, rather than once. Found by Mountaineer IT. +- Updates: once an install was a hundred or so commits behind, the update check stopped with "Data too long" and the Update page no longer showed what was new. It keeps the newest 100 commits now, and counts them all. Found by Mountaineer IT. +- Restoring from the command line into a database with no settings yet - a new install stopped at the setup wizard, as Moving to a new server has you do - stopped with a time zone error before it began. Found by Mountaineer IT. +- API: a body that isn't JSON - a form post, or none at all - and a limit under 1 or an offset under 0 answered with a server error rather than a message. Deleting a client through the API left its files behind. Found by Mountaineer IT. +- Contacts: Anonymize missed a name with an apostrophe in it (O'Brien) in the logs and in the tickets the contact raised. Found by Mountaineer IT. It also took the first name out of an email address that starts with it before the address itself, leaving the rest of the address showing. +- Assets: moving assets to another client logged "created asset (via transfer)" without the asset's name. Found by Mountaineer IT. +- New client: the duplicate warning took % and _ in the name you typed as wildcards, so a name with them in it could warn of a duplicate that isn't one. Found by Mountaineer IT. +- Admin > Access Modules: the page stopped part way when sorted by a column it doesn't have, and its New Module button, and Edit and Delete on the AI module, went nowhere. Those are gone - modules are built in. Found by Mountaineer IT. ### Developer Updates @@ -173,6 +185,8 @@ Update from Maintenance > Update — Queue Update hands the job to cron, which u - `add_payment` and `add_bulk_payment` (`agent/post/payment.php`) no longer read `balance` from the form, and the field is gone from both windows. `add_payment` checks `invoiceCanPay()` on `invoiceState()` and compares in cents; `add_bulk_payment` sums each open invoice's balance in cents and pays each `min()` of what is left and what it owes. Both set the status with `updateInvoiceStatusFromPayments()`. `add_payment_stripe` takes `pay_token` (32 hex characters, a new hidden field in Charge Saved Card), holds `GET_LOCK('itflow_invoice_pay_', 30)` from reading the invoice again to booking, charges the balance with the idempotency key `itflow-agent--`, and sets the status with `updateInvoiceStatusFromPayments()`. From Mountaineer IT's build of ITFlow. - Autopay (`cron/nightly_tasks.php`) takes Partial invoices, reads the invoice again before charging, charges `invoice_amount` less `invoicePaid()` and sets the status with `updateInvoiceStatusFromPayments()`. `stripe_create_pi` (`guest/guest_ajax.php`) answers every refusal as JSON `{"error": ...}`, compares the balance in cents and checks `billingOnlinePayAllowed()`; `guest_pay_invoice_stripe.php` checks `invoiceCanPay()` and `billingOnlinePayAllowed()` before showing the form. The guest receipt and `billingQuoteAnswer()`'s notice go on `billingEmailReady()` rather than `config_smtp_host`. `add_invoice_recurring` copies `invoice_discount_amount`. From Mountaineer IT's build of ITFlow. - `js/guest_pay_invoice_stripe.js` shows the `error` `stripe_create_pi` answers, and toggles `d-none` on the message, the spinner and the button text - the `hidden` class it toggled isn't in Bootstrap 5, and the message started as `d-none`, so it never showed. +- `includes/filter_header.php` takes `page` as at least 1, and a custom range with `dtf` and no `dtt` as ending `2099-12-31`, as All time does. `admin/modules.php` sorts only on `module_name` or `module_description`. `index.php` sends a session with `client_logged_in` to `/client/` before checking `logged`, which the portal sign-in sets too. `agent/custom/post.php` no longer requires `post/ai.php`, and it and `agent/user/post.php` read `HTTP_REFERER ?? ''`. `api/v1/validate_api_key.php` takes a body that doesn't decode to an array as `[]`, and clamps `limit` to at least 1 and `offset` to at least 0 - with no upper limit, unlike Mountaineer IT's 1000. `clients/delete` removes `uploads/clients/` from `__DIR__`. From Mountaineer IT's build of ITFlow. +- The cron dispatcher records a `CronJobStopped` with a code other than 0 (`cronJobStop($message, 1)`) as `Failed`, with its message as `cron_job_last_error` ("Stopped with an error - see App Logs" when it has none), and logs it to App Logs. `persistMailOauthTokens()` (`cron/mail_queue.php`) also sets the run's globals. `cron/update_check.php` keeps the newest 100 `pending_commits`, each subject cut to 300 characters, fewer if the JSON is still over 60,000 bytes; `updateCheckDetails()` adds `commits`, the full count, which `admin/update.php` shows. `scripts/restore_cli.php` reads the time zone only from a settings row that is there, and `backupCurrentDatabaseVersion()` answers `Unknown` when there is no settings table. Contact anonymizing redacts the stored values, longest first; the client duplicate check escapes `%` and `_`; the asset transfer log uses `$asset_name`. From Mountaineer IT's build of ITFlow. ## [26.11.8] diff --git a/admin/modules.php b/admin/modules.php index a5ee78ca3..6c2d9f638 100644 --- a/admin/modules.php +++ b/admin/modules.php @@ -6,6 +6,11 @@ $order = "DESC"; require_once "includes/inc_all_admin.php"; +// Only a column the list has - another was an SQL error that stopped the page +if (!in_array($sort, ['module_name', 'module_description'], true)) { + $sort = 'module_name'; +} + $sql = mysqli_query( $mysqli, "SELECT SQL_CALC_FOUND_ROWS module_description, module_id, module_name FROM modules @@ -20,13 +25,7 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()"));

Access Modules

-
-
- -
-
+
@@ -63,32 +62,11 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); ?> - 6) { ?> class="ajax-modal" data-modal-url="modals/modules/module_edit.php?id=" > - - +
- 6) { ?> - - N/A Predefined

"; } ?> +

N/A Predefined

@@ -105,7 +83,6 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); $empty_state = [ 'thing' => 'modules', 'icon' => 'fa-puzzle-piece', - 'actions' => [['label' => 'New Module', 'modal' => 'modals/module/module_add.php']], ]; ?> ') ? $latest_release : ''; -$commit_count = count($pending_commits); +// All of them - the stored list is the newest 100 (cron/update_check.php) +$commit_count = max(count($pending_commits), intval($details['commits'] ?? 0)); $has_upgrading_notes = (bool) array_filter($changelog, function ($section) { return preg_match('/^###\s+Upgrading/mi', $section[1]) === 1; }); @@ -435,8 +436,8 @@ $hero = [ - 100) { ?> -
  • and more
  • + min(100, count($pending_commits))) { ?> +
  • and more
  • diff --git a/agent/ajax.php b/agent/ajax.php index 75c471bf8..6879b30e7 100644 --- a/agent/ajax.php +++ b/agent/ajax.php @@ -1816,7 +1816,7 @@ if (isset($_GET['client_duplicate_check'])) { if (strlen($name) >= 5) { $sql_clients = mysqli_query($mysqli, "SELECT client_id, client_name FROM clients WHERE client_archived_at IS NULL - AND client_name LIKE '%$name%' + AND client_name LIKE '%" . addcslashes($name, '%_') . "%' ORDER BY client_id DESC LIMIT 1" ); diff --git a/agent/custom/post.php b/agent/custom/post.php index 50b8332b1..88c5d4c44 100644 --- a/agent/custom/post.php +++ b/agent/custom/post.php @@ -15,7 +15,7 @@ define('FROM_POST_HANDLER', true); // Determine which files we should load // Parse URL & get the path -$path = parse_url($_SERVER['HTTP_REFERER'], PHP_URL_PATH); +$path = parse_url($_SERVER['HTTP_REFERER'] ?? '', PHP_URL_PATH); // Get the base name (the page name) $module = explode(".", basename($path))[0]; @@ -41,6 +41,5 @@ require_once "../../post/logout.php"; // TODO: Find a home for these -require_once "../../post/ai.php"; require_once "../../post/misc.php"; diff --git a/agent/post/asset.php b/agent/post/asset.php index 962bbecaa..bcb5819bb 100644 --- a/agent/post/asset.php +++ b/agent/post/asset.php @@ -596,7 +596,7 @@ if (isset($_POST['bulk_transfer_client_asset'])) { // Log the new asset $notes = $asset_notes . "\r\n\r\n---\r\n* " . date('Y-m-d H:i:s') . ": Transferred asset $asset_name (old asset ID: $current_asset_id) from $current_client_name to $new_client_name (new asset ID: $new_asset_id)"; - logAudit("Asset", "Create", "$session_name created asset $name (via transfer)", $new_client_id, $new_asset_id); + logAudit("Asset", "Create", "$session_name created asset $asset_name (via transfer)", $new_client_id, $new_asset_id); mysqli_query($mysqli,"INSERT INTO asset_history SET asset_history_status = 'Transferred', asset_history_description = '$session_name created asset via transfer from $current_client_name', asset_history_asset_id = $new_asset_id"); logAudit("Asset", "Transfer", "$session_name Transferred asset $asset_name (old asset ID: $current_asset_id) from $current_client_name to $new_client_name (new asset ID: $new_asset_id)", $new_client_id, $new_asset_id); diff --git a/agent/post/contact.php b/agent/post/contact.php index 85d970488..2a7d0bc2e 100644 --- a/agent/post/contact.php +++ b/agent/post/contact.php @@ -701,10 +701,12 @@ if (isset($_GET['anonymize_contact'])) { } $contact_name = escapeSql($row['contact_name']); - $contact_first_name = explode(" ", $contact_name)[0]; - $contact_email = escapeSql($row['contact_email']); - $contact_phone = escapeSql($row['contact_phone']); - $info_to_redact = array($contact_name, $contact_first_name, $contact_email, $contact_phone); + // Redacted as stored, not SQL-escaped - a name with an apostrophe was missed - and never an empty value. + // Longest first, so the first name doesn't take a piece out of the email address before it goes + $info_to_redact = array_values(array_filter(array_map('strval', [$row['contact_name'], explode(" ", (string) $row['contact_name'])[0], $row['contact_email'], $row['contact_phone']]), 'strlen')); + usort($info_to_redact, function ($a, $b) { + return strlen($b) - strlen($a); + }); $client_id = intval($row['contact_client_id']); $contact_user_id = intval($row['contact_user_id']); diff --git a/agent/user/post.php b/agent/user/post.php index 48a30ac49..9d82a8483 100644 --- a/agent/user/post.php +++ b/agent/user/post.php @@ -15,7 +15,7 @@ define('FROM_POST_HANDLER', true); // Determine which files we should load // Parse URL & get the path -$path = parse_url($_SERVER['HTTP_REFERER'], PHP_URL_PATH); +$path = parse_url($_SERVER['HTTP_REFERER'] ?? '', PHP_URL_PATH); // Get the base name (the page name) $module = explode(".", basename($path))[0]; diff --git a/api/v1/clients/delete.php b/api/v1/clients/delete.php index 64b213841..3073e2484 100644 --- a/api/v1/clients/delete.php +++ b/api/v1/clients/delete.php @@ -75,7 +75,7 @@ if (!empty($client_id)) { mysqli_query($mysqli, "DELETE FROM trips WHERE trip_client_id = $client_id"); mysqli_query($mysqli, "DELETE FROM vendors WHERE vendor_client_id = $client_id"); - removeDirectory("../../uploads/clients/$client_id"); + removeDirectory(__DIR__ . "/../../../uploads/clients/$client_id"); $delete_sql = mysqli_query($mysqli, "DELETE FROM clients WHERE client_id = $client_id"); diff --git a/api/v1/validate_api_key.php b/api/v1/validate_api_key.php index 243322a93..3f2c19dbd 100644 --- a/api/v1/validate_api_key.php +++ b/api/v1/validate_api_key.php @@ -21,7 +21,9 @@ header('Content-Type: application/json'); managedRequestGuard('api'); // POST data +// Anything that isn't a JSON object - a form post, an empty body - is no fields, answered as such rather than a 500 $_POST = json_decode(file_get_contents('php://input'), true); +$_POST = is_array($_POST) ? $_POST : []; // Get IP & UA $ip = escapeSql(getIP()); @@ -149,6 +151,10 @@ if (isset($api_key)) { $offset = 0; } + // A limit under 1, or an offset under 0, was an SQL error + $limit = max(1, $limit); + $offset = max(0, $offset); + // When the key is tied to a user, enforce that user's RBAC (module + operation + client scope) require __DIR__ . '/enforce_api_rbac.php'; diff --git a/cron/cron.php b/cron/cron.php index da51e1421..5a607e3d0 100644 --- a/cron/cron.php +++ b/cron/cron.php @@ -270,9 +270,19 @@ foreach (cronJobRegistry() as $cron_dispatch_job) { require_once $cron_dispatch_path; cronJobFinished($mysqli, $cron_dispatch_job['name'], 'Completed', microtime(true) - $cron_dispatch_started); } catch (CronJobStopped $e) { - // The job ended itself early - disabled in settings, nothing configured, nothing to do - $reason = $e->getMessage(); - cronJobFinished($mysqli, $cron_dispatch_job['name'], $reason === '' ? 'Stopped' : "Stopped: $reason", microtime(true) - $cron_dispatch_started); + // The job ended itself early - disabled in settings, nothing configured, nothing to do. One that + // stopped with an error code (the mail server turned it away, say) failed, and shows as Failed + $reason = trim($e->getMessage()); + if ($e->getCode() !== 0) { + try { + logApp("Cron", "error", "Cron job {$cron_dispatch_job['name']} stopped with an error" . ($reason === '' ? '' : ": $reason")); + } catch (Throwable $log_e) { + // Logging the failure must never become a second, fatal failure + } + cronJobFinished($mysqli, $cron_dispatch_job['name'], 'Failed', microtime(true) - $cron_dispatch_started, $reason === '' ? 'Stopped with an error - see App Logs' : $reason); + } else { + cronJobFinished($mysqli, $cron_dispatch_job['name'], $reason === '' ? 'Stopped' : "Stopped: $reason", microtime(true) - $cron_dispatch_started); + } } catch (Throwable $e) { // One job throwing is not a reason to skip the rest of the cycle echo "Cron: job '{$cron_dispatch_job['name']}' failed - " . $e->getMessage() . "\n"; diff --git a/cron/mail_queue.php b/cron/mail_queue.php index 0894772ef..3888f5a70 100644 --- a/cron/mail_queue.php +++ b/cron/mail_queue.php @@ -127,6 +127,13 @@ function persistMailOauthTokens(string $access_token, string $expires_at, ?strin } mysqli_query($mysqli, "UPDATE settings SET config_mail_oauth_access_token = '{$access_token_esc}', config_mail_oauth_access_token_expires_at = '{$expires_at_esc}'{$refresh_sql} WHERE company_id = 1"); + + // This run's copy too, so the next email uses the new token rather than fetching another + $GLOBALS['config_mail_oauth_access_token'] = $access_token; + $GLOBALS['config_mail_oauth_access_token_expires_at'] = $expires_at; + if (!empty($refresh_token)) { + $GLOBALS['config_mail_oauth_refresh_token'] = $refresh_token; + } } function refreshMailOauthAccessToken(string $provider, string $oauth_client_id, string $oauth_client_secret, string $oauth_tenant_id, string $oauth_refresh_token): ?array { diff --git a/cron/ticket_email_parser.php b/cron/ticket_email_parser.php index f20a47ebc..03188192a 100644 --- a/cron/ticket_email_parser.php +++ b/cron/ticket_email_parser.php @@ -548,8 +548,7 @@ $mailbox = new Mailbox([ try { $mailbox->connect(); } catch (\Throwable $e) { - echo "Error connecting to IMAP server: " . $e->getMessage(); - cronJobStop('', 1); + cronJobStop("Error connecting to the mail server: " . $e->getMessage() . "\n", 1); } $inbox = $mailbox->inbox(); diff --git a/cron/update_check.php b/cron/update_check.php index 56d0c1398..8e25ff528 100644 --- a/cron/update_check.php +++ b/cron/update_check.php @@ -72,7 +72,19 @@ if ($updates->result !== 0) { } $update_check_latest = escapeSql($updates->latest_version); -$update_check_commits = escapeSql((string) json_encode($updates->pending_commits)); +// Kept in a TEXT column (64 KB): the newest 100 - all the Update page lists - each subject cut to 300 characters, +// and fewer if that still doesn't fit. A long run of long subjects was "Data too long" and stopped the check. +// The Update page counts them all from config_update_details (updateCheckDetails()). +$update_check_list = array_map(function ($commit) { + $commit[2] = mb_strlen((string) $commit[2]) > 300 ? mb_substr((string) $commit[2], 0, 299) . '…' : (string) $commit[2]; + return $commit; +}, array_slice($updates->pending_commits, 0, 100)); +$update_check_json = (string) json_encode($update_check_list); +while (strlen($update_check_json) > 60000 && $update_check_list) { + array_pop($update_check_list); + $update_check_json = (string) json_encode($update_check_list); +} +$update_check_commits = escapeSql($update_check_json); mysqli_query($mysqli, "UPDATE settings SET config_update_latest_commit = '$update_check_latest', diff --git a/functions/app.php b/functions/app.php index eaee0b05e..f898a9300 100644 --- a/functions/app.php +++ b/functions/app.php @@ -643,6 +643,7 @@ function checkForUpdates() { * version while work lands (develop), that version's section too * local_changes ITFlow's own files this install has changed, which updating puts back * (update_cli.php resets to the remote) - the first 50, and how many in all + * commits how many commits behind - config_update_pending_commits keeps the newest 100 * * Anything that can't be read comes back empty rather than failing the check. */ @@ -652,7 +653,7 @@ function updateCheckDetails(int $pending_commits): array require_once __DIR__ . '/../includes/app_version.php'; } - $details = ['version' => '', 'changelog' => [], 'local_changes' => [], 'local_changes_count' => 0]; + $details = ['version' => '', 'changelog' => [], 'local_changes' => [], 'local_changes_count' => 0, 'commits' => $pending_commits]; if (!shellCommandsAvailable()) { return $details; diff --git a/functions/backup.php b/functions/backup.php index 82061370c..75c1d259b 100644 --- a/functions/backup.php +++ b/functions/backup.php @@ -791,7 +791,12 @@ function backupCreate(mysqli $mysqli, string $type, string $created_by, string $ */ function backupCurrentDatabaseVersion(mysqli $mysqli): string { - $res = mysqli_query($mysqli, "SELECT config_current_database_version FROM settings WHERE company_id = 1"); + // A database with no settings yet - restoring into a new install - has no version + try { + $res = mysqli_query($mysqli, "SELECT config_current_database_version FROM settings WHERE company_id = 1"); + } catch (Throwable $e) { + return 'Unknown'; + } if ($res && $row = mysqli_fetch_assoc($res)) { return $row['config_current_database_version'] ?? 'Unknown'; } diff --git a/includes/filter_header.php b/includes/filter_header.php index d60e0e063..c80e11067 100644 --- a/includes/filter_header.php +++ b/includes/filter_header.php @@ -17,7 +17,7 @@ $url_query_strings_sort = http_build_query($get_copy); // Paging if (isset($_GET['page'])) { - $page = intval($_GET['page']); + $page = max(1, intval($_GET['page'])); // page 0 or below made a negative LIMIT - the list stopped there $record_from = (($page)-1)*$user_config_records_per_page; $record_to = $user_config_records_per_page; } else { @@ -73,7 +73,7 @@ if (empty($_GET['canned_date'])) { // Date Filter if ($_GET['canned_date'] == "custom" && !empty($_GET['dtf'])) { $dtf = escapeSql($_GET['dtf']); - $dtt = escapeSql($_GET['dtt']); + $dtt = !empty($_GET['dtt']) ? escapeSql($_GET['dtt']) : '2099-12-31'; // A start and no end is open-ended, as All time is } elseif ($_GET['canned_date'] == "today") { $dtf = date('Y-m-d'); $dtt = date('Y-m-d'); diff --git a/index.php b/index.php index 79aa23038..6408116fa 100644 --- a/index.php +++ b/index.php @@ -13,15 +13,15 @@ if (file_exists("config.php")) { // Start the session require_once "includes/session_init.php"; - // If user is an agent - if (isset($_SESSION['logged'])) { - require_once "includes/load_global_settings.php"; - header("Location: /agent/$config_start_page"); + // If user is a client - asked first, as a portal sign-in sets 'logged' too, and an agent page signs them out + if (!empty($_SESSION['client_logged_in'])) { + header("Location: /client/"); exit(); - // If user is a client - } elseif (isset($_SESSION['client_logged_in'])) { - header("Location: /client/"); + // If user is an agent + } elseif (isset($_SESSION['logged'])) { + require_once "includes/load_global_settings.php"; + header("Location: /agent/$config_start_page"); exit(); // Not logged in diff --git a/scripts/restore_cli.php b/scripts/restore_cli.php index a4ddc917d..a1f71b1c5 100644 --- a/scripts/restore_cli.php +++ b/scripts/restore_cli.php @@ -34,7 +34,18 @@ if (!file_exists("../config.php")) { } require_once "../config.php"; -require_once "../includes/inc_set_timezone.php"; + +// A restore often goes into a database that has only got as far as config.php - the setup +// wizard's database step, with no settings yet. The timezone comes from the settings when +// they are there; inc_set_timezone.php takes them for granted and would stop the restore. +try { + $restore_timezone_row = mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT config_timezone FROM settings WHERE company_id = 1")); +} catch (Throwable $e) { + $restore_timezone_row = null; +} +if (!empty($restore_timezone_row['config_timezone'])) { + require_once "../includes/inc_set_timezone.php"; +} require_once "../functions.php"; $options = getopt("", ["file:", "key:", "inspect", "yes", "help"]); @@ -90,7 +101,7 @@ if (isset($options['inspect'])) { exit(0); } -if ($meta['database_version'] !== 'Unknown' && version_compare($meta['database_version'], $current_db_version, '>')) { +if ($meta['database_version'] !== 'Unknown' && $current_db_version !== 'Unknown' && version_compare($meta['database_version'], $current_db_version, '>')) { echo "\nWARNING: this backup is from a NEWER database version than the code in this directory.\n"; echo "Update ITFlow to a matching version before restoring, or the app will error after the restore.\n"; }