AS1024/itflow
1
0
Fork 0
mirror of https://github.com/itflow-org/itflow synced 2026-08-04 22:57:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

Cron runs all jobs from a single dispatcher cron.php which should now be run every minute and all other cron jobs eliminated from cron

Browse Source
This commit is contained in:
johnnyq
2026-07-30 13:48:39 -04:00
parent 1005b73416
commit 38e8edea19
12 changed files with 1723 additions and 1402 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
CHANGELOG.md
View File

@@ -41,15 +41,22 @@ This file documents all notable changes made to ITFlow.
>
> **Back up your database before upgrading.**
- **A new cron job is required if you intend to use ticket SLAs.** `cron/ticket_sla.php` moves
tickets through their SLA warning and breach stages and sends the notifications. Without it,
SLA targets are still calculated and displayed but warnings and breaches will never fire. Add
it alongside the existing every-minute jobs:
- **The crontab collapses to a single entry.** `cron/cron.php` is now a dispatcher: it runs every
minute and decides which of the scripts in `cron/` are due. Everything the old `cron.php` did
nightly has moved to `cron/nightly_tasks.php`, which the dispatcher runs at 03:00. Replace every
ITFlow line in your crontab with this one:
```
* * * * * php /path/to/itflow/cron/ticket_sla.php
* * * * * php /path/to/itflow/cron/cron.php >/dev/null
```
If you do not use SLAs the job is a no-op and can be skipped.
An existing crontab keeps working as it is — the per-minute scripts still run and still lock
correctly, and `cron.php` still runs the nightly work at whatever time you call it — but jobs
added in this and future releases only run if the dispatcher is scheduled.
- **Ticket SLAs need no cron entry of their own.** `cron/ticket_sla.php` moves tickets through
their SLA warning and breach stages and sends the notifications. It is in the dispatcher's job
list and runs every minute once the entry above is in place. Without it, SLA targets are still
calculated and displayed but warnings and breaches will never fire. If you do not use SLAs the
job is a no-op.
- **All existing API keys are deleted by this update and must be recreated.** API keys are now
owned by a user and inherit that user's role, module, and client permissions rather than
carrying their own client scope. Existing keys predate this and cannot be safely mapped to a
@@ -67,7 +74,14 @@ This file documents all notable changes made to ITFlow.
- Several pages were renamed to drop the `_details` suffix and to use consistent singular and
plural filenames. Bookmarks or external links pointing at the old filenames will 404.
### Major Changes
- **One cron entry instead of five.** `cron/cron.php` is now a dispatcher that runs every minute
and decides which jobs are due, so scheduling lives in ITFlow rather than in the crontab and new
jobs arrive with an update instead of an install note. Jobs are tracked in a new `cron_jobs`
table, which means a job whose slot was missed runs at the next opportunity rather than waiting
a day, and each job is locked for its own run so a slow mailbox or a long nightly run no longer
delays anything else. The nightly work itself moved to `cron/nightly_tasks.php`.
- **Ticket SLAs (optional).** SLAs define a response target and an optional resolution target,
and are assigned per client and priority, with a global default and an explicit "no SLA"
override available for any combination. Targets are measured against your configured business

24
CONTRIBUTING.md
View File

@@ -24,7 +24,7 @@ There is no `composer install` or `npm install` step. All third-party libraries
| `client/` | The logged-in client portal (contacts of a client). |
| `guest/` | Unauthenticated flows via URL keys (view/pay invoice, view quote/ticket, view shared credentials/files/documents). |
| `api/v1/` | Key-authenticated JSON CRUD API, one directory per module. |
| `cron/` | Scheduled jobs: `cron.php`, mail queue, ticket email parser, domain/cert refreshers. |
| `cron/` | Scheduled jobs. `cron.php` is the dispatcher and the only entry in the crontab; everything else in the directory is a job it runs. See [Cron](#cron). |
| `functions.php` + `functions/` | Shared helper functions, split into topical files (`sanitize.php`, `auth.php`, `logging.php`, …) loaded by `functions.php`. New helpers go in the topical file that matches their concern. |
| `includes/` (root) | **Shared** across portals: session/auth bootstrap, DB, layout partials. |
| `post/` (root) | **Shared** POST handlers (logout, misc). |
@@ -77,6 +77,28 @@ Files named `agent/post/*_model.php` hold shared field collection/sanitization l
---
## Cron
One crontab entry runs everything:
```
* * * * * php /path/to/itflow/cron/cron.php >/dev/null
```
`cron/cron.php` is a dispatcher. It wakes every minute, works out which scripts in `cron/` are due, and requires them into its own process. Adding a job is a new script in `cron/` plus a line in the job table at the top of the dispatcher — `'every' => n` for interval jobs, `'daily_at' => 'HH:MM'` for daily ones. The crontab never changes again.
Due-ness is recorded in the `cron_jobs` table rather than matched against the clock, so a job whose minute was missed — machine down, previous run still going — runs at the next opportunity instead of being skipped for the day. A job is claimed *before* it runs, not after: a run that dies half way through is not repeated, which matters because `nightly_tasks.php` generates invoices and charges cards. Each job is also locked individually for the length of its own run (`includes/cron_lock.php`), so a long or hung job holds up only itself — the next minute's dispatch picks up everything else in a second process.
Because the jobs share one PHP process, job code has three rules:
1. **Never `exit()` or `die()`.** It ends the whole cycle and every job after it. Use `cronJobStop($message, $exit_code)` instead: it exits when the script was run directly and unwinds back to the dispatcher when it wasn't, so both paths behave as they always have.
2. **Never declare a function or class another job might declare.** Two jobs each declaring the same helper is a fatal `Cannot redeclare` the moment they share a process. Shared helpers belong in `functions/`.
3. **Set what you read.** One global scope and one set of `require_once` includes are shared across the cycle — a job's own `require_once "../config.php"` is a no-op if an earlier job already loaded it, and any variable an earlier job left behind is still there. Do not rely on the state a fresh process would have given you.
Every script in `cron/` still runs standalone (`php cron/mail_queue.php`) and still takes its own lock when it does, so anything can be run by hand for testing.
---
## Security rules (non-negotiable)
ITFlow does not use prepared statements or an ORM; queries are built as strings. That works **only** if every value is neutralized before interpolation. The rules:

26
admin/database_updates/2.6.0.php Normal file
View File

@@ -0,0 +1,26 @@
<?php
/*
* ITFlow - Database update to version 2.6.0 (from 2.5.9)
* Included by admin/database_updates.php - do not access directly
*/
defined('FROM_DB_UPDATER') || die("Direct file access is not allowed");
// cron/cron.php is now a dispatcher: it runs every minute and decides which of the
// scripts in cron/ are due, so the crontab only needs one line. That decision needs
// somewhere durable to record when each job last ran - a job whose minute was missed
// has to be picked up at the next opportunity rather than skipped, and one that is
// half way through must not be started again.
//
// Rows are created by the dispatcher the first time it sees a job, so adding a job
// later needs a line in cron.php and nothing here.
mysqli_query($mysqli, "CREATE TABLE IF NOT EXISTS `cron_jobs` (
`cron_job_id` int(11) NOT NULL AUTO_INCREMENT,
`cron_job_name` varchar(200) NOT NULL,
`cron_job_last_run_at` datetime DEFAULT NULL,
`cron_job_last_finished_at` datetime DEFAULT NULL,
`cron_job_last_status` varchar(200) DEFAULT NULL,
PRIMARY KEY (`cron_job_id`),
UNIQUE KEY `cron_job_name` (`cron_job_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci");

2
cron/certificate_refresher.php
View File

@@ -29,7 +29,7 @@ $config_enable_cron = intval($row['config_enable_cron']);
// Check cron is enabled
if ($config_enable_cron == 0) {
logApp("Cron-Certificate-Refresher", "error", "Cron Certificate Refresh unable to run - cron not enabled in admin settings.");
exit("Cron: is not enabled -- Quitting..");
cronJobStop("Cron: is not enabled -- Quitting..");
}
/*

1445
cron/cron.php
View File

File diff suppressed because it is too large Load Diff

2
cron/domain_refresher.php
View File

@@ -28,7 +28,7 @@ $config_enable_cron = intval($row['config_enable_cron']);
// Check cron is enabled
if ($config_enable_cron == 0) {
logApp("Cron-Domain-Refresher", "error", "Cron Domain Refresh unable to run - cron not enabled in admin settings.");
exit("Cron: is not enabled -- Quitting..");
cronJobStop("Cron: is not enabled -- Quitting..");
}
/*

25
cron/mail_queue.php
View File

@@ -77,12 +77,12 @@ $config_mail_oauth_access_token_expires_at = $row['config_mail_oauth_access_toke
if ($config_enable_cron == 0) {
logApp("Cron-Mail-Queue", "error", "Cron Mail Queue unable to run - cron not enabled in admin settings.");
exit("Cron: is not enabled -- Quitting..");
cronJobStop("Cron: is not enabled -- Quitting..");
}
if (empty($config_smtp_provider)) {
logApp("Cron-Mail-Queue", "info", "SMTP sending skipped: provider not configured.");
exit(0);
cronJobStop();
}
/** =======================================================================
@@ -102,27 +102,6 @@ function tokenIsExpired(?string $expires_at): bool {
return ($ts - 60) <= time();
}
function httpFormPost(string $url, array $fields): array {
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($fields, '', '&'));
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$raw = curl_exec($ch);
$err = curl_error($ch);
$code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
return [
'ok' => ($raw !== false && $code >= 200 && $code < 300),
'body' => $raw,
'code' => $code,
'err' => $err,
];
}
function persistMailOauthTokens(string $access_token, string $expires_at, ?string $refresh_token = null): void {
global $mysqli;

1347
cron/nightly_tasks.php Normal file
View File

File diff suppressed because it is too large Load Diff

70
cron/ticket_email_parser.php
View File

@@ -45,35 +45,13 @@ $company_phone = escapeSql(formatPhoneNumber($row['company_phone'], $row['compan
// Check setting enabled
if ($config_ticket_email_parse == 0) {
logApp("Cron-Email-Parser", "error", "Cron Email Parser unable to run - not enabled in admin settings.");
exit("Email Parser: Feature is not enabled - check Settings > Ticketing > Email-to-ticket parsing. See https://docs.itflow.org/ticket_email_parse -- Quitting..");
cronJobStop("Email Parser: Feature is not enabled - check Settings > Ticketing > Email-to-ticket parsing. See https://docs.itflow.org/ticket_email_parse -- Quitting..");
}
// System temp directory & lock
$temp_dir = sys_get_temp_dir();
$lock_file_path = "{$temp_dir}/itflow_email_parser_{$installation_id}.lock";
if (file_exists($lock_file_path)) {
$file_age = time() - filemtime($lock_file_path);
if ($file_age > 300) {
unlink($lock_file_path);
logApp("Cron-Email-Parser", "warning", "Cron Email Parser detected a lock file was present but was over 5 minutes old so it removed it.");
} else {
logApp("Cron-Email-Parser", "warning", "Lock file present. Cron Email Parser attempted to execute but was already executing, so instead it terminated.");
exit("Script is already running. Exiting.");
}
}
// Atomically create the lock ('x' fails if another process beat us to it)
if (@fopen($lock_file_path, 'x') === false) {
logApp("Cron-Email-Parser", "warning", "Lock file present (race). Cron Email Parser attempted to execute but was already executing, so instead it terminated.");
exit("Script is already running. Exiting.");
}
// Ensure lock gets removed even on fatal error
register_shutdown_function(function() use ($lock_file_path) {
if (file_exists($lock_file_path)) {
@unlink($lock_file_path);
}
});
// Overlapping runs are prevented by includes/cron_lock.php. This script used to keep a
// lock file of its own alongside that one, which needed a five minute age heuristic to
// recover from a killed run and could only end itself with exit() - fatal to a dispatched
// job. flock covers the same ground and the kernel drops it however the process ends.
// Allowed attachment extensions
$allowed_extensions = array('jpg', 'jpeg', 'gif', 'png', 'webp', 'svg', 'pdf', 'txt', 'md', 'doc', 'docx', 'csv', 'xls', 'xlsx', 'xlsm', 'zip', 'tar', 'gz');
@@ -394,19 +372,6 @@ function tokenExpired(?string $expires_at): bool {
}
// very small form-encoded POST helper using curl
function httpFormPost(string $url, array $fields): array {
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($fields, '', '&'));
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$raw = curl_exec($ch);
$err = curl_error($ch);
$code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
return ['ok' => ($raw !== false && $code >= 200 && $code < 300), 'body' => $raw, 'code' => $code, 'err' => $err];
}
/**
* Get a valid access token for Google Workspace IMAP via refresh token if needed.
* Uses settings: config_mail_oauth_client_id / _client_secret / _refresh_token / _access_token / _access_token_expires_at
@@ -524,8 +489,7 @@ if ($imap_provider === null) $imap_provider = '';
if ($imap_provider === '') {
// IMAP disabled by admin: exit cleanly
logApp("Cron-Email-Parser", "info", "IMAP polling skipped: provider not configured.");
@unlink($lock_file_path);
exit(0);
cronJobStop();
}
/** ------------------------------------------------------------------
@@ -551,8 +515,7 @@ if ($imap_provider === 'google_oauth') {
$pass = getGoogleAccessToken($user);
if (empty($pass)) {
logApp("Cron-Email-Parser", "error", "Google OAuth: no usable access token (check refresh token/client credentials).");
@unlink($lock_file_path);
exit(1);
cronJobStop('', 1);
}
} elseif ($imap_provider === 'microsoft_oauth') {
$host = 'outlook.office365.com';
@@ -562,15 +525,13 @@ if ($imap_provider === 'google_oauth') {
$pass = getMicrosoftAccessToken($user);
if (empty($pass)) {
logApp("Cron-Email-Parser", "error", "Microsoft OAuth: no usable access token (check refresh token/client credentials/tenant).");
@unlink($lock_file_path);
exit(1);
cronJobStop('', 1);
}
} else {
// standard_imap (username/password)
if (empty($host) || empty($port) || empty($user)) {
logApp("Cron-Email-Parser", "error", "Standard IMAP: missing host/port/username.");
@unlink($lock_file_path);
exit(1);
cronJobStop('', 1);
}
}
@@ -597,8 +558,7 @@ try {
$mailbox->connect();
} catch (\Throwable $e) {
echo "Error connecting to IMAP server: " . $e->getMessage();
@unlink($lock_file_path);
exit(1);
cronJobStop('', 1);
}
$inbox = $mailbox->inbox();
@@ -664,8 +624,7 @@ try {
}
} catch (\Throwable $e) {
logApp("Cron-Email-Parser", "error", "Unable to find/create target folder [$targetFolderName]: " . $e->getMessage());
@unlink($lock_file_path);
exit(1);
cronJobStop('', 1);
}
// Fetch unseen messages (headers, body & flags; BODY.PEEK so they stay unread)
@@ -1037,13 +996,6 @@ if ($processed_count || $unprocessed_count) {
logApp("Cron-Email-Parser", "info", "Cron Email Parser executed in $execution_time_formatted seconds. $processed_info");
}
// Remove the lock file
unlink($lock_file_path);
// DEBUG
echo "\nLock File Path: $lock_file_path\n";
if (file_exists($lock_file_path)) {
echo "\nLock is present\n\n";
}
echo "Processed Emails: $processed_count\n";
echo "Unprocessed Emails: $unprocessed_count\n";

18
db.sql
View File

@@ -950,6 +950,24 @@ CREATE TABLE `credits` (
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Table structure for table `cron_jobs`
--
DROP TABLE IF EXISTS `cron_jobs`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8mb4 */;
CREATE TABLE `cron_jobs` (
`cron_job_id` int(11) NOT NULL AUTO_INCREMENT,
`cron_job_name` varchar(200) NOT NULL,
`cron_job_last_run_at` datetime DEFAULT NULL,
`cron_job_last_finished_at` datetime DEFAULT NULL,
`cron_job_last_status` varchar(200) DEFAULT NULL,
PRIMARY KEY (`cron_job_id`),
UNIQUE KEY `cron_job_name` (`cron_job_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Table structure for table `custom_fields`
--

28
functions/request.php
View File

@@ -1,9 +1,35 @@
<?php
// Request/client detection (IP, UA, browser, OS) and response helpers
// Request/client detection (IP, UA, browser, OS), response helpers, and outbound HTTP
// Split from the former monolithic functions.php
/*
* Form-encoded POST, used for the OAuth token endpoints. Lived in both cron/mail_queue.php
* and cron/ticket_email_parser.php as identical copies until cron/cron.php began running
* them in one process, where the second declaration is a fatal error.
*/
function httpFormPost(string $url, array $fields): array {
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($fields, '', '&'));
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$raw = curl_exec($ch);
$err = curl_error($ch);
$code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
return [
'ok' => ($raw !== false && $code >= 200 && $code < 300),
'body' => $raw,
'code' => $code,
'err' => $err,
];
}
function getUserAgent() {
return $_SERVER['HTTP_USER_AGENT'];
}

108
includes/cron_lock.php
View File

@@ -1,7 +1,7 @@
<?php
/*
* ITFlow - Single-run guard for cron entry points
* ITFlow - Cron runtime: single-run guard and dispatcher support
*
* Required by each cron script immediately after its CLI check and before config.php,
* with the caller setting $cron_lock_script = __FILE__ first.
@@ -22,17 +22,103 @@
* a run that is still going is expected, not a fault worth reporting. A lock file that
* cannot be opened at all is a real misconfiguration and does report loudly.
*
* The handle is deliberately left open: the lock is held for the life of the process.
* TWO WAYS A CRON SCRIPT RUNS
*
* Directly (php cron/mail_queue.php): the guard at the bottom of this file takes the
* lock and holds it for the life of the process, exactly as it always has.
*
* Under the dispatcher (cron/cron.php): the dispatcher takes each job's lock itself,
* runs the job, and releases it before moving on, so a long job does not hold up the
* short ones on the next minute's dispatch. The guard below is skipped in that case -
* the lock is already held for this job, and the jobs share one PHP process, so a lock
* held for the life of the process would be a lock held for the whole cycle.
*
* Because the dispatcher shares one process across jobs, a job must never exit() to end
* itself early - that would take the rest of the cycle down with it. cronJobStop() is
* the replacement: it exits when the script was run directly and unwinds back to the
* dispatcher when it wasn't.
*/
$cron_lock_file = sys_get_temp_dir() . '/itflow_cron_' . md5($cron_lock_script) . '.lock';
$cron_lock_handle = fopen($cron_lock_file, 'c');
if ($cron_lock_handle === false) {
die("Cannot open the cron lock file at $cron_lock_file - check permissions and open_basedir.\n");
/*
* Thrown by cronJobStop() when a job ends itself early under the dispatcher. Carries the
* message and exit code the script would have exited with, so the dispatcher can record
* why the job stopped.
*/
class CronJobStopped extends Exception
{
}
if (!flock($cron_lock_handle, LOCK_EX | LOCK_NB)) {
// Exit silently. On a per-minute schedule, finding a previous run still going is
// normal operation rather than an error, and anything written to stdout here would
// be mailed to the crontab owner every single minute for the length of that run.
exit(0);
/*
* End the current cron job early. Direct runs exit exactly as they did before; dispatched
* runs unwind to the dispatcher, which records the reason and carries on with the next job.
*/
function cronJobStop(string $message = '', int $exit_code = 0): void
{
if (defined('ITFLOW_CRON_DISPATCHER')) {
throw new CronJobStopped($message, $exit_code);
}
if ($message !== '') {
echo $message;
}
exit($exit_code);
}
/*
* Take the single-run lock for a cron script. $script_path must be the script's own
* __FILE__ (or the same resolved path when the dispatcher takes it on the job's behalf),
* because that path is what the lock is named after.
*
* Returns the open handle on success, or false when another run holds the lock. The
* handle must stay open for as long as the lock is wanted - closing it releases the lock.
*/
function cronLockAcquire(string $script_path)
{
$lock_file = sys_get_temp_dir() . '/itflow_cron_' . md5($script_path) . '.lock';
$lock_handle = fopen($lock_file, 'c');
if ($lock_handle === false) {
die("Cannot open the cron lock file at $lock_file - check permissions and open_basedir.\n");
}
if (!flock($lock_handle, LOCK_EX | LOCK_NB)) {
// Closing our own handle does not disturb the lock the other run holds on theirs
fclose($lock_handle);
return false;
}
return $lock_handle;
}
/*
* Release a lock taken by cronLockAcquire(). Only the dispatcher needs this - a direct run
* holds its lock until the process ends and the kernel drops it.
*/
function cronLockRelease($lock_handle): void
{
if (is_resource($lock_handle)) {
flock($lock_handle, LOCK_UN);
fclose($lock_handle);
}
}
// Single-run guard for scripts run directly. Skipped under the dispatcher, which locks
// each job itself - see the note above.
if (!defined('ITFLOW_CRON_DISPATCHER')) {
if (!isset($cron_lock_script)) {
die("Cron scripts must set \$cron_lock_script = __FILE__ before requiring includes/cron_lock.php.\n");
}
$cron_lock_handle = cronLockAcquire($cron_lock_script);
if ($cron_lock_handle === false) {
// Exit silently. On a per-minute schedule, finding a previous run still going is
// normal operation rather than an error, and anything written to stdout here would
// be mailed to the crontab owner every single minute for the length of that run.
exit(0);
}
// The handle is deliberately left open: the lock is held for the life of the process.
}