Cron. One answer at the top - cron is running, it has stopped or never * started, it is switched off, a job failed - then every job: what it does, its schedule, * how its last run went and when the next is. Beside them: cron at a glance, the one * crontab line all of it runs from, and the switch that stops every job at once. * * Nothing runs here. Run now sets a flag the dispatcher (cron/cron.php) takes on its next * pass, so while a job is queued or running the page reloads itself (js/auto_reload.js), as * Maintenance > Update and Backup do - and while cron hasn't been heard from, so the page * says so by itself once the crontab line is in. The handlers are admin/post/cron.php. * * The jobs are the registry's (includes/cron_jobs.php), each with its row from cron_jobs. A * job the dispatcher has not met yet has no row: it shows the schedule it ships with, and * there is nothing to change about it until cron has run once. */ // The list: by schedule until a heading is clicked - the every-minute jobs, then the day's in the order they run $sort = "schedule"; $order = "ASC"; require_once "includes/inc_all_admin.php"; require_once $_SERVER['DOCUMENT_ROOT'] . '/includes/cron_jobs.php'; $csrf = $_SESSION['csrf_token']; $row = mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT config_enable_cron, config_cron_last_dispatch_at FROM settings WHERE company_id = 1")); $cron_on = intval($row['config_enable_cron']) === 1; $cron_last_dispatch_at = $row['config_cron_last_dispatch_at']; // The dispatcher writes its heartbeat before it runs anything, so one older than a few // minutes means the crontab entry itself is missing or failing - a different problem from a // job that is off or erroring, and the one people spend the longest not finding. $cron_heard = !empty($cron_last_dispatch_at) && (time() - strtotime($cron_last_dispatch_at)) < 300; // A managed instance its manager has made read-only or suspended runs only the jobs that keep it looked after $managed_mode = managedMode(); $managed_kept = $managed_mode === 'active' ? [] : (MANAGED_KEEP_JOBS[$managed_mode] ?? []); // The crontab line, and a job run by hand, as this server runs them - in this folder, as the user that owns the files $install_root = dirname(__DIR__); $file_owner = ''; if (function_exists('posix_getpwuid')) { $file_owner = (string) (posix_getpwuid((int) @fileowner($install_root . '/cron/cron.php'))['name'] ?? ''); } $shell_word = function ($word) { return preg_match('/^[A-Za-z0-9_\/.@%+=:,-]+$/', $word) ? $word : escapeshellarg($word); }; $cron_command = "* * * * * php $install_root/cron/cron.php >/dev/null"; $by_hand = function ($script) use ($install_root, $file_owner, $shell_word) { return "cd " . $shell_word($install_root) . " && " . ($file_owner !== '' ? "sudo -u " . $shell_word($file_owner) . " " : '') . "php cron/" . $shell_word($script); }; // What a job said as it stopped early, as it reads on the page $tidy_reason = function ($reason) { $reason = preg_replace('/\s*--\s*Quitting\.*\s*$/i', '', trim(ltrim((string) $reason, ': '))); if (stripos($reason, 'Cron: is not enabled') === 0) { return 'Cron is switched off'; } return rtrim($reason, ". \n\r\t"); }; $job_rows = []; $sql = mysqli_query($mysqli, "SELECT cron_job_daily_at, cron_job_enabled, cron_job_id, cron_job_interval_minutes, cron_job_last_duration, cron_job_last_error, cron_job_last_error_at, cron_job_last_run_at, cron_job_last_status, cron_job_name, cron_job_run_now, cron_job_schedule FROM cron_jobs"); while ($job_row = mysqli_fetch_assoc($sql)) { $job_rows[$job_row['cron_job_name']] = $job_row; } /* * Each job the registry lists, with what its row says. How its last run went is one of: * queued (Run now, not picked up yet), running, lost (still Running hours later - it was cut * off, and nothing recorded how), completed, failed, skipped (it stopped itself early, and * says why), paused (by a manager), never. */ $jobs = []; foreach (cronJobRegistry() as $position => $entry) { $job_row = $job_rows[$entry['name']] ?? null; $job = [ 'name' => $entry['name'], 'label' => $entry['label'], 'description' => $entry['description'], 'script' => $entry['script'], 'icon' => $entry['icon'] ?? 'fa-cog', 'position' => $position, 'id' => intval($job_row['cron_job_id'] ?? 0), 'enabled' => $job_row ? intval($job_row['cron_job_enabled']) === 1 : intval($entry['enabled'] ?? 1) === 1, 'schedule' => $job_row ? (string) $job_row['cron_job_schedule'] : (string) $entry['schedule'], 'interval_minutes' => max(1, intval($job_row ? $job_row['cron_job_interval_minutes'] : ($entry['interval_minutes'] ?? 1))), 'daily_at' => substr((string) ($job_row ? $job_row['cron_job_daily_at'] : ($entry['daily_at'] ?? '')), 0, 5), 'last_run_at' => $job_row['cron_job_last_run_at'] ?? null, 'last_duration' => $job_row['cron_job_last_duration'] ?? null, 'last_error' => trim((string) ($job_row['cron_job_last_error'] ?? '')), 'last_error_at' => $job_row['cron_job_last_error_at'] ?? null, 'reason' => '', ]; $job['schedule_text'] = cronJobScheduleDescription($job['schedule'], $job['interval_minutes'], $job['daily_at']); $job['every_minute'] = $job['schedule'] !== 'Daily' && $job['interval_minutes'] === 1; $status = (string) ($job_row['cron_job_last_status'] ?? ''); if (!empty($job_row['cron_job_run_now'])) { $job['state'] = 'queued'; } elseif ($status === 'Running') { $job['state'] = strtotime((string) $job['last_run_at']) > strtotime('-6 hours') ? 'running' : 'lost'; } elseif ($status === 'Completed') { $job['state'] = 'completed'; } elseif ($status === 'Failed') { $job['state'] = 'failed'; } elseif ($status === '') { $job['state'] = 'never'; } else { $job['reason'] = $tidy_reason(str_starts_with($status, 'Stopped') ? substr($status, 7) : $status); $job['state'] = 'skipped'; if (str_starts_with($job['reason'], 'Paused')) { $job['state'] = 'paused'; $job['reason'] = ucfirst(ltrim(substr($job['reason'], 6), ' -')); } } // When it runs next - nothing to say while cron isn't running, is switched off, or holds this job back $dispatched = $cron_heard && $cron_on && ($managed_mode === 'active' || in_array($job['name'], $managed_kept, true)); $job['next_run'] = $job_row && $job['enabled'] && $dispatched ? cronJobNextRun($job_row) : null; $jobs[$job['name']] = $job; } $jobs_on = array_filter($jobs, function ($job) { return $job['enabled']; }); $jobs_off = count($jobs) - count($jobs_on); // Failed on its last run - with an error, or cut off without a word $jobs_failed = array_filter($jobs, function ($job) { return in_array($job['state'], ['failed', 'lost'], true); }); // Failed before and has run since - its error stays until somebody dismisses it $jobs_recovered = array_filter($jobs, function ($job) { return $job['last_error'] !== '' && !in_array($job['state'], ['failed', 'lost'], true); }); // On their way: asked for, or started in the last half hour $jobs_busy = array_filter($jobs, function ($job) { return $job['state'] === 'queued' || ($job['state'] === 'running' && strtotime((string) $job['last_run_at']) > strtotime('-30 minutes')); }); // The next job that isn't one of the every-minute ones $next_job = null; foreach ($jobs as $job) { if ($job['next_run'] !== null && !$job['every_minute'] && (!$next_job || $job['next_run'] < $next_job['next_run'])) { $next_job = $job; } } /* * The one thing to say at the top. In this order, because each makes the ones after it moot: * cron not running at all, running but switched off, most of it held back by a manager, a * job that failed, then all well. */ if (!$cron_heard) { $state = empty($cron_last_dispatch_at) ? 'never' : 'stopped'; } elseif (!$cron_on) { $state = 'off'; } elseif ($managed_mode !== 'active') { $state = 'paused'; } elseif ($jobs_failed) { $state = 'failed'; } else { $state = 'running'; } $failed_count = count($jobs_failed); $hero = [ 'never' => ['icon' => 'fa-plug', 'tone' => 'warning', 'title' => "Cron hasn't run yet"], 'stopped' => ['icon' => 'fa-exclamation-triangle', 'tone' => 'danger', 'title' => 'Cron has stopped'], 'off' => ['icon' => 'fa-power-off', 'tone' => 'warning', 'title' => 'Cron is switched off'], 'paused' => ['icon' => 'fa-pause', 'tone' => 'warning', 'title' => 'Most jobs are paused'], 'failed' => ['icon' => 'fa-exclamation-triangle', 'tone' => 'danger', 'title' => $failed_count === 1 ? reset($jobs_failed)['label'] . ' failed' : "$failed_count jobs failed"], 'running' => ['icon' => 'fa-check', 'tone' => 'success', 'title' => 'Cron is running'], ][$state]; // Reload while something is on its way, and while waiting to hear from cron at all $reload_seconds = !$cron_heard ? 15 : ($jobs_busy ? 10 : 0); // A time as "3 hours ago", with the date and time behind it $ago = function ($datetime) { return '' . escapeHtml(timeAgo($datetime)) . ''; }; // How long a run took $took = function ($seconds) { $seconds = floatval($seconds); if ($seconds < 0.1) { return 'under 0.1 s'; } if ($seconds < 10) { return rtrim(rtrim(number_format($seconds, 1), '0'), '.') . ' s'; } if ($seconds < 60) { return round($seconds) . ' s'; } $minutes = intdiv((int) round($seconds), 60); if ($minutes < 60) { return $minutes . ' min ' . ((int) round($seconds) % 60) . ' s'; } return intdiv($minutes, 60) . ' h ' . ($minutes % 60) . ' min'; }; // Job names in a sentence - "Mail Queue", "Mail Queue and Backup", "Mail Queue, Backup and Nightly Tasks" - each a link to its row $job_names = function ($list) { $links = array_map(function ($job) { return '' . escapeHtml($job['label']) . ''; }, array_values($list)); $last = array_pop($links); return $links ? implode(', ', $links) . ' and ' . $last : (string) $last; }; // The crontab line, with a button that copies it $command_block = function () use ($cron_command) { return '
' . escapeHtml($cron_command) . ''
. ''
. '' . escapeHtml($file_owner) . ' here)' : '';
// The list - only these sort; anything else asked for falls back to the schedule
$sort_keys = [
'job' => function ($job) { return strtolower($job['label']); },
'schedule' => function ($job) { return $job['schedule'] === 'Daily' ? 100000 + intval(substr($job['daily_at'], 0, 2)) * 60 + intval(substr($job['daily_at'], 3, 2)) : $job['interval_minutes']; },
'last_run' => function ($job) { return $job['last_run_at'] ? strtotime($job['last_run_at']) : 0; },
// What needs a look first
'status' => function ($job) { return array_search($job['state'], ['failed', 'lost', 'queued', 'running', 'paused', 'skipped', 'never', 'completed'], true); },
];
if (!isset($sort_keys[$sort])) {
$sort = 'schedule';
}
uasort($jobs, function ($a, $b) use ($sort_keys, $sort, $order) {
$compared = $sort_keys[$sort]($a) <=> $sort_keys[$sort]($b);
return ($order === 'DESC' ? -$compared : $compared) ?: $a['position'] <=> $b['position'];
});
// A heading sorts its own way round first - names A to Z, the latest run and what needs a look at the top - then the other
$sort_first = ['job' => 'ASC', 'schedule' => 'ASC', 'last_run' => 'DESC', 'status' => 'ASC'];
$sort_link = function ($column, $label) use ($url_query_strings_sort, $sort, $disp, $order_icon, $sort_first) {
return ''
. $label . ($sort === $column ? " $order_icon" : '') . '';
};
$status_looks = [
'queued' => ['label' => 'Queued', 'tone' => 'warning'],
'running' => ['label' => 'Running', 'tone' => 'primary'],
'lost' => ['label' => "Didn't finish", 'tone' => 'danger'],
'completed' => ['label' => 'Completed', 'tone' => 'success'],
'failed' => ['label' => 'Failed', 'tone' => 'danger'],
'skipped' => ['label' => 'Skipped', 'tone' => 'secondary'],
'paused' => ['label' => 'Paused', 'tone' => 'warning'],
'never' => ['label' => 'Not run yet', 'tone' => 'secondary'],
];
?>
ITFlow does its background work from one cron job on the server - sending email, turning email into tickets, recurring invoices, reminders, backups - and it hasn't heard from it yet. Add this line to the crontab of the user that owns ITFlow's files= $owner_note ?>, or as a cron job that runs every minute in your hosting control panel:
= $command_block() ?>This page says so by itself once it has run - give it a minute.
ITFlow last heard from it = $ago($cron_last_dispatch_at) ?>. Until it runs again no email is sent, no email becomes a ticket, and nothing is invoiced or backed up. Check that the server's cron service is running, and that this line is still in the crontab of the user that owns ITFlow's files= $owner_note ?>:
= $command_block() ?>This page says so by itself when it's back.
The server still calls it every minute, but each job stops itself straight away - no email is sent, no email becomes a ticket, and nothing is invoiced or backed up. That's right for a restored backup or a test copy, which mustn't email your clients or charge their cards. Anywhere else, turn it back on - every job's schedule is as it was left.
= escapeHtml(ucfirst(managedManagerName())) ?> has made this ITFlow = $managed_mode === 'suspended' ? 'suspended' : 'read-only' ?>. = $job_names(array_intersect_key($jobs, array_flip($managed_kept))) ?> keep running; the rest wait, and run when it's active again.
Cron itself is running - it last ran = $ago($cron_last_dispatch_at) ?> - but = $job_names($jobs_failed) ?> didn't finish = $failed_count === 1 ? 'its' : 'their' ?> last run. What went wrong is under = $failed_count === 1 ? 'it' : 'each' ?> below.
It last ran = $ago($cron_last_dispatch_at) ?>. = count($jobs_on) ?> job= count($jobs_on) === 1 ? ' is' : 's are' ?> on= $jobs_off ? ", $jobs_off off" : '' ?>, and none failed on its last run. Next is = $job_names([$next_job]) ?>, = $next_job['next_run'] <= date('Y-m-d H:i:s') ? 'within a minute' : $ago($next_job['next_run']) ?>.
= $job_names($jobs_recovered) ?> failed earlier and = count($jobs_recovered) === 1 ? 'has' : 'have' ?> run since - what went wrong is kept below until you dismiss it.
= $job_names($jobs_busy) ?> = count($jobs_busy) === 1 ? 'is' : 'are' ?> on = count($jobs_busy) === 1 ? 'its' : 'their' ?> way - this page keeps itself up to date.
Cron is switched off here as well, so its jobs do nothing when it runs until it's turned on.
| = $sort_link('job', 'Job') ?> | = $sort_link('schedule', 'Schedule') ?> | = $sort_link('last_run', 'Last run') ?> | = $sort_link('status', 'Status') ?> | ||
|---|---|---|---|---|---|
| = escapeHtml($job['label']) ?> Off |
= escapeHtml($job['schedule_text']) ?>
Next = $job['next_run'] <= date('Y-m-d H:i:s') ? 'within a minute' : escapeHtml(timeAgo($job['next_run'])) ?>
|
= $ago($job['last_run_at']) ?>
Took = escapeHtml($took($job['last_duration'])) ?>
—
|
= escapeHtml($look['label']) ?> |
|
|
|
= escapeHtml($job['description']) ?>
= escapeHtml($job['schedule_text']) ?>= $job['last_run_at'] ? ' · last run ' . escapeHtml(timeAgo($job['last_run_at'])) : '' ?>
Asked to run now - = $cron_heard ? 'cron picks it up within a minute' : 'it runs as soon as cron does' ?>.
It started = $ago($job['last_run_at']) ?> and never said how it ended - most likely cut off by a restart or a time limit. It runs again when it's next due, or sooner with Run now.
It stopped early without saying why - App Logs may have it.
= escapeHtml($reason) ?>
Failed = $job['last_error_at'] ? $ago($job['last_error_at']) : 'earlier' ?>= $failing ? '' : ' - it has run since' ?>
= escapeHtml($job['last_error']) ?>
|
|||||
Everything on this page runs from one line in the server's crontab - that of the user that owns ITFlow's files= $owner_note ?>:
= $command_block() ?>It runs every minute and works out which jobs are due, so changing a job's schedule here never means changing it. On a hosting control panel, it's a cron job that runs every minute.
Stops every job at once, without touching their schedules. It's for a restored backup or a test copy, which would otherwise email your clients and charge their cards. Turning it back on picks up exactly where this leaves off.
Turn cron off