diff --git a/CHANGELOG.md b/CHANGELOG.md index 6cce6dafb..0445e4052 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ This file documents all notable changes made to ITFlow. ### Upgrading to 26.11 -Update from Maintenance > Update — Queue Update hands the job to cron, which updates the files and applies this release's database changes (2.7.9 to 3.3.6) in the same pass. Nothing else is required. +Update from Maintenance > Update — Queue Update hands the job to cron, which updates the files and applies this release's database changes (2.7.9 to 3.3.8) in the same pass. Nothing else is required. This is a big release, with fifty-seven database steps in one go, so take a backup before you update. @@ -55,6 +55,8 @@ Vendor templates keep everything they hold, and the vendors made from them stay The app logs are kept for as many days as the audit log, as before, and each can now be set on its own in Admin > Settings > Logs, which is where the log retention went from Settings > Security. A retention of 0 now keeps a log forever (see Breaking Changes, Logs). Nothing is sent to a syslog server or SIEM until you set one up there. +AI providers keep their address, key and models. Each is given a kind from where it points - OpenAI, Anthropic, Azure OpenAI, Google Gemini, Mistral, Groq, OpenRouter, Ollama or LM Studio - or Other (OpenAI-compatible), and passwords and keys are taken out of what is sent to each from now on, which can be turned off for a provider in its Edit window. Every role keeps using AI, through a new Use AI permission in Admin > Roles that can now be taken away from a role. A model's prompt is now its instructions, sent after what each feature asks for (see Breaking Changes, AI). AI Usage starts empty. + If you are coming from 26.08 or earlier, the database update still has to be run from the command line one time, exactly as described under Upgrading to 26.09 below. ### Breaking Changes and Notes @@ -223,6 +225,12 @@ If you are coming from 26.08 or earlier, the database update still has to be run - The `auth_logs` table is removed. Nothing was ever written to it. - Logs: log retention moved from Admin > Settings > Security to Admin > Settings > Logs, where the audit log (with API key use) and the app logs each keep their own number of days. A retention of 0 now keeps a log forever - it used to delete everything older than today each night. - Audit log: each thing done now has one name, in the entries already there as well as new ones. Unarchive and Restored are Restore, Deleted is Delete, and Closed, Resolved, Reopened, Merged and Sent are Close, Resolve, Reopen, Merge and Send. An agent turned away from a client they can't open is Denied, where it was Access. A saved card or payment provider deleted is Delete and a card saved is Create, where both were Update, and a contract template changed is Edit. A report or script that reads the `logs` table by the old names needs the new ones. +- AI: + - Using AI is a permission of its own, Use AI, in Admin > Roles. Every role has it after the update; a role without it gets no Reword button, no ticket summary and no document template draft. + - The Reword button is only on the editors when a model is set up for it and the role may use AI. It used to be on every editor, and say an error when pressed. + - A model's prompt is now its instructions - your house style, sent after what each feature asks of the model. Reword says what rewording is itself now, so a prompt written to make Reword work still works but can be cut down to the style you want. + - Passwords, PINs, API keys, private keys, 2FA links and card numbers in what is sent are replaced with placeholders, and put back in the answer. A provider can be set to send them as they are, in its Edit window. + - A provider's address is its chat completions address, as before - one ending /v1 now has /chat/completions put on when saved. ### New Features & Updates @@ -1046,6 +1054,18 @@ If you are coming from 26.08 or earlier, the database update still has to be run - Removing a card says first which recurring invoices stop autopaying with it. It is taken off Stripe as well, each of those recurring invoices says in its history that its autopay stopped and why, and when Stripe can't be reached the card is still removed and you are told it is saved there. - What goes through it: each provider shows its card payments in the last 30 days and when the last came, what its fees came to, and how many are still to record - flagged once they are more than two days old, as the nightly run should have had them. Record fees now does what the nightly run does, and says what it recorded, what Stripe hasn't settled yet and what it wouldn't say. Check connection asks Stripe whose account the key is, and says so - or why it isn't connected. - Settings > Invoices and billing says how card payments stand, under Paid online: none set up, turned off, or on - with live or test keys, where payments go, the card limit and how many came in the last 30 days. The Settings overview flags test keys, card payments going into an archived account, and Stripe's fees not being recorded, and the Invoices and billing tile says whether cards are on. +- **AI Providers** (rebuilt) + - Admin > AI Providers lists each provider with its kind and address, whether its key is saved (it is never shown), whether it runs on your own network, whether passwords and keys go to it as they are, its models and what each is for, and its requests and tokens in the last 30 days, with how many failed. Above the list, which model General, Tickets and Documentation each use - their own, or General's. Search finds a provider by a model on it too, and the list sorts by name, kind, models, requests or when it was last used. + - Add and Edit are one window. Pick its kind - OpenAI, Anthropic (Claude), Azure OpenAI, Google Gemini, Mistral, Groq, OpenRouter, Ollama, LM Studio, or any other OpenAI-compatible service - and its address and name are filled in, with a link to where its keys are made. Test asks it for its models: it says it answered and how many there are, or why not - a key it turned away, an address that isn't there or doesn't answer - beside the field. Save checks first the same way, and a provider that doesn't answer right now (one on your network that is off, say) can still be saved, once it has said so. The key is no longer put into the window; left empty it stays, and Remove the saved key takes it off. + - Adding the first provider asks for the model to use, with the ones it has suggested as you type, so one window sets AI up for everything. + - Test on a provider's menu says whether it answers with its key, and names any model set up on it that it doesn't have. Delete says which models go with it and which features stop. + - Admin > AI Models: each model with its provider, what it is for, its instructions and temperature, and its use in the last 30 days. A second model for a use case is marked Not used, as only the first is, and adding one says so. Filter by provider; sort by name, provider, use case or requests. + - A model's window suggests the models its provider has as you type, says what the use case picked has now - that this one will be used for it, or that another is and this one won't be - and Try it sends the model a one-line request as it is in the window, unsaved, and shows what came back and how long it took. Its instructions go with every request it handles: your house style, things to always or never say. + - Admin > AI Usage: every request - when, the feature (Reword, Reword a ticket reply, Ticket summary, Document template draft, Test), who asked, the client it was for, the model and provider, the tokens it took, how long, how many secrets were kept out, and why it failed when it did. Totals for what is shown above the list - requests, tokens in and out, failed, secrets kept out and answer time. Tabs for the failed ones, the ones secrets were kept out of, and tests; filter by date, provider, model, feature and who; search by any of them or the client. Kept as long as the audit log. + - Keeping secrets out: passwords, PINs, API keys (OpenAI, Anthropic, Stripe, AWS, GitHub, GitLab, Slack, Google), JWTs, private keys, 2FA setup links and card numbers in a reply or ticket are taken out before it goes to the provider, as [secret-1] and so on, and put back in the answer - the provider never sees them, and AI Usage counts them. On by default for every provider; it can be turned off for one you run yourself. + - Use AI permission in Admin > Roles, so a role can be kept from AI altogether. + - The Settings overview says when AI requests have failed in the last day and are still failing, with the provider's reason. + - Reword is the first button on the editors where it is there, says when there is nothing to reword yet, keeps your formatting, placeholders, links and names, and says Undo puts your words back - which it does, in one step. It works on the admin pages' editors too. ### Bug Fixes @@ -1207,6 +1227,10 @@ If you are coming from 26.08 or earlier, the database update still has to be run - Settings > Notifications, and recurring invoices: the note that recurring invoices are made as drafts pointed to the billing settings, but the switch is in Notifications. - Mail Queue: cancelling an email marked it Failed, and an email to a bad address or with no from address showed Failed with no time. - Mail Queue: Resend was only offered once all four tries were used up, and the next failure quietly gave a resent email three more. +- AI: Reword on the admin pages' editors (document templates and the like) always failed - it asked a page that isn't there. +- AI: the AI provider window showed its API key; adding or changing one said "AI Model ... created" or "edited"; and the model window picked the last provider in the list whichever one the model was on. +- AI: with no prompt on its model, Reword sent the text with nothing to say what to do with it. +- AI: a ticket summary and a document template draft went into the page as the model wrote them, so something in a ticket could get a script into the summary. They go through the same clean-up as documents now. - `setup_cli.php`: a name with an apostrophe in it stopped the install part-way, a password with a space at either end couldn't sign in, and it wrote config.php before knowing the tables could be made - over a database that already had tables in it, too. ### Developer Updates @@ -1779,7 +1803,11 @@ If you are coming from 26.08 or earlier, the database update still has to be run - Mail Queue: new `functions/mail_queue.php` - `MAIL_QUEUE_TRIES` (4), `MAIL_QUEUE_RETRY_MINUTES` (30) and `MAIL_QUEUE_KEEP_DAYS` (90), which `cron/mail_queue.php`'s retry pass and the nightly clean-up now use too; `mailQueueTabs()`, `mailQueueState($row)`, `mailQueueCan($row, $action)`, `mailQueueAct($action, $ids)` and `mailQueueHealth()`. `email_status` 4 is Cancelled - never selected by the worker. Retry now sets status 0, no attempts and no failure time. Handlers: `retry_mail`, `cancel_mail`, `delete_mail` (GET with the CSRF token) and `bulk_retry_emails`, `bulk_cancel_emails`, `bulk_delete_emails`; `send_failed_mail` still works. The worker sets `email_failed_at` on the bad sender, bad recipient and no-MX paths. - Mail Queue: database 3.3.7 adds `email_queue.email_error` varchar(500) - why it last failed, or "Cancelled by ..."; cleared when sent or retried. The worker writes it on every failure path, through `mailQueueErrorSql()`, which leaves it out until the update has run, so an install between updating its files and its database still sends; `mailQueueErrorColumn()` does the same for reads. The retry pass's bad-recipient path now sets `email_failed_at`, so its backoff counts from that try. An alert (`appNotify`) when the retry pass gives up. The email's window shows the queued HTML in an iframe `srcdoc` with `sandbox="allow-popups allow-popups-to-escape-sandbox"` (no scripts, no same origin) and a `script-src 'none'` policy, instead of HTMLPurifier in the page. - Mail Queue: `mailQueueSummary()` - waiting, retrying, failed in the last day and week, and the last sent - read once a request, for the Settings overview (attention items and the Mail tile), Mail's Tests tab and the admin menu's count (`admin/includes/side_nav.php`). `globalSearchPages()` has Mail Queue for admins. -- `js/app.js`: the four `tinymce.init()` calls go through `itflowTinyMce`, which does nothing where TinyMCE isn't loaded. +- AI: database 3.3.8 adds `ai_providers.ai_provider_kind` varchar(50) (guessed from the address for those already there, else `custom`) and `ai_provider_redact` tinyint(1) default 1, widens `ai_provider_api_url` and `ai_provider_api_key` to varchar(500), adds the `ai_usage` table (feature, use case, provider and model ids and names, user, client, tokens in and out, ms, redacted, ok, error, created at) and the `module_ai` module, given level 1 on every non-admin role. `setup/seed_data.php` and `db.sql` match. +- AI: `functions/ai.php` - `aiProviderKinds()` (address, key, how it is sent - bearer or Azure's `api-key` - how its models are listed, local or not, where keys are made, an example model), `aiProviderKind()`, `aiUseCases()`, `aiFeatures()`, `aiCan()` (the `module_ai` permission), `aiReady($use_case)`, `aiClientConfig()`. `getAiModel()` joins the provider (an `INNER JOIN` now), returns its id, kind and redact with the model's id, and is cached per request. `callAiApi($model, $messages, $context)` takes `feature` and `client_id`, sends the key as the kind takes it, takes secrets out with `aiRedact()` and puts them back with `aiRestore()`, puts the model's prompt after the feature's system message as the company's instructions, and records every request with `aiUsageRecord()` (until 3.3.8 has run, `aiUsageTableExists()` says so and nothing is recorded or read from the new columns). `aiCleanHtml()` takes code fences off and runs `documentHtmlPurify()`. Also `aiUsageTotals()`, `aiTokensText()`, `aiTemperatureText()`, `aiModelsUrl()`, `aiProviderTest($kind, $url, $key)` (the models it lists; Anthropic's own `/v1/models` with `x-api-key`), `aiProviderUrl()` (a base address gets `/chat/completions`), `aiProviderRow()`, `aiModelRow()`, `aiProviderCheck()`, `aiProviderConnectionChanged()`, `aiProviderWrite()` and `aiModelWrite()` (raw values, escaped with `mysqli_real_escape_string`, not `escapeSql()`), `aiProviderChangeLines()`, `aiModelCheck()`, `aiModelChangeLines()`, `aiUseCaseMap($without)`, `aiUseCasesStopping()`, `aiProviderIcon()`, `aiUseCaseStripHtml()` and `aiProblems()`. +- AI: handlers - `check_ai_provider` (JSON: `ok`, `errors` by field, `soft` when only not answering, `models`, `listed`, `url` as it will be saved; `test` to ask even when unchanged) before `add_ai_provider` / `edit_ai_provider`, which take `kind`, `name`, `url`, `api_key`, `clear_key`, `redact`, `save_anyway` and, adding the first, `first_model`; `ai_provider_models` (JSON, cached ten minutes); `test_ai_provider` (GET with the CSRF token); `check_ai_model` (JSON, `try` to send a one-line request as feature `test`) before `add_ai_model` / `edit_ai_model`. `admin/modals/ai/ai_provider_add.php` and `ai_model_add.php` are gone - the edit windows add with no id, and `ai_model_edit.php?provider=` picks the provider. New page `admin/ai_usage.php`. +- AI: `agent/ajax.php` - `ai_reword`, `ai_ticket_summary` and `ai_create_document_template` need `module_ai` (a JSON 403 otherwise), pass their feature and client to `callAiApi()`, and answer purified HTML. `ai_reword` takes `client_id` in its JSON and has its own instructions, by use case. The nightly run deletes `ai_usage` rows older than the audit log's retention. +- `js/app.js`: the four `tinymce.init()` calls go through `itflowTinyMce`, which does nothing where TinyMCE isn't loaded. `itflowAiOn(ticket)` and `itflowAiReword(editor, ticket)` add Reword, first in the toolbar, only when `window.itflowAi` (printed by `includes/footer.php` from `aiClientConfig()`) says so, posting to its `endpoint`, `/agent/ajax.php`. A textarea's `data-ai-client-id` goes with the request. ### Library Updates diff --git a/admin/ai_models.php b/admin/ai_models.php index ecea6371e..04d00c96b 100644 --- a/admin/ai_models.php +++ b/admin/ai_models.php @@ -1,131 +1,210 @@ AI Models - the models on the AI providers, each for a use case + * (functions/ai.php aiUseCases()) with General the fallback. Above, which + * model each use case lands on; in the list, a second model for a use case is + * marked as not used, the provider each is on, its instructions and + * temperature, and its use in the last 30 days. Add and Edit are one window + * (modals/ai/ai_model_edit.php), with Try it; the rest are + * admin/post/ai_models.php. + */ + // Default Column Sortby Filter $sort = "ai_model_name"; $order = "ASC"; require_once "includes/inc_all_admin.php"; -$sql = mysqli_query($mysqli, "SELECT SQL_CALC_FOUND_ROWS ai_model_id, ai_model_name, ai_model_prompt, ai_model_use_case, ai_provider_id, - ai_provider_name FROM ai_models LEFT JOIN ai_providers ON ai_model_ai_provider_id = ai_provider_id - WHERE ai_model_name LIKE '%$q%' OR ai_provider_name LIKE '%$q%' - ORDER BY $sort $order LIMIT $record_from, $record_to"); +$page_title = 'AI Models'; + +// Only these sort - an unknown column would take the page down +$sortable_columns = [ + 'ai_model_name' => "ai_model_name $order", + 'ai_provider_name' => "ai_provider_name $order, ai_model_name ASC", + 'ai_model_use_case' => "FIELD(ai_model_use_case, 'General', 'Tickets', 'Documentation') " . ($order === 'ASC' ? 'ASC' : 'DESC') . ", ai_model_id ASC", + 'requests' => "requests $order, ai_model_name ASC", +]; +if (!isset($sortable_columns[$sort])) { + $sort = 'ai_model_name'; +} + +$provider_filter = intval($_GET['provider'] ?? 0); + +$sql = mysqli_query($mysqli, "SELECT SQL_CALC_FOUND_ROWS ai_model_id, ai_model_name, ai_model_prompt, ai_model_use_case, ai_model_temperature, + ai_provider_id, ai_provider_name, ai_provider_kind, + COALESCE(used.requests, 0) AS requests, COALESCE(used.failed, 0) AS failed, COALESCE(used.tokens_in, 0) AS tokens_in, COALESCE(used.tokens_out, 0) AS tokens_out, used.last_used + FROM ai_models + JOIN ai_providers ON ai_model_ai_provider_id = ai_provider_id + LEFT JOIN ( + SELECT ai_usage_model_id, + SUM(ai_usage_created_at >= NOW() - INTERVAL 30 DAY) AS requests, + SUM(ai_usage_created_at >= NOW() - INTERVAL 30 DAY AND ai_usage_ok = 0) AS failed, + SUM(IF(ai_usage_created_at >= NOW() - INTERVAL 30 DAY, ai_usage_tokens_in, 0)) AS tokens_in, + SUM(IF(ai_usage_created_at >= NOW() - INTERVAL 30 DAY, ai_usage_tokens_out, 0)) AS tokens_out, + MAX(ai_usage_created_at) AS last_used + FROM ai_usage WHERE ai_usage_feature <> 'test' GROUP BY ai_usage_model_id + ) used ON used.ai_usage_model_id = ai_model_id + WHERE (ai_model_name LIKE '%$q%' OR ai_provider_name LIKE '%$q%' OR ai_model_use_case LIKE '%$q%' OR ai_model_prompt LIKE '%$q%') + " . ($provider_filter ? "AND ai_provider_id = $provider_filter" : '') . " + ORDER BY {$sortable_columns[$sort]}, ai_model_id ASC + LIMIT $record_from, $record_to"); $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); +$unused = aiUseCaseMap()['unused']; +$providers = mysqli_query($mysqli, "SELECT ai_provider_id, ai_provider_name, ai_provider_kind FROM ai_providers ORDER BY ai_provider_name"); +$any_provider = mysqli_num_rows($providers) > 0; + +$sort_link = function ($column, $label) use ($url_query_strings_sort, $sort, $disp, $order_icon) { + return '' + . $label . ($sort === $column ? " $order_icon" : '') . ''; +}; + +$add_url = 'modals/ai/ai_model_edit.php' . ($provider_filter ? "?provider=$provider_filter" : ''); +$csrf = $_SESSION['csrf_token']; + ?>
    -
  1. - Admin -
    2. -
  2. - AI Providers -
    3. +
  3. Admin
    4. +
  4. AI Providers
  5. AI Models
-

AI Models

+

AI Models

- + Usage + + +
-
+
- - + +
+
+ +
+ +
+
- -
- - "> +
+
+ "> - - - - - + + + + + + - $model_id]); + $then = $is_unused ? null : aiUseCaseMap(['model' => $model_id])['map'][$use_case] ?? null; + if ($is_unused) { + $delete_text = 'Nothing uses it - ' . ($unused[$model_id] ? $unused[$model_id]['ai_model_name'] . " is the one used for $use_case." : "$use_case isn't a use case."); + } elseif ($stopping) { + $delete_text = 'AI stops for ' . implode(' and ', $stopping) . ' until another model is set up.'; + } else { + $delete_text = "$use_case uses " . ($then && $then['model'] ? $then['model']['ai_model_name'] . ($then['own'] ? '' : ' (General)') : 'nothing') . ' instead.'; + } ?> - + - - - - + + + + - - - +
- - Model - - - - Provider - - - - Use Case - - - Prompt - ActionInstructions
- - - + + + ">Not used + +
+ +
Temperature
+ 		-
- +
+ + +
+
+ 		+ +
+ + None + + 		+ + +
in · out
+ + + + +
+ + +
Last
+ + 		+
+
- - Edit + + Edit + + + Usage
- + Delete
'AI models', - 'icon' => 'fa-robot', - 'text' => 'The models ITFlow uses for ticket summaries and rewording - add a provider first.', - 'actions' => [['label' => 'Add Model', 'modal' => 'modals/ai/ai_model_add.php']], + 'icon' => 'fa-brain', + 'text' => $any_provider ? 'The model on each provider that ITFlow uses - one for everything (General), and others for tickets or documents if you like.' : 'Add an AI provider first - then the models on it ITFlow uses.', + 'actions' => $any_provider ? [['label' => 'Add Model', 'modal' => $add_url]] : [['label' => 'AI Providers', 'href' => 'ai_providers.php', 'icon' => 'fa-robot']], ]; ?> diff --git a/admin/ai_providers.php b/admin/ai_providers.php index 73fc5d268..251d6e1aa 100644 --- a/admin/ai_providers.php +++ b/admin/ai_providers.php @@ -1,113 +1,209 @@ AI Providers - the AI services ITFlow asks (functions/ai.php): a + * ready-made kind each (OpenAI, Claude, Gemini, Azure, a model on your own + * network...), its address, whether its key is saved (never shown), whether + * passwords and keys are taken out of what is sent to it, the models on it and + * what each is for, and its use in the last 30 days. Above, which model each + * use case lands on. Test asks it for its models; Add and Edit are one window + * (modals/ai/ai_provider_edit.php); the rest are admin/post/ai_provider.php. + */ + // Default Column Sortby Filter $sort = "ai_provider_name"; $order = "ASC"; require_once "includes/inc_all_admin.php"; -$sql = mysqli_query($mysqli, "SELECT SQL_CALC_FOUND_ROWS ai_provider_api_key, ai_provider_api_url, ai_provider_id, ai_provider_name FROM ai_providers - WHERE ai_provider_name LIKE '%$q%' OR ai_provider_api_url LIKE '%$q%' - ORDER BY $sort $order LIMIT $record_from, $record_to"); +// Only these sort - an unknown column would take the page down +$sortable_columns = [ + 'ai_provider_name' => "ai_provider_name $order", + 'ai_provider_kind' => "ai_provider_kind $order, ai_provider_name ASC", + 'model_count' => "model_count $order, ai_provider_name ASC", + 'requests' => "requests $order, ai_provider_name ASC", + 'last_used' => "last_used IS NULL, last_used $order, ai_provider_name ASC", +]; +if (!isset($sortable_columns[$sort])) { + $sort = 'ai_provider_name'; +} + +$sql = mysqli_query($mysqli, "SELECT SQL_CALC_FOUND_ROWS ai_provider_id, ai_provider_name, ai_provider_kind, ai_provider_api_url, ai_provider_redact, + ai_provider_api_key IS NOT NULL AND ai_provider_api_key <> '' AS has_key, + (SELECT COUNT(*) FROM ai_models WHERE ai_model_ai_provider_id = ai_provider_id) AS model_count, + COALESCE(used.requests, 0) AS requests, COALESCE(used.failed, 0) AS failed, COALESCE(used.tokens_in, 0) AS tokens_in, COALESCE(used.tokens_out, 0) AS tokens_out, used.last_used + FROM ai_providers + LEFT JOIN ( + SELECT ai_usage_provider_id, + SUM(ai_usage_created_at >= NOW() - INTERVAL 30 DAY) AS requests, + SUM(ai_usage_created_at >= NOW() - INTERVAL 30 DAY AND ai_usage_ok = 0) AS failed, + SUM(IF(ai_usage_created_at >= NOW() - INTERVAL 30 DAY, ai_usage_tokens_in, 0)) AS tokens_in, + SUM(IF(ai_usage_created_at >= NOW() - INTERVAL 30 DAY, ai_usage_tokens_out, 0)) AS tokens_out, + MAX(ai_usage_created_at) AS last_used + FROM ai_usage WHERE ai_usage_feature <> 'test' GROUP BY ai_usage_provider_id + ) used ON used.ai_usage_provider_id = ai_provider_id + WHERE (ai_provider_name LIKE '%$q%' OR ai_provider_kind LIKE '%$q%' OR ai_provider_api_url LIKE '%$q%' + OR EXISTS (SELECT 1 FROM ai_models WHERE ai_model_ai_provider_id = ai_provider_id AND ai_model_name LIKE '%$q%')) + ORDER BY {$sortable_columns[$sort]}, ai_provider_id ASC + LIMIT $record_from, $record_to"); $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); +// Each provider's models, with what they are for - and those nothing uses +$models_by_provider = []; +$model_sql = mysqli_query($mysqli, "SELECT ai_model_id, ai_model_name, ai_model_use_case, ai_model_ai_provider_id FROM ai_models ORDER BY FIELD(ai_model_use_case, 'General', 'Tickets', 'Documentation'), ai_model_name"); +while ($model = mysqli_fetch_assoc($model_sql)) { + $models_by_provider[intval($model['ai_model_ai_provider_id'])][] = $model; +} +$unused = aiUseCaseMap()['unused']; +$any_provider = intval(mysqli_fetch_row(mysqli_query($mysqli, "SELECT COUNT(*) FROM ai_providers"))[0]) > 0; + +$sort_link = function ($column, $label) use ($url_query_strings_sort, $sort, $disp, $order_icon) { + return '' + . $label . ($sort === $column ? " $order_icon" : '') . ''; +}; + +$add_url = 'modals/ai/ai_provider_edit.php'; +$csrf = $_SESSION['csrf_token']; + ?>

AI Providers

- + Models + Usage +
-
+
- - + +
+ +
+
- -
- - "> +
+
+ "> - - - - - + + + + + + - $provider_id]); + $model_names = array_column($models, 'ai_model_name'); + $delete_text = ($models ? (count($models) === 1 ? "Its model, {$model_names[0]}, goes with it." : 'Its ' . count($models) . ' models go with it - ' . implode(', ', $model_names) . '.') : 'It has no models.') + . ($stopping ? ' AI stops for ' . implode(' and ', $stopping) . ' until another model is set up.' : ''); ?> - + - - - + + + + - - - +
- - Provider - - - - URL - - - - Key - - - Models - ActionKey
- - - +
+ +
+ + + Your network + + + Secrets sent + +
+ +
+
+
- - -
- +
+ + Saved + + Needs one - add it in Edit + + None + + + +
+ + · + + Not used + +
+ + + Add a model + + 		+ + +
in · out
+ + + + +
+ + +
Last
+ + 		+
+
- - Edit + + Edit + + + Test + + + Add a model + + + Usage
- + Delete
@@ -115,8 +211,8 @@ $num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); $empty_state = [ 'thing' => 'AI providers', 'icon' => 'fa-robot', - 'text' => 'Connect an AI service, then pick the models ITFlow uses from it.', - 'actions' => [['label' => 'Add Provider', 'modal' => 'modals/ai/ai_provider_add.php']], + 'text' => 'Connect OpenAI, Claude, Gemini, Azure OpenAI, or a model on your own network like Ollama - then agents can reword replies, summarize tickets and draft documents, with passwords and keys taken out of what is sent.', + 'actions' => [['label' => 'Add Provider', 'modal' => $add_url]], ]; ?> diff --git a/admin/ai_usage.php b/admin/ai_usage.php new file mode 100644 index 000000000..6d25a213c --- /dev/null +++ b/admin/ai_usage.php @@ -0,0 +1,276 @@ + AI Usage - every request made to an AI provider (ai_usage, + * recorded by callAiApi() in functions/ai.php): when, the feature, who asked, + * the client it was about, the model and provider, the tokens it took, how + * long, how many passwords and keys were taken out of it first, and whether + * it worked - why not, when it didn't. Totals for what is shown sit above. + * Searched, and filtered by date, provider, model, feature and who; views for + * the failed ones, the ones secrets were taken out of, and tests. Kept as long + * as the audit log is (Admin > Settings > Logs). + */ + +// Default Column Sortby Filter +$sort = "ai_usage_id"; +$order = "DESC"; + +require_once "includes/inc_all_admin.php"; + +$page_title = 'AI Usage'; + +// Only these sort - an unknown column would take the page down +$sortable_columns = [ + 'ai_usage_id' => "ai_usage_id $order", + 'ai_usage_feature' => "ai_usage_feature $order, ai_usage_id DESC", + 'user_name' => "user_name IS NULL, user_name $order, ai_usage_id DESC", + 'client_name' => "client_name IS NULL, client_name $order, ai_usage_id DESC", + 'ai_usage_model' => "ai_usage_model $order, ai_usage_id DESC", + 'tokens' => "(ai_usage_tokens_in + ai_usage_tokens_out) $order, ai_usage_id DESC", + 'ai_usage_ms' => "ai_usage_ms $order, ai_usage_id DESC", +]; +if (!isset($sortable_columns[$sort])) { + $sort = 'ai_usage_id'; +} + +$features = aiFeatures(); +$views = [ + '' => ['label' => 'All', 'icon' => 'fa-list', 'where' => "ai_usage_feature <> 'test'", 'title' => 'Every request but tests'], + 'failed' => ['label' => 'Failed', 'icon' => 'fa-times-circle', 'where' => "ai_usage_feature <> 'test' AND ai_usage_ok = 0", 'title' => 'The provider turned them away, or couldn\'t be reached'], + 'redacted' => ['label' => 'Secrets kept out', 'icon' => 'fa-user-secret', 'where' => "ai_usage_redacted > 0", 'title' => 'Passwords or keys were taken out before they were sent'], + 'test' => ['label' => 'Tests', 'icon' => 'fa-vial', 'where' => "ai_usage_feature = 'test'", 'title' => 'Try it, from a model\'s window'], +]; +$view = is_string($_GET['view'] ?? null) && isset($views[$_GET['view']]) ? $_GET['view'] : ''; +$provider_filter = intval($_GET['provider'] ?? 0); +$model_filter = intval($_GET['model'] ?? 0); +$user_filter = intval($_GET['user'] ?? 0); +$feature_filter = is_string($_GET['feature'] ?? null) && isset($features[$_GET['feature']]) ? $_GET['feature'] : ''; +$from = preg_match('/^\d{4}-\d{2}-\d{2}$/', (string) $dtf) ? $dtf : '1970-01-01'; +$to = preg_match('/^\d{4}-\d{2}-\d{2}$/', (string) $dtt) ? $dtt : '2099-12-31'; + +// Everything but the view - the views are counted with it +$where = ["DATE(ai_usage_created_at) BETWEEN '$from' AND '$to'"]; +if ($q !== '') { + $where[] = "(ai_usage_model LIKE '%$q%' OR ai_usage_provider_name LIKE '%$q%' OR ai_usage_feature LIKE '%$q%' OR ai_usage_use_case LIKE '%$q%' + OR user_name LIKE '%$q%' OR client_name LIKE '%$q%' OR ai_usage_error LIKE '%$q%')"; +} +if ($provider_filter) { + $where[] = "ai_usage_provider_id = $provider_filter"; +} +if ($model_filter) { + $where[] = "ai_usage_model_id = $model_filter"; +} +if ($user_filter) { + $where[] = "ai_usage_user_id = $user_filter"; +} +if ($feature_filter !== '') { + $where[] = "ai_usage_feature = '" . escapeSql($feature_filter) . "'"; +} +$where_sql = implode(' AND ', $where); +$from_sql = "FROM ai_usage LEFT JOIN users ON user_id = ai_usage_user_id LEFT JOIN clients ON client_id = ai_usage_client_id"; + +$sql = mysqli_query($mysqli, "SELECT SQL_CALC_FOUND_ROWS ai_usage.*, user_name, client_name + $from_sql + WHERE $where_sql AND {$views[$view]['where']} + ORDER BY {$sortable_columns[$sort]} + LIMIT $record_from, $record_to"); +$num_rows = mysqli_fetch_row(mysqli_query($mysqli, "SELECT FOUND_ROWS()")); + +// Each view counted for the rest of the filters, and the totals for this one +$count_sql = []; +foreach ($views as $definition) { + $count_sql[] = "COALESCE(SUM({$definition['where']}), 0)"; +} +$counts = array_combine(array_keys($views), array_map('intval', mysqli_fetch_row(mysqli_query($mysqli, "SELECT " . implode(', ', $count_sql) . " $from_sql WHERE $where_sql")))); +$totals = mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT COUNT(*) AS requests, COALESCE(SUM(ai_usage_tokens_in), 0) AS tokens_in, COALESCE(SUM(ai_usage_tokens_out), 0) AS tokens_out, + COALESCE(SUM(ai_usage_ok = 0), 0) AS failed, COALESCE(SUM(ai_usage_redacted), 0) AS redacted, COALESCE(AVG(IF(ai_usage_ok = 1, ai_usage_ms, NULL)), 0) AS ms, COUNT(DISTINCT ai_usage_user_id) AS people + $from_sql WHERE $where_sql AND {$views[$view]['where']}")); + +$providers = mysqli_query($mysqli, "SELECT ai_provider_id, ai_provider_name, ai_provider_kind FROM ai_providers ORDER BY ai_provider_name"); +$people = mysqli_query($mysqli, "SELECT DISTINCT user_id, user_name FROM ai_usage JOIN users ON user_id = ai_usage_user_id ORDER BY user_name"); +$model_row = $model_filter ? aiModelRow($model_filter) : null; +$retention = appLogRetention(); + +$sort_link = function ($column, $label) use ($url_query_strings_sort, $sort, $disp, $order_icon) { + return '' + . $label . ($sort === $column ? " $order_icon" : '') . ''; +}; +// A view keeps the filters, and starts from its first page +$view_url = function ($key) { + $query = $_GET; + unset($query['page']); + $query['view'] = $key; + return '?' . http_build_query(array_filter($query, function ($value) { + return $value !== '' && $value !== null && $value !== []; + })); +}; +// The page without one filter +$without_url = function ($key) { + $query = $_GET; + unset($query['page'], $query[$key]); + return '?' . http_build_query($query); +}; +$ms_text = function ($ms) { + $ms = intval($ms); + return $ms >= 1000 ? number_format($ms / 1000, 1) . 's' : $ms . 'ms'; +}; + +?> + +
    +
  1. Admin
    2. +
  2. AI Providers
    3. +
  3. AI Usage
    4. +
+ +
+
+

AI Usage

+
+ Models + Keeping +
+
+
+
+ $view, 'model' => $model_filter ?: ''] as $keep => $value) { ?> + + +
+
+
+ + +
+
+
+ + + + +
+
+ +
+
+ +
+
+ +
+
+
+ +
+ Just + +
+ + + +
+
Requests
+
Tokens in
+
Tokens out
+
Failed
+
Secrets kept out
+
Answer time
+
+ +
+
+ + "> + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+ 		+
+ +
+ + +
secret kept out
+ +
+ 		' . (intval($row['ai_usage_user_id']) ? 'Removed' : 'ITFlow') . '' ?> + + + + - + + +
+
+ 		+ +
in
+
out
+ + - + +
+
+ 'AI requests', + 'icon' => 'fa-chart-bar', + 'views' => ['view'], + 'view' => $view !== '' ? ['icon' => $views[$view]['icon'], 'title' => 'None ' . ($view === 'test' ? 'tried yet' : 'yet'), 'text' => escapeHtml($views[$view]['title']) . ' - they show here.'] : null, + 'text' => 'Each time an agent rewords a reply, summarizes a ticket or drafts a document, it shows here - who, for what, the model and the tokens it took. ' . ($retention['audit'] ? 'Kept for ' . escapeHtml(appLogRetentionText($retention['audit'])) . ', as the audit log is.' : 'Kept as long as the audit log is.'), + ]; + ?> + +
+ + 0; + }; + + // AI providers - which kind each is (how it is called and its models listed), + // and whether passwords and keys are taken out of what is sent to it. Room for + // the longer keys and Azure's longer addresses. + if (!$itflow_column('ai_providers', 'ai_provider_kind')) { + mysqli_query($mysqli, "ALTER TABLE `ai_providers` ADD `ai_provider_kind` varchar(50) NOT NULL DEFAULT 'custom' AFTER `ai_provider_name`"); + // Those set up already, by where they point + foreach ([ + 'api.openai.com' => 'openai', 'openai.azure.com' => 'azure', 'api.anthropic.com' => 'anthropic', 'generativelanguage.googleapis.com' => 'gemini', + 'api.mistral.ai' => 'mistral', 'api.groq.com' => 'groq', 'openrouter.ai' => 'openrouter', ':11434' => 'ollama', ':1234' => 'lmstudio', + ] as $itflow_host => $itflow_kind) { + mysqli_query($mysqli, "UPDATE `ai_providers` SET `ai_provider_kind` = '$itflow_kind' WHERE `ai_provider_kind` = 'custom' AND `ai_provider_api_url` LIKE '%$itflow_host%'"); + } + } + if (!$itflow_column('ai_providers', 'ai_provider_redact')) { + mysqli_query($mysqli, "ALTER TABLE `ai_providers` ADD `ai_provider_redact` tinyint(1) NOT NULL DEFAULT 1 AFTER `ai_provider_api_key`"); + } + mysqli_query($mysqli, "ALTER TABLE `ai_providers` MODIFY `ai_provider_api_url` varchar(500) NOT NULL, MODIFY `ai_provider_api_key` varchar(500) DEFAULT NULL"); + + // Each AI request - what it was for, who asked, the model, tokens and how long it took + mysqli_query($mysqli, "CREATE TABLE IF NOT EXISTS `ai_usage` ( + `ai_usage_id` int(11) NOT NULL AUTO_INCREMENT, + `ai_usage_feature` varchar(50) NOT NULL, + `ai_usage_use_case` varchar(50) DEFAULT NULL, + `ai_usage_provider_id` int(11) NOT NULL DEFAULT 0, + `ai_usage_provider_name` varchar(200) DEFAULT NULL, + `ai_usage_model_id` int(11) NOT NULL DEFAULT 0, + `ai_usage_model` varchar(200) DEFAULT NULL, + `ai_usage_user_id` int(11) NOT NULL DEFAULT 0, + `ai_usage_client_id` int(11) NOT NULL DEFAULT 0, + `ai_usage_tokens_in` int(11) NOT NULL DEFAULT 0, + `ai_usage_tokens_out` int(11) NOT NULL DEFAULT 0, + `ai_usage_ms` int(11) NOT NULL DEFAULT 0, + `ai_usage_redacted` int(11) NOT NULL DEFAULT 0, + `ai_usage_ok` tinyint(1) NOT NULL DEFAULT 1, + `ai_usage_error` varchar(255) DEFAULT NULL, + `ai_usage_created_at` datetime NOT NULL DEFAULT current_timestamp(), + PRIMARY KEY (`ai_usage_id`), + KEY `ai_usage_created_at` (`ai_usage_created_at`), + KEY `ai_usage_provider_id` (`ai_usage_provider_id`), + KEY `ai_usage_model_id` (`ai_usage_model_id`) + ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci"); + + // Using AI is a permission of its own - every role keeps it, as anyone could use it before + $itflow_module = mysqli_fetch_row(mysqli_query($mysqli, "SELECT module_id FROM modules WHERE module_name = 'module_ai' LIMIT 1")); + if (!$itflow_module) { + mysqli_query($mysqli, "INSERT INTO modules SET module_name = 'module_ai', module_description = 'Using AI - rewording, summaries and drafts'"); + $itflow_module_id = intval(mysqli_insert_id($mysqli)); + mysqli_query($mysqli, "INSERT INTO user_role_permissions (user_role_id, module_id, user_role_permission_level) + SELECT role_id, $itflow_module_id, 1 FROM user_roles WHERE role_is_admin = 0"); + } + + unset($itflow_column, $itflow_host, $itflow_kind, $itflow_module, $itflow_module_id); diff --git a/admin/includes/side_nav.php b/admin/includes/side_nav.php index a8e0445b4..c7024785d 100644 --- a/admin/includes/side_nav.php +++ b/admin/includes/side_nav.php @@ -102,7 +102,7 @@
  • + class="nav-link ">

    AI Providers

     diff --git a/admin/modals/ai/ai_model_add.php b/admin/modals/ai/ai_model_add.php deleted file mode 100644 index d6053a065..000000000 --- a/admin/modals/ai/ai_model_add.php +++ /dev/null @@ -1,78 +0,0 @@ - - -
    -
    Add AI Model
    - -
    -
    - - -
    - -
    - -
    - - -
    -
    - -
    - -
    - - -
    -
    - -
    - -
    - - -
    -
    - -
    - -
    - - -
    - Optional. Leave blank to let the provider use its default - some newer models reject every other value. -
    - -
    - -
    - -
    -
    - - -
    -
    - - AI Models - one model's window: Add (no id - ?provider= picks its + * provider) or Edit. The model's name, suggested from what its provider lists; + * what it is for, saying what that use case uses now; its temperature; and + * the company's instructions that go with every request it handles. Try it + * sends it a one-line request as it is in the window, unsaved. Save asks first + * (check_ai_model, admin/post/ai_models.php), so what is wrong shows beside + * each field. + */ + require_once '../../includes/modal_header.php'; -$model_id = intval($_GET['id']); +$model_id = intval($_GET['id'] ?? 0); +$row = $model_id ? aiModelRow($model_id) : null; +if ($model_id && !$row) { + echo json_encode(['error' => 'That AI model no longer exists.']); + exit; +} -$sql = mysqli_query($mysqli, "SELECT ai_model_ai_provider_id, ai_model_id, ai_model_name, ai_model_prompt, ai_model_use_case, ai_model_temperature FROM ai_models WHERE ai_model_id = $model_id LIMIT 1"); +$provider_id = $row ? intval($row['ai_model_ai_provider_id']) : intval($_GET['provider'] ?? 0); +$providers = []; +$sql = mysqli_query($mysqli, "SELECT ai_provider_id, ai_provider_name, ai_provider_kind FROM ai_providers ORDER BY ai_provider_name"); +while ($provider = mysqli_fetch_assoc($sql)) { + $providers[intval($provider['ai_provider_id'])] = $provider; +} +if (!isset($providers[$provider_id])) { + $provider_id = count($providers) === 1 ? array_key_first($providers) : 0; +} +$use_cases = aiUseCases(); +$map = aiUseCaseMap(['model' => $model_id])['map']; +// A new one goes where there is nothing yet - General first +$use_case = $row ? (string) $row['ai_model_use_case'] : 'General'; +if (!$row) { + foreach ($use_cases as $case => $about) { + if (!$map[$case]['own']) { + $use_case = $case; + break; + } + } +} +// What each use case has now, without this one - so picking one says what it takes over, or that it won't be used +$taken = []; +foreach ($map as $case => $lands) { + $taken[$case] = $lands['own'] ? ['name' => $lands['model']['ai_model_name'], 'provider' => $lands['model']['ai_provider_name'], 'first' => intval($lands['model']['ai_model_id']) < ($model_id ?: PHP_INT_MAX)] : null; +} +$temperature = $row && $row['ai_model_temperature'] !== null ? rtrim(rtrim(number_format(floatval($row['ai_model_temperature']), 2, '.', ''), '0'), '.') : ''; -$row = mysqli_fetch_assoc($sql); -$ai_model_ai_provider_id = intval($row['ai_model_ai_provider_id']); -$model_id = intval($row['ai_model_id']); -$model_name = escapeHtml($row['ai_model_name']); -$use_case = escapeHtml($row['ai_model_use_case']); -$temperature = escapeHtml($row['ai_model_temperature']); -$prompt = escapeHtml($row['ai_model_prompt']); +$error_for = function ($field) { + return '
    '; +}; -// Generate the HTML form content using output buffering. ob_start(); -?> +?>
    -
    Editing:
    +
    ' . escapeHtml($row['ai_model_name']) . '' : 'Add AI Model' ?>
    -
    +
    -
    - -
    - - + + $provider) { ?> +
    +
    - -
    - - + +
    + +
    + + +
    As the provider names it.
    - -
    - - + $about) { ?> + + + + +
    + +
    - -
    - - -
    - Optional. Leave blank to let the provider use its default - some newer models reject every other value. + + + +
    Optional - your house style, sent with everything this model is asked, after what each feature asks of it.
    +
    - + +
    + + +
    + +
    Leave it empty - newer models, Claude's and OpenAI's reasoning ones among them, turn away any other. Lower is plainer, higher more varied.
    +
    + +
    + +
    - +
    - +// The provider's models suggested, what a use case has now said, Try it, and Save asked first +(function () { + var form = document.querySelector('[data-ai-model-form]:not([data-ready])'); + if (!form) { + return; + } + form.setAttribute('data-ready', '1'); + var taken = JSON.parse(form.getAttribute('data-taken')); + var provider = form.querySelector('[data-ai-model-provider]'); + var modelName = form.querySelector('[data-ai-model-name]'); + var modelList = form.querySelector('[data-ai-model-list]'); + var modelHint = form.querySelector('[data-ai-model-hint]'); + var useCase = form.querySelector('[data-ai-model-use-case]'); + var useCaseNote = form.querySelector('[data-ai-use-case-note]'); + var tryButton = form.querySelector('[data-ai-try]'); + var tryResult = form.querySelector('[data-ai-try-result]'); + var save = form.querySelector('[data-ai-save]'); + var checked = false; + function escapeText(text) { + var div = document.createElement('div'); + div.textContent = text; + return div.innerHTML; + } + + function showUseCase() { + var now = taken[useCase.value]; + if (!now) { + useCaseNote.textContent = useCase.value === 'General' ? 'Everything without a model of its own uses it.' : 'Nothing else is for ' + useCase.value + ' - this one will be used for it, instead of General.'; + useCaseNote.className = 'form-text'; + } else if (now.first) { + useCaseNote.textContent = now.name + ' on ' + now.provider + ' is for ' + useCase.value + ' already, and is the one used - this one won\'t be, while it is.'; + useCaseNote.className = 'form-text text-warning-emphasis'; + } else { + useCaseNote.textContent = 'This one is used for ' + useCase.value + ' - ' + now.name + ' is for it too, but was added after.'; + useCaseNote.className = 'form-text'; + } + } + useCase.addEventListener('change', showUseCase); + showUseCase(); + + // What the provider has - only the answer for the provider picked last counts + var asking = 0; + function loadModels() { + var asked = ++asking; + modelList.innerHTML = ''; + var option = provider.options[provider.selectedIndex]; + var example = option ? option.getAttribute('data-example') : ''; + modelName.placeholder = example ? 'Like ' + example : 'As the provider names it'; + if (!provider.value) { + modelHint.textContent = 'As the provider names it.'; + return; + } + modelHint.textContent = 'Asking the provider what it has...'; + var data = new FormData(); + data.append('csrf_token', form.getAttribute('data-csrf')); + data.append('ai_provider_models', provider.value); + fetch(form.getAttribute('action'), { method: 'POST', body: data, credentials: 'same-origin' }) + .then(function (response) { return response.json(); }) + .then(function (answer) { + if (asked !== asking) { + return; + } + modelList.innerHTML = ''; + (answer.models || []).forEach(function (model) { + var item = document.createElement('option'); + item.value = model; + modelList.appendChild(item); + }); + if (!answer.ok) { + modelHint.textContent = 'As the provider names it - it couldn\'t be asked what it has: ' + answer.error; + } else if (!answer.listed) { + modelHint.textContent = 'Your deployment\'s name, as it is in Azure.'; + } else { + modelHint.textContent = 'As the provider names it - ' + answer.models.length + ' to pick from as you type.'; + } + modelHint.setAttribute('data-ai-models-loaded', String((answer.models || []).length)); + }) + .catch(function () { + modelHint.textContent = 'As the provider names it.'; + }); + } + provider.addEventListener('change', loadModels); + loadModels(); + + function clearErrors() { + form.querySelectorAll('.is-invalid').forEach(function (el) { el.classList.remove('is-invalid'); }); + form.querySelectorAll('[data-error-for]').forEach(function (el) { + el.textContent = ''; + el.classList.remove('d-block'); + }); + } + + function showErrors(errors) { + var first = null; + Object.keys(errors).forEach(function (field) { + var input = form.querySelector('[name="' + field + '"]'); + var box = form.querySelector('[data-error-for="' + field + '"]'); + if (input) { + input.classList.add('is-invalid'); + first = first || input; + } + if (box) { + box.textContent = errors[field]; + box.classList.add('d-block'); + } + }); + if (first && first.offsetParent !== null) { + first.focus(); + } + } + + function ask(tryIt) { + var data = new FormData(form); + data.append('check_ai_model', '1'); + if (tryIt) { + data.append('try', '1'); + } + return fetch(form.getAttribute('action'), { method: 'POST', body: data, credentials: 'same-origin', headers: { 'Accept': 'application/json' } }) + .then(function (response) { return response.json(); }); + } + + tryButton.addEventListener('click', function () { + var label = tryButton.innerHTML; + tryButton.disabled = true; + tryButton.innerHTML = 'Asking'; + tryResult.innerHTML = ''; + ask(true) + .then(function (answer) { + clearErrors(); + if (answer.ok) { + tryResult.innerHTML = '
    It answered in ' + (answer.ms / 1000).toFixed(1) + 's:
    ' + + '
    ' + escapeText(answer.reply) + '
    '; + } else { + showErrors(answer.errors || {}); + tryResult.innerHTML = 'Not yet - see above.'; + } + }) + .catch(function () { + tryResult.innerHTML = 'Couldn\'t try it - try again.'; + }) + .finally(function () { + tryButton.disabled = false; + tryButton.innerHTML = label; + }); + }); + + form.addEventListener('submit', function (e) { + if (checked) { + return; + } + e.preventDefault(); + var label = save.innerHTML; + save.disabled = true; + save.innerHTML = 'Checking'; + ask(false) + .then(function (answer) { + clearErrors(); + save.disabled = false; + if (answer.ok) { + checked = true; + save.innerHTML = 'Saving'; + form.requestSubmit(save); + save.disabled = true; + return; + } + save.innerHTML = label; + showErrors(answer.errors || {}); + }) + .catch(function () { + checked = true; + save.disabled = false; + save.innerHTML = label; + form.requestSubmit(save); + }); + }); +})(); + + + - -
    -
    New AI Provider
    - -
    -
    - - -
    - -
    - -
    - - -
    -
    - -
    - -
    - - -
    -
    - -
    - -
    - - -
    -
    - -
    -
    - - -
    -
    - - AI Providers - one provider's window: Add (no id) or Edit. Its kind + * fills in its address and says where its key is made; the key is never put + * back into the page, so left empty it stays. Test asks the provider for its + * models, which shows they work and suggests the model to start with. Save + * asks first (check_ai_provider, admin/post/ai_provider.php): what is wrong + * shows beside each field and the window stays open - a provider that doesn't + * answer can still be saved, once said. + */ + require_once '../../includes/modal_header.php'; -$provider_id = intval($_GET['id']); +$provider_id = intval($_GET['id'] ?? 0); +$row = $provider_id ? aiProviderRow($provider_id) : null; +if ($provider_id && !$row) { + echo json_encode(['error' => 'That AI provider no longer exists.']); + exit; +} -$sql = mysqli_query($mysqli, "SELECT ai_provider_api_key, ai_provider_api_url, ai_provider_name FROM ai_providers WHERE ai_provider_id = $provider_id LIMIT 1"); +$kinds = aiProviderKinds(); +$kind_key = $row ? (string) $row['ai_provider_kind'] : 'openai'; +$kind_key = isset($kinds[$kind_key]) ? $kind_key : 'custom'; +$kind = $kinds[$kind_key]; +$has_key = $row && trim((string) $row['ai_provider_api_key']) !== ''; +$redact = $row ? intval($row['ai_provider_redact']) === 1 : true; +// Adding the first, the model to start with comes along - for everything +$offer_first_model = !$row && !getAiModel('General'); -$row = mysqli_fetch_assoc($sql); -$provider_name = escapeHtml($row['ai_provider_name']); -$url = escapeHtml($row['ai_provider_api_url']); -$key = escapeHtml($row['ai_provider_api_key']); +// What the window's script needs of each kind +$kinds_js = []; +foreach ($kinds as $key => $info) { + $kinds_js[$key] = ['name' => $info['name'], 'url' => $info['url'], 'key' => $info['key'], 'keysUrl' => $info['keys_url'], 'local' => $info['local'], 'example' => $info['example']]; +} + +// Where what is wrong with a field shows +$error_for = function ($field) { + return '
    '; +}; -// Generate the HTML form content using output buffering. ob_start(); -?> +?>
    -
    Editing:
    +
    ' . escapeHtml($row['ai_provider_name']) . '' : 'Add AI Provider' ?>
    -
    + +
    - -
    - - + +
    + +
    +
    - -
    - - + +
    + +
    +
    - -
    - - + +
    + +
    + +
    Its chat completions address - filled in for the kind picked. One ending /v1 has /chat/completions put on.
    +
    + +
    + +
    + + +
    + +
    + +
    + + +
    + +
    + +
    +
    + > + +
    +
    Passwords, PINs, API keys, private keys, 2FA links and card numbers in a reply or ticket go as placeholders, and are put back in the answer - the provider never sees them.
    +
    + + +
    + +
    + + +
    + +
    For everything - Reword, ticket summaries and document drafts. Test lists the ones it has; more can be added in AI Models.
    +
    + + + +
    + +
    - +
    - +// Its kind fills in the address and name, and says about the key. Test and Save ask first, so what is wrong shows here +(function () { + var form = document.querySelector('[data-ai-provider-form]:not([data-ready])'); + if (!form) { + return; + } + form.setAttribute('data-ready', '1'); + var kinds = JSON.parse(form.getAttribute('data-kinds')); + var hasKey = form.getAttribute('data-has-key') === '1'; + var kindSelect = form.querySelector('[data-ai-kind]'); + var name = form.querySelector('[name="name"]'); + var url = form.querySelector('[name="url"]'); + var keyHint = form.querySelector('[data-ai-key-hint]'); + var keyRequired = form.querySelector('[data-ai-key-required]'); + var save = form.querySelector('[data-ai-save]'); + var saveLabel = save.innerHTML; + var saveAnyway = form.querySelector('[data-save-anyway]'); + var testButton = form.querySelector('[data-ai-test]'); + var testResult = form.querySelector('[data-ai-test-result]'); + var modelList = form.querySelector('[data-ai-model-list]'); + var firstModel = form.querySelector('[data-ai-first-model]'); + var kind = kindSelect.value; + var checked = false; + function escapeText(text) { + var div = document.createElement('div'); + div.textContent = text; + return div.innerHTML; + } + + function isPreset(field, value) { + return value === '' || Object.keys(kinds).some(function (key) { return kinds[key][field] === value; }); + } + + function showKind() { + var info = kinds[kind]; + keyRequired.classList.toggle('d-none', !info.key || hasKey); + if (info.local) { + keyHint.textContent = info.name + ' runs on your own network and needs no key unless you set one. ITFlow has to be able to reach it - localhost is the ITFlow server itself.'; + } else if (info.keysUrl) { + keyHint.innerHTML = 'Made in ' + escapeText(info.name) + '\'s console - give ITFlow a key of its own, so it can be turned off on its own.'; + } else { + keyHint.textContent = 'If it needs one - sent as a bearer token.'; + } + if (firstModel) { + firstModel.placeholder = info.example; + } + } + + // Anything changed - asked again before it is saved + function reset() { + checked = false; + saveAnyway.value = ''; + save.innerHTML = saveLabel; + save.classList.remove('btn-warning'); + save.classList.add('btn-primary'); + } + form.querySelectorAll('input:not([type=hidden])').forEach(function (el) { + el.addEventListener('input', reset); + }); + + kindSelect.addEventListener('change', function () { + var was = kinds[kind]; + kind = kindSelect.value; + var info = kinds[kind]; + // What the last kind filled in follows the new one; what was typed stays. Another kind is named by you + if (isPreset('name', name.value.trim())) { + name.value = kind === 'custom' ? '' : info.name; + } + if (isPreset('url', url.value.trim()) || url.value.trim() === was.url) { + url.value = info.url; + } + showKind(); + reset(); + }); + showKind(); + + function clearErrors() { + form.querySelectorAll('.is-invalid').forEach(function (el) { el.classList.remove('is-invalid'); }); + form.querySelectorAll('[data-error-for]').forEach(function (el) { + el.textContent = ''; + el.classList.remove('d-block'); + }); + } + + function showErrors(errors) { + var first = null; + Object.keys(errors).forEach(function (field) { + var input = form.querySelector('[name="' + field + '"]'); + var box = form.querySelector('[data-error-for="' + field + '"]'); + if (input) { + input.classList.add('is-invalid'); + first = first || input; + } + if (box) { + box.textContent = errors[field]; + box.classList.add('d-block'); + } + }); + if (first && first.offsetParent !== null) { + first.focus(); + } + } + + function showModels(answer) { + modelList.innerHTML = ''; + (answer.models || []).forEach(function (model) { + var option = document.createElement('option'); + option.value = model; + modelList.appendChild(option); + }); + // As it will be saved - /chat/completions put on + if (answer.url && answer.url !== url.value.trim()) { + url.value = answer.url; + } + var count = (answer.models || []).length; + if (!answer.listed) { + testResult.innerHTML = '' + escapeText(kinds[kind].name) + ' doesn\'t list its models - use Try it on its model once it is added.'; + } else { + testResult.innerHTML = 'It answered - ' + count + ' model' + (count === 1 ? '' : 's') + ' there.' + + (firstModel && count ? ' Pick the one to use above.' : ''); + } + } + + function ask(test) { + var data = new FormData(form); + data.append('check_ai_provider', '1'); + if (test) { + data.append('test', '1'); + } + return fetch(form.getAttribute('action'), { method: 'POST', body: data, credentials: 'same-origin', headers: { 'Accept': 'application/json' } }) + .then(function (response) { return response.json(); }); + } + + testButton.addEventListener('click', function () { + var label = testButton.innerHTML; + testButton.disabled = true; + testButton.innerHTML = 'Testing'; + testResult.innerHTML = ''; + ask(true) + .then(function (answer) { + clearErrors(); + if (answer.ok) { + showModels(answer); + } else { + showErrors(answer.errors || {}); + testResult.innerHTML = 'Not yet - see above.'; + } + }) + .catch(function () { + testResult.innerHTML = 'Couldn\'t test it - try again.'; + }) + .finally(function () { + testButton.disabled = false; + testButton.innerHTML = label; + }); + }); + + form.addEventListener('submit', function (e) { + // Asked already - or said it doesn't answer, and Save anyway pressed: it goes + if (checked || saveAnyway.value === '1') { + checked = true; + return; + } + e.preventDefault(); + save.disabled = true; + save.innerHTML = 'Checking'; + ask(false) + .then(function (answer) { + clearErrors(); + save.disabled = false; + if (answer.ok) { + if (answer.asked) { + showModels(answer); + } + checked = true; + save.innerHTML = 'Saving'; + form.requestSubmit(save); + save.disabled = true; + return; + } + save.innerHTML = saveLabel; + showErrors(answer.errors || {}); + // Only that it didn't answer - it can be saved anyway + if (answer.soft) { + saveAnyway.value = '1'; + save.innerHTML = 'Save anyway'; + save.classList.remove('btn-primary'); + save.classList.add('btn-warning'); + } + }) + .catch(function () { + // Couldn't ask - send it as it is, and the save says what is wrong + checked = true; + save.disabled = false; + save.innerHTML = saveLabel; + form.requestSubmit(save); + }); + }); +})(); + + + diff --git a/admin/post/ai_models.php b/admin/post/ai_models.php index cf927ff51..2b364c52b 100644 --- a/admin/post/ai_models.php +++ b/admin/post/ai_models.php @@ -1,77 +1,118 @@ AI Models (admin/ai_models.php): adding one, changing it, + * trying it and deleting it. + * + * The window asks first (check_ai_model, JSON) and says what is wrong beside + * each field; with 'try' it also sends the model a one-line request, as it is + * in the window - unsaved - and says what came back, how long it took and the + * tokens it used (recorded in AI Usage as a test). What changed is in the + * audit log, from what to what - the instructions only as changed. */ defined('FROM_POST_HANDLER') || die("Direct file access is not allowed"); -if (isset($_POST['add_ai_model'])) { +if (isset($_POST['check_ai_model']) || isset($_POST['add_ai_model']) || isset($_POST['edit_ai_model'])) { validateCSRFToken(); - $provider_id = intval($_POST['provider']); - $model = escapeSql($_POST['model']); - $prompt = escapeSql($_POST['prompt']); - $use_case = escapeSql($_POST['use_case']); + $checking = isset($_POST['check_ai_model']); + $model_id = intval($_POST['model_id'] ?? 0); + $before = $model_id ? aiModelRow($model_id) : null; + $answer = function ($ok, $errors, $extra = []) { + header('Content-Type: application/json; charset=utf-8'); + echo json_encode(['ok' => $ok, 'errors' => (object) $errors] + $extra); + exit; + }; - // Blank means "send no temperature at all" - the only setting that works on every - // provider. Anything else rides as a numeric literal, so no quoting. - $temperature = ($_POST['temperature'] ?? '') === '' ? 'NULL' : floatval($_POST['temperature']); + if ($model_id && !$before) { + if ($checking) { + $answer(false, ['model' => 'That model no longer exists - close this and look again.']); + } + flashAlert('That AI model no longer exists.', 'error'); + redirect('ai_models.php'); + } - mysqli_query($mysqli,"INSERT INTO ai_models SET ai_model_name = '$model', ai_model_prompt = '$prompt', ai_model_use_case = '$use_case', ai_model_temperature = $temperature, ai_model_ai_provider_id = $provider_id"); + [$after, $errors, $provider] = aiModelCheck($_POST); - if (!mysqli_affected_rows($mysqli)) { - logApp('AI', 'error', 'Failed to create AI Model ' . $model . ': ' . mysqli_error($mysqli)); - flashAlert("AI Model $model could not be created - see Admin > App Logs", 'error'); + if ($checking) { + $tried = []; + if (!$errors && !empty($_POST['try'])) { + // As it is in the window, on its provider + $model = $provider + [ + 'ai_model_id' => $model_id, + 'ai_model_name' => $after['ai_model_name'], + 'ai_model_use_case' => $after['ai_model_use_case'], + 'ai_model_temperature' => $after['ai_model_temperature'], + 'ai_model_prompt' => $after['ai_model_prompt'], + ]; + $started = microtime(true); + $result = callAiApi($model, [ + ['role' => 'system', 'content' => 'You are checking that you are set up and working. Answer in one short, friendly sentence.'], + ['role' => 'user', 'content' => 'Say hello, and which model you are.'], + ], ['feature' => 'test']); + $ms = intval(round((microtime(true) - $started) * 1000)); + if ($result['ok']) { + $tried = ['reply' => mb_substr(trim(strip_tags($result['content'])), 0, 400), 'ms' => $ms]; + } else { + $errors['model'] = $result['error']; + } + } + $answer(!$errors, $errors, $tried); + } + + // Only reached without the window's script - say the first thing wrong + if ($errors) { + flashAlert('Not saved - ' . escapeHtml(reset($errors)), 'error'); redirect(); } - logAudit("AI Model", "Create", "$session_name created AI Model $model"); + $name = $after['ai_model_name']; + $use_case = $after['ai_model_use_case']; - flashAlert("AI Model $model created"); - - redirect(); - -} - -if (isset($_POST['edit_ai_model'])) { - - validateCSRFToken(); - - $model_id = intval($_POST['model_id']); - $model = escapeSql($_POST['model']); - $prompt = escapeSql($_POST['prompt']); - $use_case = escapeSql($_POST['use_case']); - - // Blank means "send no temperature at all" - the only setting that works on every - // provider. Anything else rides as a numeric literal, so no quoting. - $temperature = ($_POST['temperature'] ?? '') === '' ? 'NULL' : floatval($_POST['temperature']); - - mysqli_query($mysqli,"UPDATE ai_models SET ai_model_name = '$model', ai_model_prompt = '$prompt', ai_model_use_case = '$use_case', ai_model_temperature = $temperature WHERE ai_model_id = $model_id"); - - logAudit("AI Model", "Edit", "$session_name edited AI Model $model"); - - flashAlert("AI Model $model edited"); + if (!$before) { + // A second one for a use case isn't used - the first one is - so it is said + $already = getAiModel($use_case); + $model_id = aiModelWrite($after); + logAudit("AI Model", "Create", "$session_name added AI model " . escapeSql("$name on {$provider['ai_provider_name']}, for $use_case"), 0, $model_id); + if ($already && $already['ai_model_use_case'] === $use_case) { + flashAlert('' . escapeHtml($name) . ' added - but ' . escapeHtml($already['ai_model_name']) . " is for $use_case already and is the one used. Delete it, or set it to something else, to use this one", 'warning'); + } else { + flashAlert('' . escapeHtml($name) . " added - it is used for " . ($use_case === 'General' ? 'everything without a model of its own' : $use_case)); + } + redirect('ai_models.php'); + } + $lines = aiModelChangeLines($before, $after); + if (!$lines) { + flashAlert('Nothing changed on ' . escapeHtml($name) . '', 'info'); + redirect(); + } + aiModelWrite($after, $model_id); + logAudit("AI Model", "Edit", "$session_name changed AI model " . escapeSql("$name: " . implode('; ', $lines)), 0, $model_id); + flashAlert('' . escapeHtml($name) . ' saved - ' . escapeHtml(count($lines) === 1 ? $lines[0] : count($lines) . ' changes')); redirect(); } +// Delete a model - what it was for falls back on General, or stops when it was General if (isset($_GET['delete_ai_model'])) { validateCSRFToken(); - $model_id = intval($_GET['delete_ai_model']); + $row = aiModelRow($_GET['delete_ai_model']); + if (!$row) { + flashAlert('That AI model no longer exists.', 'error'); + redirect('ai_models.php'); + } + $model_id = intval($row['ai_model_id']); + $name = $row['ai_model_name']; - $model_name = escapeSql(getFieldById('ai_models', $model_id, 'ai_model_name')); + mysqli_query($mysqli, "DELETE FROM ai_models WHERE ai_model_id = $model_id"); - mysqli_query($mysqli,"DELETE FROM ai_models WHERE ai_model_id = $model_id"); - - logAudit("AI Model", "Delete", "$session_name deleted AI Model $model_name"); - - flashAlert("AI Model $model_name deleted", 'error'); - - redirect(); + logAudit("AI Model", "Delete", "$session_name deleted AI model " . escapeSql("$name on {$row['ai_provider_name']}, for {$row['ai_model_use_case']}"), 0, $model_id); + flashAlert('' . escapeHtml($name) . ' deleted', 'error'); + redirect('ai_models.php'); } diff --git a/admin/post/ai_provider.php b/admin/post/ai_provider.php index acf1d55da..337908191 100644 --- a/admin/post/ai_provider.php +++ b/admin/post/ai_provider.php @@ -1,65 +1,195 @@ AI Providers (admin/ai_providers.php): adding one, + * changing it, testing it and deleting it. + * + * Saving checks what was sent (aiProviderCheck(), functions/ai.php) and, when + * its kind, address or key is new or changed, asks the provider for its models + * - which says whether they work. The window asks first (check_ai_provider, + * which answers in JSON) and says what is wrong beside each field; a provider + * that doesn't answer right now can still be saved once the window has said + * so (save_anyway). Adding one can add its first model too. What changed is in + * the audit log, from what to what - the key only as changed. */ defined('FROM_POST_HANDLER') || die("Direct file access is not allowed"); -if (isset($_POST['add_ai_provider'])) { +if (isset($_POST['check_ai_provider']) || isset($_POST['add_ai_provider']) || isset($_POST['edit_ai_provider'])) { validateCSRFToken(); - $provider = escapeSql($_POST['provider']); - $url = escapeSql($_POST['url']); - $model = escapeSql($_POST['model']); - $api_key = escapeSql($_POST['api_key']); + $checking = isset($_POST['check_ai_provider']); + $provider_id = intval($_POST['provider_id'] ?? 0); + $before = $provider_id ? aiProviderRow($provider_id) : null; + $answer = function ($ok, $errors, $extra = []) { + header('Content-Type: application/json; charset=utf-8'); + echo json_encode(['ok' => $ok, 'errors' => (object) $errors] + $extra); + exit; + }; - mysqli_query($mysqli,"INSERT INTO ai_providers SET ai_provider_name = '$provider', ai_provider_api_url = '$url', ai_provider_api_key = '$api_key'"); + if ($provider_id && !$before) { + if ($checking) { + $answer(false, ['name' => 'That provider no longer exists - close this and look again.']); + } + flashAlert('That AI provider no longer exists.', 'error'); + redirect('ai_providers.php'); + } - $ai_provider_id = mysqli_insert_id($mysqli); + [$after, $errors] = aiProviderCheck($_POST, $before); - logAudit("AI Provider", "Create", "$session_name created AI Provider $provider"); + // Its first model, for everything - adding one while there is nothing for everything yet + $first_model = !$before && !getAiModel('General') ? trim((string) ($_POST['first_model'] ?? '')) : ''; + if ($first_model !== '' && (mb_strlen($first_model) > 200 || (preg_match('/\s/', $first_model) && !in_array($after['ai_provider_kind'], ['azure', 'custom'], true)))) { + $errors['first_model'] = "That isn't a model's name - " . aiProviderKind($after['ai_provider_kind'])['example'] . ', say.'; + } - flashAlert("AI Model $provider created"); + // New or changed - does it answer? Once it has, not asked again for ten minutes + $test = ['ok' => true, 'models' => [], 'listed' => false, 'asked' => false]; + $soft = false; + $save_anyway = !empty($_POST['save_anyway']); + if (!array_intersect_key($errors, ['kind' => 1, 'url' => 1, 'api_key' => 1]) && (aiProviderConnectionChanged($after, $before) || !empty($_POST['test'])) && !($save_anyway && !$checking)) { + $asked_key = hash('sha256', $after['ai_provider_kind'] . '|' . $after['ai_provider_api_url'] . '|' . $after['ai_provider_api_key']); + $asked_at = $_SESSION['ai_provider_asked'][$asked_key] ?? 0; + if ($checking || time() - $asked_at > 600) { + $test = aiProviderTest($after['ai_provider_kind'], $after['ai_provider_api_url'], $after['ai_provider_api_key']) + ['asked' => true]; + if ($test['ok']) { + $_SESSION['ai_provider_asked'] = [$asked_key => time()]; + } else { + // A key turned away is the key; anything else is the address, or it being down right now + $errors[strpos($test['error'], 'turned the key away') !== false ? 'api_key' : 'url'] = $test['error']; + $soft = true; + } + } + } + if ($checking) { + $answer(!$errors, $errors, [ + 'soft' => $soft && count($errors) === 1, + 'models' => $test['models'], + 'listed' => $test['listed'], + 'asked' => $test['asked'], + 'url' => $after['ai_provider_api_url'], + ]); + } + + // Only reached without the window's script - say the first thing wrong + if ($errors) { + flashAlert('Not saved - ' . escapeHtml(reset($errors)), 'error'); + redirect(); + } + + $name = $after['ai_provider_name']; + $kind_name = aiProviderKind($after['ai_provider_kind'])['name']; + + if (!$before) { + $provider_id = aiProviderWrite($after); + logAudit("AI Provider", "Create", "$session_name added AI provider " . escapeSql("$name ($kind_name, {$after['ai_provider_api_url']})" . ($after['ai_provider_redact'] ? '' : ' - passwords and keys sent as they are')), 0, $provider_id); + if ($first_model !== '') { + $model_id = aiModelWrite(['ai_model_ai_provider_id' => $provider_id, 'ai_model_name' => $first_model, 'ai_model_use_case' => 'General', 'ai_model_temperature' => null, 'ai_model_prompt' => '']); + logAudit("AI Model", "Create", "$session_name added AI model " . escapeSql("$first_model on $name, for General"), 0, $model_id); + flashAlert('' . escapeHtml($name) . ' added, with ' . escapeHtml($first_model) . ' for everything - AI is ready to use'); + } else { + flashAlert('' . escapeHtml($name) . ' added - now add the model to use on it'); + } + redirect('ai_providers.php'); + } + + $lines = aiProviderChangeLines($before, $after); + if (!$lines) { + flashAlert('Nothing changed on ' . escapeHtml($name) . '', 'info'); + redirect(); + } + aiProviderWrite($after, $provider_id); + logAudit("AI Provider", "Edit", "$session_name changed AI provider " . escapeSql("$name: " . implode('; ', $lines)), 0, $provider_id); + flashAlert('' . escapeHtml($name) . ' saved - ' . escapeHtml(count($lines) === 1 ? $lines[0] : count($lines) . ' changes')); redirect(); } -if (isset($_POST['edit_ai_provider'])) { +// The models a provider has - for the model window's suggestions. JSON. +if (isset($_POST['ai_provider_models'])) { validateCSRFToken(); - $provider_id = intval($_POST['provider_id']); - $provider = escapeSql($_POST['provider']); - $url = escapeSql($_POST['url']); - $api_key = escapeSql($_POST['api_key']); + header('Content-Type: application/json; charset=utf-8'); + $row = aiProviderRow($_POST['ai_provider_models']); + if (!$row) { + echo json_encode(['ok' => false, 'models' => [], 'error' => 'That provider no longer exists.', 'listed' => false, 'example' => '']); + exit; + } + // Asked once in ten minutes per provider - its models don't change from one window to the next + $cache_key = hash('sha256', $row['ai_provider_kind'] . '|' . $row['ai_provider_api_url'] . '|' . $row['ai_provider_api_key']); + $cached = $_SESSION['ai_provider_models'][$cache_key] ?? null; + if ($cached && time() - $cached['at'] < 600) { + echo json_encode($cached['answer']); + exit; + } + $test = aiProviderTest($row['ai_provider_kind'], $row['ai_provider_api_url'], $row['ai_provider_api_key']); + $answer = ['ok' => $test['ok'], 'models' => $test['models'], 'error' => $test['error'], 'listed' => $test['listed'], 'example' => aiProviderKind($row['ai_provider_kind'])['example']]; + if ($test['ok']) { + $_SESSION['ai_provider_models'] = [$cache_key => ['at' => time(), 'answer' => $answer]]; + } + echo json_encode($answer); + exit; - mysqli_query($mysqli,"UPDATE ai_providers SET ai_provider_name = '$provider', ai_provider_api_url = '$url', ai_provider_api_key = '$api_key' WHERE ai_provider_id = $provider_id"); +} - logAudit("AI Provider", "Edit", "$session_name edited AI Provider $provider"); +// Test a provider - from its menu: does it answer, with its key, and has it the models set up on it? +if (isset($_GET['test_ai_provider'])) { - flashAlert("AI Model $provider edited"); + validateCSRFToken(); + $row = aiProviderRow($_GET['test_ai_provider']); + if (!$row) { + flashAlert('That AI provider no longer exists.', 'error'); + redirect('ai_providers.php'); + } + $name = escapeHtml($row['ai_provider_name']); + $test = aiProviderTest($row['ai_provider_kind'], $row['ai_provider_api_url'], $row['ai_provider_api_key']); + if (!$test['ok']) { + logApp('AI', 'warning', "Test of AI provider {$row['ai_provider_name']} - {$test['error']}"); + flashAlert("$name didn't answer as it should - " . escapeHtml($test['error']), 'error'); + } elseif (!$test['listed']) { + flashAlert("$name doesn't list its models, so it can't be tested on its own - use Try it on one of its models", 'info'); + } else { + $count = count($test['models']); + $ours = []; + $sql = mysqli_query($mysqli, "SELECT ai_model_name FROM ai_models WHERE ai_model_ai_provider_id = " . intval($row['ai_provider_id']) . " ORDER BY ai_model_name"); + while ($model = mysqli_fetch_row($sql)) { + $ours[] = $model[0]; + } + $missing = $count ? array_values(array_diff($ours, $test['models'])) : []; + flashAlert("$name answered - $count model" . ($count === 1 ? '' : 's') . ' there' + . ($missing ? '. It has no ' . implode(' or ', array_map('escapeHtml', $missing)) . ' - check the name in AI Models' : ''), $missing ? 'warning' : 'success'); + } redirect(); } +// Delete a provider - its models go with it if (isset($_GET['delete_ai_provider'])) { validateCSRFToken(); - - $provider_id = intval($_GET['delete_ai_provider']); - $provider_name = escapeSql(getFieldById('ai_providers', $provider_id, 'ai_provider_name')); + $row = aiProviderRow($_GET['delete_ai_provider']); + if (!$row) { + flashAlert('That AI provider no longer exists.', 'error'); + redirect('ai_providers.php'); + } + $provider_id = intval($row['ai_provider_id']); + $name = $row['ai_provider_name']; + $models = []; + $sql = mysqli_query($mysqli, "SELECT ai_model_name FROM ai_models WHERE ai_model_ai_provider_id = $provider_id ORDER BY ai_model_name"); + while ($model = mysqli_fetch_row($sql)) { + $models[] = $model[0]; + } - mysqli_query($mysqli,"DELETE FROM ai_providers WHERE ai_provider_id = $provider_id"); + // Its models cascade with it + mysqli_query($mysqli, "DELETE FROM ai_providers WHERE ai_provider_id = $provider_id"); - logAudit("AI Provider", "Delete", "$session_name deleted AI Provider $provider_name", 'error'); - - flashAlert("AI Provider $provider_name deleted", 'error'); - - redirect(); + logAudit("AI Provider", "Delete", "$session_name deleted AI provider " . escapeSql($name . ($models ? ' - and with it ' . implode(', ', $models) : '')), 0, $provider_id); + flashAlert('' . escapeHtml($name) . ' deleted' . ($models ? ', with ' . count($models) . ' model' . (count($models) === 1 ? '' : 's') : ''), 'error'); + redirect('ai_providers.php'); } diff --git a/admin/settings.php b/admin/settings.php index e14790356..3e6ef704a 100644 --- a/admin/settings.php +++ b/admin/settings.php @@ -80,6 +80,10 @@ if ($config_module_enable_accounting) { $attention[] = $problem; } } +// AI requests failing - and still failing +foreach (aiProblems() as $problem) { + $attention[] = $problem; +} if (trim((string) $config_timezone) === '') { $attention[] = ['warning', 'fa-clock', 'No timezone', 'Dates and times are in the server\'s timezone until one is picked.', 'settings_localization.php#setting=config_timezone', 'Pick one']; } diff --git a/agent/ajax.php b/agent/ajax.php index f0496fe98..8084a89e3 100644 --- a/agent/ajax.php +++ b/agent/ajax.php @@ -1938,32 +1938,61 @@ if (isset($_GET['contact_email_check'])) { echo json_encode($response); } +/* + * AI - Reword, the ticket summary and a document template draft. Each needs + * the role's Use AI permission (module_ai) and a model for its use case + * (functions/ai.php), and names itself so AI Usage can tell them apart. + * Passwords and keys are taken out of what is sent, and put back in the answer, + * by callAiApi(). What comes back as HTML goes through the purifier first. + */ + +// The agent may not use AI - said as the rest of ajax.php says it, JSON with a 403 +$aiRefuse = function () { + permissionRefuse(permissionRefusedText('module_ai', 1, 0)); +}; + if (isset($_GET['ai_reword'])) { header('Content-Type: application/json'); + if (!aiCan()) { + $aiRefuse(); + } - // The reword button sits on every TinyMCE instance, so the ticket editor asks for - // the Tickets model and everything else gets General. Anything unrecognised is - // treated as General rather than trusted into the query. + // The ticket reply editor asks for the Tickets model, every other editor General. + // Anything unrecognised is treated as General rather than trusted into the query. $use_case = ($_GET['use_case'] ?? '') === 'Tickets' ? 'Tickets' : 'General'; - $model = getAiModel($use_case); - if (!$model) { echo json_encode(['error' => aiModelMissingError($use_case)]); exit; } - // Collecting the input data from the AJAX request. - $inputJSON = file_get_contents('php://input'); - $input = json_decode($inputJSON, TRUE); // Convert JSON into array. + $input = json_decode(file_get_contents('php://input'), true); + $text = is_array($input) ? (string) ($input['text'] ?? '') : ''; + if (trim(str_replace("\xC2\xA0", ' ', html_entity_decode(strip_tags($text), ENT_QUOTES, 'UTF-8'))) === '') { + echo json_encode(['error' => 'There is nothing to reword yet - write something first.']); + exit; + } + if (mb_strlen($text) > 60000) { + echo json_encode(['error' => 'That is too long to reword in one go - reword it a part at a time.']); + exit; + } + // The client it is for, when the editor knows - for AI Usage + $reword_client_id = intval($input['client_id'] ?? 0); + if ($reword_client_id && !getFieldById('clients', $reword_client_id, 'client_id')) { + $reword_client_id = 0; + } - $userText = $input['text'] ?? ''; + $instructions = $use_case === 'Tickets' + ? 'You edit the replies an IT services company sends on support tickets. Rewrite the reply you are given so it reads clearly, professionally and kindly - fix spelling and grammar, and make it easy for someone who isn\'t technical to follow.' + : 'You edit what an IT services company writes. Rewrite the text you are given so it reads clearly and professionally - fix spelling and grammar, and tighten it.'; + $instructions .= ' Keep its meaning and every fact, name, number, date, link, command and placeholder in square brackets exactly as they are. Write in the language it is written in. Don\'t add anything it doesn\'t say.' + . ' Keep its HTML formatting - paragraphs, line breaks, lists, bold and links - and answer with the reworded HTML only: no introduction, no notes, no code fences.'; $result = callAiApi($model, [ - ["role" => "system", "content" => $model['ai_model_prompt']], - ["role" => "user", "content" => $userText], - ]); + ['role' => 'system', 'content' => $instructions], + ['role' => 'user', 'content' => $text], + ], ['feature' => $use_case === 'Tickets' ? 'reword_ticket' : 'reword', 'client_id' => $reword_client_id]); // Report failures as an error, never as reworded text - the editor writes // rewordedText straight back over the user's content @@ -1971,44 +2000,38 @@ if (isset($_GET['ai_reword'])) { echo json_encode(['error' => $result['error']]); exit; } + $reworded = aiCleanHtml($result['content']); + if ($reworded === '') { + echo json_encode(['error' => 'The AI provider answered with nothing - try again.']); + exit; + } - $content = $result['content']; - - // Clean any leading "html" word or other unwanted text at the beginning. - $content = preg_replace('/^html/i', '', $content); // Remove any occurrence of 'html' at the start - - // Clean the response content to remove backticks or code block markers. - $cleanedContent = str_replace('```', '', $content); // Remove backticks if they exist. - - // Trim any leading/trailing whitespace. - $cleanedContent = trim($cleanedContent); - - echo json_encode(['rewordedText' => $cleanedContent]); + echo json_encode(['rewordedText' => $reworded]); + exit; } if (isset($_GET['ai_create_document_template'])) { enforceUserPermission('module_support'); + if (!aiCan()) { + $aiRefuse(); + } header('Content-Type: text/html; charset=UTF-8'); - $prompt = $_POST['prompt'] ?? ''; - - // Basic validation - if (empty($prompt)) { + $prompt = trim((string) ($_POST['prompt'] ?? '')); + if ($prompt === '') { echo "No prompt provided."; exit; } $model = getAiModel('Documentation'); - if (!$model) { echo escapeHtml(aiModelMissingError('Documentation')); exit; } - // Prepare prompt // It writes in the words a document template fills in from the client, and asks for the rest $system_message = "You are a helpful IT documentation assistant. You will create a well-structured HTML template for IT documentation based on a given prompt. Include headings, subheadings, bullet points, and possibly tables for clarity. No Lorem Ipsum, professional language." . " These words in square brackets are filled in from the client when a document is made from the template - use them where those details belong: " . implode(', ', array_map(function ($word) { @@ -2017,49 +2040,49 @@ if (isset($_GET['ai_create_document_template'])) { . " These put in the client's records, as a table when on a line of their own (

    [Networks]

    ) and as a list of names in a sentence - use them instead of drawing those tables yourself: " . implode(', ', array_map(function ($word) { return "[$word]"; }, array_keys(DOCUMENT_TEMPLATE_TABLES))) . "." - . " For anything else someone fills in, use a short name of its own in square brackets, like [Firewall Model] or [Backup Schedule] - letters, digits and spaces, starting with a letter."; + . " For anything else someone fills in, use a short name of its own in square brackets, like [Firewall Model] or [Backup Schedule] - letters, digits and spaces, starting with a letter." + . " Answer with the HTML only - no , or , no introduction, no code fences."; $user_message = "Create an HTML formatted IT documentation template based on the following request:\n\n\"$prompt\"\n\nThe template should be structured, professional, and useful for IT staff. Include relevant sections, instructions, prerequisites, and best practices."; $result = callAiApi($model, [ ["role" => "system", "content" => $system_message], ["role" => "user", "content" => $user_message] - ]); + ], ['feature' => 'document_template']); if (!$result['ok']) { echo "

    " . escapeHtml($result['error']) . "

    "; exit; } - // Print the generated HTML template directly - echo $result['content']; + echo aiCleanHtml($result['content']); + exit; + } if (isset($_GET['ai_ticket_summary'])) { enforceUserPermission('module_support'); + if (!aiCan()) { + $aiRefuse(); + } header('Content-Type: text/html; charset=UTF-8'); $model = getAiModel('Tickets'); - if (!$model) { echo escapeHtml(aiModelMissingError('Tickets')); exit; } - // Retrieve the ticket_id from POST - $ticket_id = intval($_POST['ticket_id']); - - // Query the database for ticket details - $sql = mysqli_query($mysqli, " + $ticket_id = intval($_POST['ticket_id'] ?? 0); + $row = mysqli_fetch_assoc(mysqli_query($mysqli, " SELECT ticket_subject, ticket_details, ticket_source, ticket_priority, ticket_status_name, category_name, ticket_client_id FROM tickets LEFT JOIN ticket_statuses ON ticket_status = ticket_status_id LEFT JOIN categories ON ticket_category = category_id WHERE ticket_id = $ticket_id LIMIT 1 - "); - $row = mysqli_fetch_assoc($sql); + ")); if (!$row) { echo 'That ticket no longer exists.'; exit; @@ -2077,7 +2100,6 @@ if (isset($_GET['ai_ticket_summary'])) { enforceClientAccess(); } - // Get ticket replies $sql_replies = mysqli_query($mysqli, " SELECT ticket_reply, ticket_reply_type, user_name FROM ticket_replies @@ -2132,14 +2154,15 @@ if (isset($_GET['ai_ticket_summary'])) { $result = callAiApi($model, [ ["role" => "system", "content" => "Your task is to summarize IT support tickets with clear, concise details."], ["role" => "user", "content" => $prompt] - ]); + ], ['feature' => 'ticket_summary', 'client_id' => $client_id]); if (!$result['ok']) { echo "

    " . escapeHtml($result['error']) . "

    "; exit; } - echo $result['content']; + echo aiCleanHtml($result['content']); + exit; } // Stops people trying to use sub-domains in the domains tracker diff --git a/agent/modals/ticket/ticket_summary.php b/agent/modals/ticket/ticket_summary.php index da2624055..969088841 100644 --- a/agent/modals/ticket/ticket_summary.php +++ b/agent/modals/ticket/ticket_summary.php @@ -5,6 +5,11 @@ require_once '../../../includes/modal_header.php'; $ticket = ticketForModal($_GET['ticket_id'] ?? 0, 1); $ticket_id = intval($ticket['ticket_id']); +// Using AI is a permission of its own +if (!aiCan()) { + permissionRefuse(permissionRefusedText('module_ai', 1, 0)); +} + ob_start(); ?> @@ -34,15 +39,20 @@ itflowReady(function () { }) .then(function (res) { if (!res.ok) { - throw new Error('HTTP ' + res.status); + // Turned away - ajax.php says why, in JSON + return res.json().then(function (answer) { + throw new Error(answer.error || ''); + }, function () { + throw new Error(''); + }); } return res.text(); }) .then(function (html) { target.innerHTML = html; }) - .catch(function () { - target.textContent = 'Error generating summary.'; + .catch(function (error) { + target.textContent = error && error.message ? error.message : 'Error generating summary.'; }); }); diff --git a/agent/ticket.php b/agent/ticket.php index b9f9b9006..2f82e0c64 100644 --- a/agent/ticket.php +++ b/agent/ticket.php @@ -451,8 +451,8 @@ if (isset($_GET['ticket_id'])) { $resolve_confirm_text .= "It closes on its own after $config_ticket_autoclose_hours hours unless the client replies."; } - // Only offer the summary when an AI model is actually set up for it - $ai_summary_available = (bool) getAiModel('Tickets'); + // Only offer the summary when an AI model is set up for it and the role may use AI + $ai_summary_available = aiReady('Tickets'); // Billing actions, decided once - the header shows Invoice once the work is done $show_quote_button = $can_edit_ticket && $config_module_enable_accounting && $ticket_billable == 1 && empty($quote_id) && empty($invoice_id) && lookupUserPermission("module_sales") >= 2; @@ -765,7 +765,7 @@ if (isset($_GET['ticket_id'])) { ?>
    - +
    diff --git a/cron/nightly_tasks.php b/cron/nightly_tasks.php index fb4006517..19591df63 100644 --- a/cron/nightly_tasks.php +++ b/cron/nightly_tasks.php @@ -179,10 +179,13 @@ backupRunRetention($mysqli); // Cleanup old logs - each kept for its own number of days, 0 for keep forever (Admin > Settings > Logs, functions/app_log.php) $log_retention = appLogRetention(); -// The audit log, and API key use with it +// The audit log, and API key and AI use with it if ($log_retention['audit'] > 0) { mysqli_query($mysqli, "DELETE FROM logs WHERE log_created_at < CURDATE() - INTERVAL {$log_retention['audit']} DAY"); mysqli_query($mysqli, "DELETE FROM api_key_usage WHERE api_key_usage_date < CURDATE() - INTERVAL {$log_retention['audit']} DAY"); + if (aiUsageTableExists()) { + mysqli_query($mysqli, "DELETE FROM ai_usage WHERE ai_usage_created_at < CURDATE() - INTERVAL {$log_retention['audit']} DAY"); + } } // The app logs diff --git a/db.sql b/db.sql index 07feb19a9..e107f8b2f 100644 --- a/db.sql +++ b/db.sql @@ -96,14 +96,47 @@ DROP TABLE IF EXISTS `ai_providers`; CREATE TABLE `ai_providers` ( `ai_provider_id` int(11) NOT NULL AUTO_INCREMENT, `ai_provider_name` varchar(200) NOT NULL, - `ai_provider_api_url` varchar(200) NOT NULL, - `ai_provider_api_key` varchar(200) DEFAULT NULL, + `ai_provider_kind` varchar(50) NOT NULL DEFAULT 'custom', + `ai_provider_api_url` varchar(500) NOT NULL, + `ai_provider_api_key` varchar(500) DEFAULT NULL, + `ai_provider_redact` tinyint(1) NOT NULL DEFAULT 1, `ai_provider_created_at` datetime NOT NULL DEFAULT current_timestamp(), `ai_provider_updated_at` datetime DEFAULT NULL ON UPDATE current_timestamp(), PRIMARY KEY (`ai_provider_id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; /*!40101 SET character_set_client = @saved_cs_client */; +-- +-- Table structure for table `ai_usage` +-- + +DROP TABLE IF EXISTS `ai_usage`; +/*!40101 SET @saved_cs_client = @@character_set_client */; +/*!40101 SET character_set_client = utf8mb4 */; +CREATE TABLE `ai_usage` ( + `ai_usage_id` int(11) NOT NULL AUTO_INCREMENT, + `ai_usage_feature` varchar(50) NOT NULL, + `ai_usage_use_case` varchar(50) DEFAULT NULL, + `ai_usage_provider_id` int(11) NOT NULL DEFAULT 0, + `ai_usage_provider_name` varchar(200) DEFAULT NULL, + `ai_usage_model_id` int(11) NOT NULL DEFAULT 0, + `ai_usage_model` varchar(200) DEFAULT NULL, + `ai_usage_user_id` int(11) NOT NULL DEFAULT 0, + `ai_usage_client_id` int(11) NOT NULL DEFAULT 0, + `ai_usage_tokens_in` int(11) NOT NULL DEFAULT 0, + `ai_usage_tokens_out` int(11) NOT NULL DEFAULT 0, + `ai_usage_ms` int(11) NOT NULL DEFAULT 0, + `ai_usage_redacted` int(11) NOT NULL DEFAULT 0, + `ai_usage_ok` tinyint(1) NOT NULL DEFAULT 1, + `ai_usage_error` varchar(255) DEFAULT NULL, + `ai_usage_created_at` datetime NOT NULL DEFAULT current_timestamp(), + PRIMARY KEY (`ai_usage_id`), + KEY `ai_usage_created_at` (`ai_usage_created_at`), + KEY `ai_usage_provider_id` (`ai_usage_provider_id`), + KEY `ai_usage_model_id` (`ai_usage_model_id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; +/*!40101 SET character_set_client = @saved_cs_client */; + -- -- Table structure for table `api_key_usage` -- diff --git a/functions/ai.php b/functions/ai.php index 76e88e94f..43e7773d2 100644 --- a/functions/ai.php +++ b/functions/ai.php @@ -1,125 +1,142 @@ AI Providers - their kind (OpenAI, Azure, + * a model run on your own network...) decides how each is called - the models + * on them, each for a use case with General the fallback, and every request + * made to one: who may make it (the role's Use AI permission), the passwords + * and keys taken out of it first and put back in the answer (aiRedact()), and + * what it cost (ai_usage - Admin > AI Usage). + * + * The features themselves - Reword, the ticket summary, a document template + * draft - are the endpoints in agent/ajax.php, each naming itself + * (aiFeatures()) so its requests can be told apart. */ -// A provider that never answers would otherwise hold a PHP worker open until -// max_execution_time +// A provider that never answers would otherwise hold a PHP worker open until max_execution_time DEFINE("AI_REQUEST_TIMEOUT", 60); /* - * The model configured for a use case - one of the values the add/edit modals offer: - * General, Tickets, Documentation. - * - * A feature-specific model wins, a General model is the fallback, so an install with - * a single General model keeps working everywhere. Returns null when nothing usable - * is configured; callers report that rather than posting to an empty URL. + * The kinds of provider, ready made - name, chat completions address, whether + * it needs a key, how the key is sent (bearer, or Azure's api-key header), + * whether and how it lists its models, whether it runs on your own network (so + * nothing leaves it), where its keys are made and a model to start with. + * 'custom' is any other OpenAI-compatible endpoint. */ -function getAiModel($use_case = 'General') { +function aiProviderKinds() { + return [ + 'openai' => ['name' => 'OpenAI', 'url' => 'https://api.openai.com/v1/chat/completions', 'key' => true, 'auth' => 'bearer', 'models' => 'openai', 'local' => false, + 'keys_url' => 'https://platform.openai.com/api-keys', 'example' => 'gpt-5-mini'], + 'anthropic' => ['name' => 'Anthropic (Claude)', 'url' => 'https://api.anthropic.com/v1/chat/completions', 'key' => true, 'auth' => 'bearer', 'models' => 'anthropic', 'local' => false, + 'keys_url' => 'https://console.anthropic.com/settings/keys', 'example' => 'claude-sonnet-5-5'], + 'azure' => ['name' => 'Azure OpenAI', 'url' => 'https://YOUR-RESOURCE.openai.azure.com/openai/v1/chat/completions', 'key' => true, 'auth' => 'api-key', 'models' => '', 'local' => false, + 'keys_url' => 'https://portal.azure.com', 'example' => 'your deployment name'], + 'gemini' => ['name' => 'Google Gemini', 'url' => 'https://generativelanguage.googleapis.com/v1beta/openai/chat/completions', 'key' => true, 'auth' => 'bearer', 'models' => 'openai', 'local' => false, + 'keys_url' => 'https://aistudio.google.com/apikey', 'example' => 'gemini-2.5-flash'], + 'mistral' => ['name' => 'Mistral', 'url' => 'https://api.mistral.ai/v1/chat/completions', 'key' => true, 'auth' => 'bearer', 'models' => 'openai', 'local' => false, + 'keys_url' => 'https://console.mistral.ai/api-keys', 'example' => 'mistral-small-latest'], + 'groq' => ['name' => 'Groq', 'url' => 'https://api.groq.com/openai/v1/chat/completions', 'key' => true, 'auth' => 'bearer', 'models' => 'openai', 'local' => false, + 'keys_url' => 'https://console.groq.com/keys', 'example' => 'llama-3.3-70b-versatile'], + 'openrouter' => ['name' => 'OpenRouter', 'url' => 'https://openrouter.ai/api/v1/chat/completions', 'key' => true, 'auth' => 'bearer', 'models' => 'openai', 'local' => false, + 'keys_url' => 'https://openrouter.ai/keys', 'example' => 'openai/gpt-5-mini'], + 'ollama' => ['name' => 'Ollama', 'url' => 'http://localhost:11434/v1/chat/completions', 'key' => false, 'auth' => 'bearer', 'models' => 'openai', 'local' => true, + 'keys_url' => '', 'example' => 'llama3.1'], + 'lmstudio' => ['name' => 'LM Studio', 'url' => 'http://localhost:1234/v1/chat/completions', 'key' => false, 'auth' => 'bearer', 'models' => 'openai', 'local' => true, + 'keys_url' => '', 'example' => 'the model loaded in LM Studio'], + 'custom' => ['name' => 'Other (OpenAI-compatible)', 'url' => '', 'key' => false, 'auth' => 'bearer', 'models' => 'openai', 'local' => false, + 'keys_url' => '', 'example' => 'its model name'], + ]; +} - global $mysqli; +/* A provider's kind, as aiProviderKinds() describes it - custom for one it doesn't know. */ +function aiProviderKind($kind) { + $kinds = aiProviderKinds(); + return $kinds[(string) $kind] ?? $kinds['custom']; +} - $use_case = escapeSql($use_case); +/* What a model can be for, and the features each covers. A use case with no model of its own uses General. */ +function aiUseCases() { + return [ + 'General' => 'Everything without a model of its own - Reword in any editor.', + 'Tickets' => 'Ticket summaries, and Reword in the ticket reply editor.', + 'Documentation' => 'Drafting document templates.', + ]; +} - // Feature-specific first, then General - FIELD() keeps that preference in SQL so - // one query answers both - $preference = ($use_case === 'General') ? "'General'" : "'$use_case', 'General'"; +/* Each AI feature - its name as Admin > AI Usage shows it. */ +function aiFeatures() { + return [ + 'reword' => 'Reword', + 'reword_ticket' => 'Reword a ticket reply', + 'ticket_summary' => 'Ticket summary', + 'document_template' => 'Document template draft', + 'test' => 'Test', + ]; +} - $sql = mysqli_query($mysqli, - "SELECT ai_model_name, ai_model_prompt, ai_model_use_case, ai_model_temperature, - ai_provider_name, ai_provider_api_url, ai_provider_api_key - FROM ai_models - LEFT JOIN ai_providers ON ai_model_ai_provider_id = ai_provider_id - WHERE ai_model_use_case IN ($preference) - ORDER BY FIELD(ai_model_use_case, $preference), ai_model_id ASC - LIMIT 1" - ); - - $model = mysqli_fetch_assoc($sql); - - // A model row with no provider behind it (or no endpoint) can't be called - if (!$model || empty($model['ai_model_name']) || empty($model['ai_provider_api_url'])) { - return null; - } - - return $model; +/* Whether the signed-in agent may use AI - their role's Use AI permission (admins always). */ +function aiCan() { + return (bool) lookupUserPermission('module_ai'); } /* - * Posts a chat-completion request. Returns: - * - * ['ok' => true, 'content' => '...'] - * ['ok' => false, 'error' => 'short message safe to show the user'] - * - * Provider detail - status code, error type, code and message - goes to the app log - * so a misconfiguration is diagnosable. The API key and the message bodies never do. + * The model configured for a use case - one of aiUseCases(). A feature-specific + * model wins, a General model is the fallback, so an install with a single + * General model works everywhere. Null when nothing usable is configured; + * callers report that rather than posting to an empty URL. */ -function callAiApi($model, $messages) { - - $data = [ - 'model' => $model['ai_model_name'], - 'messages' => $messages, - ]; - - // Only send a temperature when the model has one configured. Newer OpenAI models - // accept nothing but their own default and 400 on anything else, which is what - // the old hardcoded 0.5 / 0.3 ran into. - if (isset($model['ai_model_temperature']) && $model['ai_model_temperature'] !== '') { - $data['temperature'] = floatval($model['ai_model_temperature']); +function getAiModel($use_case = 'General') { + global $mysqli; + static $models = []; + $use_case = isset(aiUseCases()[$use_case]) ? $use_case : 'General'; + if (array_key_exists($use_case, $models)) { + return $models[$use_case]; } - $ch = curl_init($model['ai_provider_api_url']); - curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); - curl_setopt($ch, CURLOPT_POST, true); - curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data)); - curl_setopt($ch, CURLOPT_TIMEOUT, AI_REQUEST_TIMEOUT); - curl_setopt($ch, CURLOPT_HTTPHEADER, [ - 'Content-Type: application/json', - 'Authorization: Bearer ' . $model['ai_provider_api_key'], - ]); + // Feature-specific first, then General - FIELD() keeps that preference in SQL so one query answers both + $preference = ($use_case === 'General') ? "'General'" : "'$use_case', 'General'"; - $response = curl_exec($ch); - $status = intval(curl_getinfo($ch, CURLINFO_RESPONSE_CODE)); - $transport_error = curl_error($ch); - curl_close($ch); + // Before the 3.3.8 update, every provider is called as an OpenAI-compatible one, secrets taken out + $provider_columns = aiUsageTableExists() ? 'ai_provider_kind, ai_provider_redact' : "'custom' AS ai_provider_kind, 1 AS ai_provider_redact"; - // Enough to tell one provider/model pairing from another in the log - $context = $model['ai_provider_name'] . ' / ' . $model['ai_model_name']; + $model = mysqli_fetch_assoc(mysqli_query($mysqli, + "SELECT ai_model_id, ai_model_name, ai_model_prompt, ai_model_use_case, ai_model_temperature, + ai_provider_id, ai_provider_name, $provider_columns, ai_provider_api_url, ai_provider_api_key + FROM ai_models + JOIN ai_providers ON ai_model_ai_provider_id = ai_provider_id + WHERE ai_model_use_case IN ($preference) + ORDER BY FIELD(ai_model_use_case, $preference), ai_model_id ASC + LIMIT 1")); - if ($response === false) { - logApp('AI', 'error', "$context - could not reach the provider: $transport_error"); - return ['ok' => false, 'error' => 'Could not reach the AI provider.']; + // A model with no name, or a provider with no address, can't be called + if (!$model || trim((string) $model['ai_model_name']) === '' || trim((string) $model['ai_provider_api_url']) === '') { + $model = null; } + return $models[$use_case] = $model; +} - $decoded = json_decode($response, true); +/* Whether AI is there for this agent, for a use case - set up, and their role may use it. */ +function aiReady($use_case = 'General') { + return aiCan() && getAiModel($use_case) !== null; +} - if ($status < 200 || $status > 299) { - $provider_error = $decoded['error'] ?? []; - $detail = "$context - HTTP $status"; - foreach (['type', 'code', 'param'] as $field) { - if (!empty($provider_error[$field])) { - $detail .= " $field=" . $provider_error[$field]; - } - } - if (!empty($provider_error['message'])) { - $detail .= ' - ' . $provider_error['message']; - } - logApp('AI', 'error', $detail); - return ['ok' => false, 'error' => 'The AI provider rejected the request - see Admin > App Logs.']; +/* + * What the editors' script needs to know (js/app.js reads window.itflowAi) - + * whether Reword is there in a ticket reply and elsewhere, and where to ask. + * False when it is in neither - then the editors have no Reword button. + */ +function aiClientConfig() { + if (empty($_SESSION['logged']) || !aiCan()) { + return false; } - - if (!isset($decoded['choices'][0]['message']['content'])) { - logApp('AI', 'error', "$context - HTTP $status but the response carried no choices[0].message.content"); - return ['ok' => false, 'error' => 'The AI provider returned an unexpected response - see Admin > App Logs.']; + $general = getAiModel('General') !== null; + $tickets = getAiModel('Tickets') !== null; + if (!$general && !$tickets) { + return false; } - - return ['ok' => true, 'content' => $decoded['choices'][0]['message']['content']]; + return ['reword' => $general, 'rewordTicket' => $tickets, 'endpoint' => '/agent/ajax.php']; } /* @@ -128,5 +145,709 @@ function callAiApi($model, $messages) { */ function aiModelMissingError($use_case) { logApp('AI', 'warning', "No AI model configured for use case '$use_case' and no General model to fall back on"); - return "No AI model is configured for $use_case. Add one under Admin > AI Models."; + return "No AI model is set up for $use_case. Add one in Admin > AI Providers."; +} + +/* ------------------------------------------------------------- Keeping secrets out */ + +/* + * Passwords, keys and the like taken out of text before it goes to a provider: + * [the text with each replaced by [secret-1], [secret-2]..., the placeholders => + * what they stand for]. The same secret twice gets the same placeholder. What + * comes back is put right with aiRestore(), so the provider never sees them and + * the answer still has them. Works on plain text and on an editor's HTML. + */ +function aiRedact($text, array &$secrets = []) { + $text = (string) $text; + $keep = function ($secret) use (&$secrets) { + $found = array_search($secret, $secrets, true); + if ($found !== false) { + return $found; + } + $placeholder = '[secret-' . (count($secrets) + 1) . ']'; + $secrets[$placeholder] = $secret; + return $placeholder; + }; + $whole = function ($m) use ($keep) { + return $keep($m[0]); + }; + + // Things that are secret whole - private keys, 2FA setup links, the keys services hand out + foreach ([ + '/-----BEGIN [A-Z ]*PRIVATE KEY-----.*?-----END [A-Z ]*PRIVATE KEY-----/s', + '#otpauth://[^\s<>"\']+#i', + '/\b(?:sk|rk|pk)_live_[A-Za-z0-9]{10,}\b/', + '/\b(?:sk|rk)_test_[A-Za-z0-9]{10,}\b/', + '/\bsk-(?:proj-|ant-[a-z0-9]*-?|or-v1-)?[A-Za-z0-9_-]{20,}/', + '/\b(?:AKIA|ASIA)[0-9A-Z]{16}\b/', + '/\b(?:gh[pousr]_[A-Za-z0-9]{30,}|github_pat_[A-Za-z0-9_]{30,})\b/', + '/\bglpat-[A-Za-z0-9_-]{20,}/', + '/\bxox[abprs]-[A-Za-z0-9-]{10,}/', + '/\bAIza[0-9A-Za-z_-]{35}\b/', + '/\beyJ[A-Za-z0-9_-]{10,}\.eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}/', + ] as $pattern) { + $text = preg_replace_callback($pattern, $whole, $text); + } + + // What follows "Password:", "PIN =", "API key:" and the like - the label stays. + // An editor's tags may sit around the colon (Password: ...). + $gap = '(?:\s| |<\/?[a-z][^>]*>)*'; + $value = '((?:&(?!nbsp;)|[^\s<>"\'&\x{201C}\x{201D}]){3,})'; + $tidy = function ($m) use ($keep) { + $secret = $m[2]; + if (strpos($secret, '[secret-') === 0) { + return $m[0]; + } + // A full stop or comma after it ends the sentence, not the password + $after = ''; + if (preg_match('/^(.{3,}?)([.,;)]+)$/su', $secret, $split)) { + [$secret, $after] = [$split[1], $split[2]]; + } + return $m[1] . $keep($secret) . $after; + }; + $labels = 'password|passwd|pwd|passphrase|passcode|pin|secret|client[ _-]?secret|api[ _-]?key|access[ _-]?key|secret[ _-]?key|token|otp|(?:2fa|mfa|totp) code|recovery codes?|wi-?fi key|psk'; + // In quotes, spaces and all - Password: "correct horse battery" + $text = preg_replace_callback("/(\\b(?:$labels)\\b{$gap}[:=]{$gap})([\"'\\x{201C}])([^\"'\\x{201D}<>\\n]{1,200})([\"'\\x{201D}])/iu", function ($m) use ($keep) { + return $m[1] . $m[2] . $keep($m[3]) . $m[4]; + }, $text); + $text = preg_replace_callback("/(\\b(?:$labels)\\b{$gap}[:=]{$gap}[\"'\\x{201C}]?)$value/iu", $tidy, $text); + // "The password is Summer2024!" - only when it looks like one, so "the password is wrong" stays + $text = preg_replace_callback("/(\\b(?:password|passcode|passphrase|pin)\\s+(?:is|was|will be)\\s+{$gap}[\"'\\x{201C}]?)(?=[^\\s<>\"']*[0-9!@#\$%^*_+=?~-])$value/iu", $tidy, $text); + + // Card numbers - 13 to 19 digits that pass the Luhn check + $text = preg_replace_callback('/\b\d(?:[ -]?\d){12,18}\b/', function ($m) use ($keep) { + $digits = preg_replace('/\D/', '', $m[0]); + $sum = 0; + foreach (array_reverse(str_split($digits)) as $i => $digit) { + $digit = intval($digit); + if ($i % 2) { + $digit *= 2; + $digit = $digit > 9 ? $digit - 9 : $digit; + } + $sum += $digit; + } + return $sum % 10 === 0 ? $keep($m[0]) : $m[0]; + }, $text); + + return [$text, $secrets]; +} + +/* An answer with the secrets aiRedact() took out put back where it left them. */ +function aiRestore($text, array $secrets) { + return $secrets ? strtr((string) $text, $secrets) : (string) $text; +} + +/* ------------------------------------------------------------- Calling a provider */ + +/* The headers a provider is sent - its key, as its kind takes it. */ +function aiProviderHeaders($kind, $api_key, $json = true) { + $headers = $json ? ['Content-Type: application/json'] : []; + $api_key = trim((string) $api_key); + if ($api_key !== '') { + $headers[] = aiProviderKind($kind)['auth'] === 'api-key' ? "api-key: $api_key" : "Authorization: Bearer $api_key"; + } + return $headers; +} + +/* + * Posts a chat-completion request for a feature (aiFeatures() - $context + * 'feature', and 'client_id' when it is about a client's things). Returns: + * + * ['ok' => true, 'content' => '...'] + * ['ok' => false, 'error' => 'short message safe to show the user'] + * + * The first message is the feature's own instructions (a system message). The + * model's instructions from Admin > AI Providers follow them. Passwords and keys + * are taken out first when the provider is set to, and put back in the answer. + * Every request is recorded in ai_usage. Provider detail on a failure - status, + * error type, code and message - goes to the app log so a misconfiguration is + * diagnosable. The API key and the message bodies never do. + */ +function callAiApi($model, $messages, array $context = []) { + $messages = array_values($messages); + if (($messages[0]['role'] ?? '') !== 'system') { + array_unshift($messages, ['role' => 'system', 'content' => 'You are a helpful assistant for an IT services company.']); + } + + $secrets = []; + if (!empty($model['ai_provider_redact'])) { + foreach ($messages as $i => $message) { + if ($message['role'] !== 'system') { + [$messages[$i]['content']] = aiRedact($message['content'], $secrets); + } + } + if ($secrets) { + $messages[0]['content'] .= "\n\nSome passwords and keys have been replaced with placeholders like [secret-1]. Keep each placeholder exactly as it is wherever it belongs in your answer, and don't remark on them."; + } + } + + // The company's own instructions for the model - house style, things to always or never say + $instructions = trim((string) ($model['ai_model_prompt'] ?? '')); + if ($instructions !== '') { + array_splice($messages, 1, 0, [['role' => 'system', 'content' => "Also follow these instructions from the company:\n$instructions"]]); + } + + $data = ['model' => $model['ai_model_name'], 'messages' => $messages]; + + // Only send a temperature when the model has one configured. Newer models + // accept nothing but their own default and 400 on anything else. + if (isset($model['ai_model_temperature']) && $model['ai_model_temperature'] !== '') { + $data['temperature'] = floatval($model['ai_model_temperature']); + } + + $started = microtime(true); + $ch = curl_init($model['ai_provider_api_url']); + curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); + curl_setopt($ch, CURLOPT_POST, true); + curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data)); + curl_setopt($ch, CURLOPT_TIMEOUT, AI_REQUEST_TIMEOUT); + curl_setopt($ch, CURLOPT_HTTPHEADER, aiProviderHeaders($model['ai_provider_kind'] ?? 'custom', $model['ai_provider_api_key'])); + + $response = curl_exec($ch); + $status = intval(curl_getinfo($ch, CURLINFO_RESPONSE_CODE)); + $transport_error = curl_error($ch); + curl_close($ch); + + // Enough to tell one provider/model pairing from another in the log + $label = $model['ai_provider_name'] . ' / ' . $model['ai_model_name']; + $usage = [ + 'feature' => (string) ($context['feature'] ?? 'other'), + 'model' => $model, + 'client_id' => intval($context['client_id'] ?? 0), + 'ms' => intval(round((microtime(true) - $started) * 1000)), + 'redacted' => count($secrets), + ]; + + if ($response === false) { + logApp('AI', 'error', "$label - could not reach the provider: $transport_error"); + aiUsageRecord($usage + ['ok' => false, 'error' => 'Could not reach the provider']); + return ['ok' => false, 'error' => 'Could not reach the AI provider.']; + } + + $decoded = json_decode($response, true); + + if ($status < 200 || $status > 299) { + $provider_error = $decoded['error'] ?? []; + $provider_error = is_array($provider_error) ? $provider_error : ['message' => (string) $provider_error]; + $detail = "$label - HTTP $status"; + foreach (['type', 'code', 'param'] as $field) { + if (!empty($provider_error[$field]) && is_scalar($provider_error[$field])) { + $detail .= " $field=" . $provider_error[$field]; + } + } + if (!empty($provider_error['message']) && is_scalar($provider_error['message'])) { + $detail .= ' - ' . $provider_error['message']; + } + logApp('AI', 'error', $detail); + aiUsageRecord($usage + ['ok' => false, 'error' => "HTTP $status" . (!empty($provider_error['message']) && is_scalar($provider_error['message']) ? ' - ' . $provider_error['message'] : '')]); + return ['ok' => false, 'error' => "The AI provider turned the request away (HTTP $status) - see Admin > App Logs."]; + } + + $content = $decoded['choices'][0]['message']['content'] ?? null; + if (!is_string($content)) { + logApp('AI', 'error', "$label - HTTP $status but the response carried no choices[0].message.content"); + aiUsageRecord($usage + ['ok' => false, 'error' => 'No answer in the response']); + return ['ok' => false, 'error' => 'The AI provider returned an unexpected response - see Admin > App Logs.']; + } + + aiUsageRecord($usage + ['ok' => true, 'tokens_in' => intval($decoded['usage']['prompt_tokens'] ?? 0), 'tokens_out' => intval($decoded['usage']['completion_tokens'] ?? 0)]); + + return ['ok' => true, 'content' => aiRestore($content, $secrets)]; +} + +/* + * An answer that should be HTML, as safe HTML - without the code fences or a + * leading "html" models like to add, and through the purifier the document + * pages use, so nothing in a ticket can make it carry a script. + */ +function aiCleanHtml($content) { + $content = trim((string) $content); + $content = preg_replace('/^```[a-z]*\s*/i', '', $content); + $content = preg_replace('/\s*```$/', '', $content); + $content = preg_replace('/^html\b\s*/i', '', $content); + return trim(documentHtmlPurify(str_replace('```', '', $content))); +} + +/* ------------------------------------------------------------- Usage */ + +/* + * Whether the 3.3.8 update has run - the providers' kind and redact columns, + * and ai_usage. Until it has (new files, database not yet updated), AI works + * as before and nothing is recorded. The database version says, where it is + * known; otherwise the table and column are looked for. + */ +function aiUsageTableExists() { + global $mysqli; + static $exists = null; + if ($exists === null) { + if (defined('CURRENT_DATABASE_VERSION') && version_compare(CURRENT_DATABASE_VERSION, '3.3.8', '>=')) { + $exists = true; + } else { + // Both what it adds - the table, and the providers' kind + $row = mysqli_fetch_row(mysqli_query($mysqli, "SELECT COUNT(*) FROM information_schema.COLUMNS WHERE TABLE_SCHEMA = DATABASE() + AND ((TABLE_NAME = 'ai_usage' AND COLUMN_NAME = 'ai_usage_id') OR (TABLE_NAME = 'ai_providers' AND COLUMN_NAME = 'ai_provider_kind'))")); + $exists = intval($row[0] ?? 0) === 2; + } + } + return $exists; +} + +/* Record one request - its feature, model (the row callAiApi() was given), client, tokens, time, how many secrets were taken out, and whether it worked. */ +function aiUsageRecord(array $usage) { + global $mysqli, $session_user_id; + if (!aiUsageTableExists()) { + return; + } + $model = $usage['model'] ?? []; + $text = function ($value, $length) { + return "'" . mysqli_real_escape_string($GLOBALS['mysqli'], mb_substr((string) $value, 0, $length)) . "'"; + }; + mysqli_query($mysqli, "INSERT INTO ai_usage SET + ai_usage_feature = " . $text($usage['feature'] ?? 'other', 50) . ", + ai_usage_use_case = " . $text($model['ai_model_use_case'] ?? '', 50) . ", + ai_usage_provider_id = " . intval($model['ai_provider_id'] ?? 0) . ", + ai_usage_provider_name = " . $text($model['ai_provider_name'] ?? '', 200) . ", + ai_usage_model_id = " . intval($model['ai_model_id'] ?? 0) . ", + ai_usage_model = " . $text($model['ai_model_name'] ?? '', 200) . ", + ai_usage_user_id = " . intval($session_user_id ?? 0) . ", + ai_usage_client_id = " . intval($usage['client_id'] ?? 0) . ", + ai_usage_tokens_in = " . intval($usage['tokens_in'] ?? 0) . ", + ai_usage_tokens_out = " . intval($usage['tokens_out'] ?? 0) . ", + ai_usage_ms = " . intval($usage['ms'] ?? 0) . ", + ai_usage_redacted = " . intval($usage['redacted'] ?? 0) . ", + ai_usage_ok = " . (!empty($usage['ok']) ? 1 : 0) . ", + ai_usage_error = " . (!empty($usage['error']) ? $text($usage['error'], 250) : 'NULL')); +} + +/* + * The last $days by provider or model ($by 'provider' or 'model') - id => + * requests, failed, tokens in and out, and when it was last used. Tests aren't counted. + */ +function aiUsageTotals($by = 'provider', $days = 30) { + global $mysqli; + if (!aiUsageTableExists()) { + return []; + } + $column = $by === 'model' ? 'ai_usage_model_id' : 'ai_usage_provider_id'; + $totals = []; + $sql = mysqli_query($mysqli, "SELECT $column, COUNT(*), SUM(ai_usage_ok = 0), SUM(ai_usage_tokens_in), SUM(ai_usage_tokens_out), MAX(ai_usage_created_at) + FROM ai_usage WHERE ai_usage_created_at >= NOW() - INTERVAL " . intval($days) . " DAY AND ai_usage_feature <> 'test' GROUP BY $column"); + while ($row = mysqli_fetch_row($sql)) { + $totals[intval($row[0])] = ['requests' => intval($row[1]), 'failed' => intval($row[2]), 'tokens_in' => intval($row[3]), 'tokens_out' => intval($row[4]), 'last' => $row[5]]; + } + return $totals; +} + +/* A number of tokens as it reads - 950, 12.4k, 3.1M. */ +function aiTokensText($tokens) { + $tokens = intval($tokens); + foreach ([1000000 => 'M', 1000 => 'k'] as $size => $suffix) { + if ($tokens >= $size) { + return rtrim(rtrim(number_format($tokens / $size, 1, '.', ''), '0'), '.') . $suffix; + } + } + return (string) $tokens; +} + +/* ------------------------------------------------------------- Testing a provider */ + +/* Where a provider lists its models - worked out from its chat completions address. '' when it doesn't list them. */ +function aiModelsUrl($kind, $url) { + $listing = aiProviderKind($kind)['models']; + if ($listing === 'anthropic') { + return 'https://api.anthropic.com/v1/models?limit=1000'; + } + $path = strtok((string) $url, '?'); + if ($listing === '' || !preg_match('#/chat/completions/?$#', (string) $path)) { + return ''; + } + return preg_replace('#/chat/completions/?$#', '/models', $path); +} + +/* + * Ask a provider which models it has - which says whether its address and key + * work. ['ok' => bool, 'models' => their names, 'error' => what went wrong in + * words, 'listed' => false when it doesn't list them (Azure) - then it isn't asked]. + */ +function aiProviderTest($kind, $url, $api_key) { + $models_url = aiModelsUrl($kind, $url); + if ($models_url === '') { + return ['ok' => true, 'models' => [], 'error' => '', 'listed' => false]; + } + $headers = aiProviderHeaders($kind, $api_key, false); + if (aiProviderKind($kind)['models'] === 'anthropic') { + $headers = ['x-api-key: ' . trim((string) $api_key), 'anthropic-version: 2023-06-01']; + } + $ch = curl_init($models_url); + curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); + curl_setopt($ch, CURLOPT_TIMEOUT, 20); + curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); + $response = curl_exec($ch); + $status = intval(curl_getinfo($ch, CURLINFO_RESPONSE_CODE)); + $transport_error = curl_error($ch); + curl_close($ch); + + if ($response === false) { + return ['ok' => false, 'models' => [], 'error' => "It couldn't be reached - $transport_error.", 'listed' => true]; + } + $decoded = json_decode($response, true); + if ($status === 401 || $status === 403) { + return ['ok' => false, 'models' => [], 'error' => "It turned the key away (HTTP $status) - check it is right and still active.", 'listed' => true]; + } + if ($status < 200 || $status > 299 || !is_array($decoded)) { + $message = is_array($decoded['error'] ?? null) ? (string) ($decoded['error']['message'] ?? '') : (is_string($decoded['error'] ?? null) ? $decoded['error'] : ''); + return ['ok' => false, 'models' => [], 'error' => "It answered HTTP $status" . ($message !== '' ? " - " . mb_substr($message, 0, 200) : '') . ' - check the address.', 'listed' => true]; + } + $models = []; + foreach ((array) ($decoded['data'] ?? $decoded['models'] ?? []) as $item) { + $id = is_array($item) ? (string) ($item['id'] ?? $item['name'] ?? '') : (string) $item; + if ($id !== '') { + $models[] = preg_replace('#^models/#', '', $id); + } + } + $models = array_values(array_unique($models)); + sort($models, SORT_NATURAL | SORT_FLAG_CASE); + return ['ok' => true, 'models' => $models, 'error' => '', 'listed' => true]; +} + +/* ------------------------------------------------------------- Admin > AI Providers */ + +/* A provider - with how many models it has - or null. */ +function aiProviderRow($provider_id) { + global $mysqli; + $provider_id = intval($provider_id); + if ($provider_id <= 0) { + return null; + } + return mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT ai_providers.*, (SELECT COUNT(*) FROM ai_models WHERE ai_model_ai_provider_id = ai_provider_id) AS model_count + FROM ai_providers WHERE ai_provider_id = $provider_id")) ?: null; +} + +/* A model, with its provider's name and kind - or null. */ +function aiModelRow($model_id) { + global $mysqli; + $model_id = intval($model_id); + if ($model_id <= 0) { + return null; + } + return mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT ai_models.*, ai_provider_name, ai_provider_kind FROM ai_models + LEFT JOIN ai_providers ON ai_provider_id = ai_model_ai_provider_id WHERE ai_model_id = $model_id")) ?: null; +} + +/* + * A provider's address as it is called - its chat completions address. One + * pasted as the base (https://api.example.com/v1) gets /chat/completions put + * on, as the OpenAI libraries would; an Azure resource on its own gets the + * rest of the path too. + */ +function aiProviderUrl($kind, $url) { + $url = trim((string) $url); + if ($url === '' || !preg_match('#^https?://#i', $url)) { + return $url; + } + $path = (string) parse_url($url, PHP_URL_PATH); + if ($kind === 'azure' && trim($path, '/') === '') { + return rtrim($url, '/') . '/openai/v1/chat/completions'; + } + if (preg_match('#/v\d+(?:beta\d*)?(?:/openai)?/?$#i', $path)) { + return rtrim($url, '/') . '/chat/completions'; + } + return $url; +} + +/* + * A provider as sent from its window, checked - [the row to save (raw values), + * field => what is wrong]. The key isn't put back into the page, so left empty + * on an existing provider it stays; 'clear_key' takes it away. + */ +function aiProviderCheck(array $post, $before = null) { + $kinds = aiProviderKinds(); + $errors = []; + $kind = (string) ($post['kind'] ?? ($before['ai_provider_kind'] ?? 'custom')); + if (!isset($kinds[$kind])) { + $errors['kind'] = 'Pick what kind of provider it is.'; + $kind = 'custom'; + } + $info = $kinds[$kind]; + + $name = trim(preg_replace('/\s+/', ' ', (string) ($post['name'] ?? ''))); + if ($name === '') { + $errors['name'] = 'Give it a name - ' . $info['name'] . ', or what you call it.'; + } elseif (mb_strlen($name) > 200) { + $errors['name'] = 'That name is longer than 200 characters.'; + } else { + global $mysqli; + $taken = mysqli_fetch_row(mysqli_query($mysqli, "SELECT ai_provider_id FROM ai_providers WHERE ai_provider_name = '" . mysqli_real_escape_string($mysqli, $name) . "' + AND ai_provider_id <> " . intval($before['ai_provider_id'] ?? 0) . " LIMIT 1")); + if ($taken) { + $errors['name'] = 'There is a provider called that already - give this one a name of its own.'; + } + } + + $url = aiProviderUrl($kind, $post['url'] ?? ''); + if ($url === '') { + $errors['url'] = 'Its address is needed - where ITFlow sends requests.'; + } elseif (mb_strlen($url) > 500) { + $errors['url'] = 'That address is longer than 500 characters.'; + } elseif (!filter_var($url, FILTER_VALIDATE_URL) || !preg_match('#^https?://#i', $url)) { + $errors['url'] = "That isn't a web address - it starts https:// (or http:// for one on your own network)."; + } elseif (stripos($url, 'YOUR-RESOURCE') !== false) { + $errors['url'] = 'Put your Azure resource name in place of YOUR-RESOURCE - it is in the Azure portal, under Keys and Endpoint.'; + } + + $key = (string) ($post['api_key'] ?? ''); + $key = trim($key); + $saved_key = (string) ($before['ai_provider_api_key'] ?? ''); + if (!empty($post['clear_key'])) { + $key_after = ''; + } else { + $key_after = $key !== '' ? $key : $saved_key; + } + if (mb_strlen($key_after) > 500) { + $errors['api_key'] = 'That key is longer than 500 characters.'; + } elseif ($key !== '' && preg_match('/\s/', $key)) { + $errors['api_key'] = "A key has no spaces in it - check it was copied whole."; + } elseif ($info['key'] && $key_after === '') { + $errors['api_key'] = $info['name'] . ' needs an API key' . ($info['keys_url'] !== '' ? ' - make one in its console.' : '.'); + } + + $after = [ + 'ai_provider_name' => $name, + 'ai_provider_kind' => $kind, + 'ai_provider_api_url' => $url, + 'ai_provider_api_key' => $key_after, + 'ai_provider_redact' => !empty($post['redact']) ? 1 : 0, + ]; + return [$after, $errors]; +} + +/* Whether what a provider is called through - kind, address, key - is new or changed, so worth asking it. */ +function aiProviderConnectionChanged(array $after, $before) { + if (!$before) { + return true; + } + return $after['ai_provider_kind'] !== (string) $before['ai_provider_kind'] || $after['ai_provider_api_url'] !== (string) $before['ai_provider_api_url'] + || $after['ai_provider_api_key'] !== (string) $before['ai_provider_api_key']; +} + +/* Save a provider - added when there is no $provider_id. Returns its id. */ +function aiProviderWrite(array $after, $provider_id = 0) { + global $mysqli; + $set = []; + foreach ($after as $column => $value) { + $set[] = is_int($value) ? "$column = $value" : "$column = " . ($column === 'ai_provider_api_key' && $value === '' ? 'NULL' : "'" . mysqli_real_escape_string($mysqli, $value) . "'"); + } + $provider_id = intval($provider_id); + if ($provider_id) { + mysqli_query($mysqli, "UPDATE ai_providers SET " . implode(', ', $set) . " WHERE ai_provider_id = $provider_id"); + return $provider_id; + } + mysqli_query($mysqli, "INSERT INTO ai_providers SET " . implode(', ', $set)); + return intval(mysqli_insert_id($mysqli)); +} + +/* What changed on a provider, a line each, from what to what - the key only as changed. Plain text. */ +function aiProviderChangeLines(array $before, array $after) { + $lines = []; + if ((string) $before['ai_provider_name'] !== $after['ai_provider_name']) { + $lines[] = "Name {$before['ai_provider_name']} → {$after['ai_provider_name']}"; + } + if ((string) $before['ai_provider_kind'] !== $after['ai_provider_kind']) { + $lines[] = 'Kind ' . aiProviderKind($before['ai_provider_kind'])['name'] . ' → ' . aiProviderKind($after['ai_provider_kind'])['name']; + } + if ((string) $before['ai_provider_api_url'] !== $after['ai_provider_api_url']) { + $lines[] = "Address {$before['ai_provider_api_url']} → {$after['ai_provider_api_url']}"; + } + $was_key = (string) $before['ai_provider_api_key']; + if ($was_key !== $after['ai_provider_api_key']) { + $lines[] = $was_key === '' ? 'API key added' : ($after['ai_provider_api_key'] === '' ? 'API key removed' : 'API key changed'); + } + if (intval($before['ai_provider_redact']) !== $after['ai_provider_redact']) { + $lines[] = $after['ai_provider_redact'] ? 'Passwords and keys taken out of what is sent' : 'Passwords and keys sent as they are'; + } + return $lines; +} + +/* A model as sent from its window, checked - [the row to save (raw values), field => what is wrong]. */ +function aiModelCheck(array $post) { + $errors = []; + $provider = aiProviderRow($post['provider'] ?? 0); + if (!$provider) { + $errors['provider'] = 'Pick the provider it is on.'; + } + $name = trim((string) ($post['model'] ?? '')); + if ($name === '') { + $errors['model'] = "Its name is needed, as the provider knows it" . ($provider ? ' - like ' . aiProviderKind($provider['ai_provider_kind'])['example'] : '') . '.'; + } elseif (mb_strlen($name) > 200) { + $errors['model'] = 'That name is longer than 200 characters.'; + } elseif (preg_match('/\s/', $name) && !in_array($provider['ai_provider_kind'] ?? '', ['azure', 'custom'], true)) { + $errors['model'] = "A model's name has no spaces - " . aiProviderKind($provider['ai_provider_kind'] ?? 'custom')['example'] . ', say.'; + } + $use_case = (string) ($post['use_case'] ?? ''); + if (!isset(aiUseCases()[$use_case])) { + $errors['use_case'] = 'Pick what it is for.'; + } + $temperature = trim((string) ($post['temperature'] ?? '')); + if ($temperature !== '' && (!is_numeric($temperature) || floatval($temperature) < 0 || floatval($temperature) > 2)) { + $errors['temperature'] = 'A temperature is from 0 to 2 - or leave it empty for the model\'s own.'; + } + $prompt = trim(str_replace("\r\n", "\n", (string) ($post['prompt'] ?? ''))); + if (mb_strlen($prompt) > 4000) { + $errors['prompt'] = 'Keep the instructions under 4,000 characters - they go with every request.'; + } + $after = [ + 'ai_model_ai_provider_id' => intval($provider['ai_provider_id'] ?? 0), + 'ai_model_name' => $name, + 'ai_model_use_case' => $use_case, + 'ai_model_temperature' => $temperature === '' ? null : round(floatval($temperature), 2), + 'ai_model_prompt' => $prompt, + ]; + return [$after, $errors, $provider]; +} + +/* Save a model - added when there is no $model_id. Returns its id. */ +function aiModelWrite(array $after, $model_id = 0) { + global $mysqli; + $set = "ai_model_ai_provider_id = " . intval($after['ai_model_ai_provider_id']) . ", + ai_model_name = '" . mysqli_real_escape_string($mysqli, $after['ai_model_name']) . "', + ai_model_use_case = '" . mysqli_real_escape_string($mysqli, $after['ai_model_use_case']) . "', + ai_model_temperature = " . ($after['ai_model_temperature'] === null ? 'NULL' : floatval($after['ai_model_temperature'])) . ", + ai_model_prompt = " . ($after['ai_model_prompt'] === '' ? 'NULL' : "'" . mysqli_real_escape_string($mysqli, $after['ai_model_prompt']) . "'"); + $model_id = intval($model_id); + if ($model_id) { + mysqli_query($mysqli, "UPDATE ai_models SET $set WHERE ai_model_id = $model_id"); + return $model_id; + } + mysqli_query($mysqli, "INSERT INTO ai_models SET $set"); + return intval(mysqli_insert_id($mysqli)); +} + +/* A temperature as it reads - "the model's own" when there is none. */ +function aiTemperatureText($temperature) { + return $temperature === null || $temperature === '' ? "the model's own" : rtrim(rtrim(number_format(floatval($temperature), 2, '.', ''), '0'), '.'); +} + +/* What changed on a model, a line each, from what to what - the instructions only as changed. Plain text. */ +function aiModelChangeLines(array $before, array $after) { + $lines = []; + if (intval($before['ai_model_ai_provider_id']) !== $after['ai_model_ai_provider_id']) { + $lines[] = 'Provider ' . ($before['ai_provider_name'] ?? '#' . $before['ai_model_ai_provider_id']) . ' → ' . (aiProviderRow($after['ai_model_ai_provider_id'])['ai_provider_name'] ?? ''); + } + if ((string) $before['ai_model_name'] !== $after['ai_model_name']) { + $lines[] = "Model {$before['ai_model_name']} → {$after['ai_model_name']}"; + } + if ((string) $before['ai_model_use_case'] !== $after['ai_model_use_case']) { + $lines[] = "For {$before['ai_model_use_case']} → {$after['ai_model_use_case']}"; + } + $was = $before['ai_model_temperature'] === null ? null : round(floatval($before['ai_model_temperature']), 2); + if ($was !== $after['ai_model_temperature']) { + $lines[] = 'Temperature ' . aiTemperatureText($was) . ' → ' . aiTemperatureText($after['ai_model_temperature']); + } + if (trim((string) $before['ai_model_prompt']) !== $after['ai_model_prompt']) { + $lines[] = $after['ai_model_prompt'] === '' ? 'Instructions removed' : (trim((string) $before['ai_model_prompt']) === '' ? 'Instructions added' : 'Instructions changed'); + } + return $lines; +} + +/* + * Which model each use case lands on - use case => ['model' => the model row + * it uses (or null), 'own' => whether that is its own or General's]. And + * 'unused' - the models nothing lands on, model id => the row used instead: a + * second model for a use case is never used, the first one added is. $without + * leaves out a provider's models, or a model ('provider' / 'model' => id) - what + * it would be once they are deleted. + */ +function aiUseCaseMap(array $without = []) { + global $mysqli; + $first = []; + $unused = []; + $sql = mysqli_query($mysqli, "SELECT ai_model_id, ai_model_name, ai_model_use_case, ai_model_ai_provider_id, ai_provider_name FROM ai_models + JOIN ai_providers ON ai_provider_id = ai_model_ai_provider_id + WHERE ai_model_ai_provider_id <> " . intval($without['provider'] ?? 0) . " AND ai_model_id <> " . intval($without['model'] ?? 0) . " + ORDER BY ai_model_id ASC"); + while ($row = mysqli_fetch_assoc($sql)) { + $use_case = (string) $row['ai_model_use_case']; + if (!isset(aiUseCases()[$use_case])) { + $unused[intval($row['ai_model_id'])] = null; + } elseif (isset($first[$use_case])) { + $unused[intval($row['ai_model_id'])] = $first[$use_case]; + } else { + $first[$use_case] = $row; + } + } + $map = []; + foreach (aiUseCases() as $use_case => $about) { + $own = $first[$use_case] ?? null; + $map[$use_case] = ['model' => $own ?? ($first['General'] ?? null), 'own' => $own !== null]; + } + return ['map' => $map, 'unused' => $unused]; +} + +/* What would stop if a provider or model went ($without as aiUseCaseMap()) - the use cases that have a model now and wouldn't. */ +function aiUseCasesStopping(array $without) { + $now = aiUseCaseMap()['map']; + $then = aiUseCaseMap($without)['map']; + $stopping = []; + foreach ($now as $use_case => $lands) { + if ($lands['model'] && !$then[$use_case]['model']) { + $stopping[] = $use_case; + } + } + return $stopping; +} + +/* A provider kind's icon - Font Awesome classes. */ +function aiProviderIcon($kind) { + $icons = ['azure' => 'fab fa-microsoft', 'gemini' => 'fab fa-google', 'ollama' => 'fas fa-server', 'lmstudio' => 'fas fa-server']; + return $icons[(string) $kind] ?? 'fas fa-robot'; +} + +/* + * Which model each use case lands on, a pill each - for the top of AI + * Providers and AI Models, so it is plain what Reword, the ticket summary and + * document drafts will use. + */ +function aiUseCaseStripHtml() { + $html = '
    '; + foreach (aiUseCaseMap()['map'] as $use_case => $lands) { + $model = $lands['model']; + if (!$model) { + $says = 'nothing set up'; + $tone = 'border-warning-subtle bg-warning-subtle'; + } elseif ($lands['own']) { + $says = '' . escapeHtml($model['ai_model_name']) . ' on ' . escapeHtml($model['ai_provider_name']) . ''; + $tone = 'bg-body'; + } else { + $says = 'General\'s - ' . escapeHtml($model['ai_model_name']) . ''; + $tone = 'bg-body'; + } + $html .= '
    ' + . '' + . escapeHtml($use_case) . ' ' . $says . '
    '; + } + return $html . '
    '; +} + +/* + * What Admin > Settings says needs attention about AI - requests failing in + * the last day while the last one failed too (so it is still failing). Items + * as the overview takes them: [level, icon, title, text, url, button], plain text. + */ +function aiProblems() { + global $mysqli; + if (!aiUsageTableExists()) { + return []; + } + $row = mysqli_fetch_row(mysqli_query($mysqli, "SELECT COUNT(*) FROM ai_usage WHERE ai_usage_feature <> 'test' AND ai_usage_ok = 0 AND ai_usage_created_at >= NOW() - INTERVAL 1 DAY")); + $failed = intval($row[0] ?? 0); + if (!$failed) { + return []; + } + $last = mysqli_fetch_assoc(mysqli_query($mysqli, "SELECT ai_usage_ok, ai_usage_provider_name, ai_usage_error FROM ai_usage WHERE ai_usage_feature <> 'test' ORDER BY ai_usage_id DESC LIMIT 1")); + if (!$last || intval($last['ai_usage_ok']) === 1) { + return []; + } + return [['warning', 'fa-robot', $failed . ' AI request' . ($failed === 1 ? '' : 's') . ' failed today', + 'The last, to ' . $last['ai_usage_provider_name'] . ', too' . ($last['ai_usage_error'] ? ' - ' . $last['ai_usage_error'] : '') . '. Reword and summaries fail for agents until it is put right.', + 'ai_usage.php?view=failed', 'See why']]; } diff --git a/functions/audit_log.php b/functions/audit_log.php index 9b86dd28f..165f80128 100644 --- a/functions/audit_log.php +++ b/functions/audit_log.php @@ -109,7 +109,7 @@ function auditLogKinds() { 'Custom Field' => ['icon' => 'fa-th-list'], 'Custom Link' => ['icon' => 'fa-external-link-alt'], 'AI Provider' => ['icon' => 'fa-robot'], - 'AI Model' => ['icon' => 'fa-robot'], + 'AI Model' => ['icon' => 'fa-brain'], 'Starter Content' => ['icon' => 'fa-seedling'], 'Demo Data' => ['icon' => 'fa-flask'], 'Report' => ['icon' => 'fa-chart-bar', 'module' => 'module_reporting'], diff --git a/functions/roles.php b/functions/roles.php index 631335e2c..9e71527dd 100644 --- a/functions/roles.php +++ b/functions/roles.php @@ -52,6 +52,10 @@ function roleModuleInfo() { 'label' => 'Reports', 'icon' => 'fa-chart-line', 'about' => 'The reports - each needs read access to what it reports on too', 'levels' => [1 => 'Open the reports'], 'max' => 1, ], + 'module_ai' => [ + 'label' => 'AI', 'icon' => 'fa-robot', 'about' => 'Rewording, summaries and drafts, by the AI providers in Admin - each needs access to what it works on too', + 'levels' => [1 => 'Use AI'], 'max' => 1, + ], ]; } diff --git a/includes/footer.php b/includes/footer.php index a69261cd0..40b0d7fee 100644 --- a/includes/footer.php +++ b/includes/footer.php @@ -76,6 +76,8 @@ + + diff --git a/js/app.js b/js/app.js index e410401dd..15dfeea98 100644 --- a/js/app.js +++ b/js/app.js @@ -83,6 +83,77 @@ function itflowCannedAutocompleter(editor) { }); } +/* + * Reword with AI - on an editor only when AI is set up for it and the agent's + * role may use it: window.itflowAi, printed by includes/footer.php from + * aiClientConfig() (functions/ai.php). The ticket editor asks the Tickets + * model. The answer replaces the text in one step, so Undo puts it back. A + * textarea's data-ai-client-id says the client it is for, for AI Usage. + */ +function itflowAiOn(ticket) { + var ai = window.itflowAi || null; + return !!(ai && (ticket ? ai.rewordTicket : ai.reword)); +} + +function itflowAiReword(editor, ticket) { + if (!itflowAiOn(ticket)) { + return; + } + var ai = window.itflowAi; + var buttonApi = null; + var say = function (text, type) { + editor.notificationManager.open({ text: text, type: type, timeout: type === 'success' ? 4000 : 8000 }); + }; + editor.ui.registry.addButton('reword', { + icon: 'ai', + tooltip: 'Reword with AI', + onAction: function () { + if (editor.getContent({ format: 'text' }).trim() === '') { + say('Write something first - Reword tidies up what is there.', 'info'); + return; + } + var element = editor.getElement(); + var clientId = element ? parseInt(element.getAttribute('data-ai-client-id') || '0', 10) : 0; + buttonApi.setEnabled(false); + editor.setProgressState(true); + fetch(ai.endpoint + '?ai_reword' + (ticket ? '&use_case=Tickets' : ''), { + method: 'POST', + credentials: 'same-origin', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ text: editor.getContent(), client_id: clientId || 0 }) + }) + .then(function (response) { + return response.json().catch(function () { + throw new Error('HTTP ' + response.status); + }); + }) + .then(function (data) { + // Leave the user's text alone if the reword failed + if (data.error || !data.rewordedText) { + say(data.error || 'Could not reword the text.', 'error'); + return; + } + editor.undoManager.add(); + editor.undoManager.transact(function () { + editor.setContent(data.rewordedText); + }); + say('Reworded - Undo puts your words back.', 'success'); + }) + .catch(function () { + say('Could not reach ITFlow to reword it - try again.', 'error'); + }) + .finally(function () { + editor.setProgressState(false); + buttonApi.setEnabled(true); + }); + }, + onSetup: function (api) { + buttonApi = api; + return function () {}; + } + }); +} + /** * TinyMCE skin options for the current colour mode. * @@ -288,7 +359,9 @@ function itflowInit() { branding: false, menubar: false, statusbar: false, + // Reword first, where it is there - in a narrow editor the end of the toolbar folds away toolbar: [ + ...(itflowAiOn(false) ? [{ name: 'ai', items: ['reword'] }] : []), { name: 'styles', items: ['styles'] }, { name: 'formatting', items: ['bold', 'italic', 'forecolor'] }, { name: 'link', items: ['link'] }, @@ -297,7 +370,7 @@ function itflowInit() { { name: 'indentation', items: ['outdent', 'indent'] }, { name: 'table', items: ['table'] }, { name: 'extra', items: ['code', 'fullscreen'] }, - { name: 'ai', items: ['reword', 'undo', 'redo'] } + { name: 'history', items: ['undo', 'redo'] } ], mobile: { menubar: false, @@ -320,73 +393,7 @@ function itflowInit() { } }); - var rewordButtonApi; - - editor.ui.registry.addButton('reword', { - icon: 'ai', - tooltip: 'Reword Text', - onAction: function() { - var content = editor.getContent(); - - // Disable the Reword button - rewordButtonApi.setEnabled(false); - - // Show the progress indicator - editor.setProgressState(true); - - fetch('ajax.php?ai_reword', { - method: 'POST', - headers: { - 'Content-Type': 'application/json', - }, - body: JSON.stringify({ text: content }), - }) - .then(response => { - if (!response.ok) { - throw new Error('Network response was not ok'); - } - return response.json(); - }) - .then(data => { - editor.setProgressState(false); - rewordButtonApi.setEnabled(true); - - // Leave the user's text alone if the reword failed - if (data.error || !data.rewordedText) { - editor.notificationManager.open({ - text: data.error || 'Could not reword the text.', - type: 'error', - timeout: 8000 - }); - return; - } - - editor.undoManager.transact(function() { - editor.setContent(data.rewordedText); - }); - - editor.notificationManager.open({ - text: 'Text reworded successfully!', - type: 'success', - timeout: 3000 - }); - }) - .catch(error => { - console.error('Error:', error); - editor.setProgressState(false); - rewordButtonApi.setEnabled(true); - editor.notificationManager.open({ - text: 'An error occurred while rewording the text.', - type: 'error', - timeout: 5000 - }); - }); - }, - onSetup: function(buttonApi) { - rewordButtonApi = buttonApi; - return function() {}; - } - }); + itflowAiReword(editor, false); } }); @@ -404,24 +411,26 @@ function itflowInit() { menubar: false, statusbar: false, toolbar: [ + ...(itflowAiOn(true) ? [{ name: 'ai', items: ['reword'] }] : []), { name: 'styles', items: ['styles'] }, { name: 'formatting', items: ['bold', 'italic', 'forecolor'] }, { name: 'link', items: ['link'] }, { name: 'lists', items: ['bullist', 'numlist'] }, { name: 'indentation', items: ['outdent', 'indent'] }, - { name: 'ai', items: ['reword', 'undo', 'redo'] }, + { name: 'history', items: ['undo', 'redo'] }, { name: 'custom', items: ['redactButton'] }, { name: 'code', items: ['code'] }, ], mobile: { menubar: false, toolbar: [ + ...(itflowAiOn(true) ? [{ name: 'ai', items: ['reword'] }] : []), { name: 'styles', items: ['styles'] }, { name: 'formatting', items: ['bold', 'italic', 'forecolor'] }, { name: 'link', items: ['link'] }, { name: 'lists', items: ['bullist', 'numlist'] }, { name: 'indentation', items: ['outdent', 'indent'] }, - { name: 'ai', items: ['reword', 'undo', 'redo'] }, + { name: 'history', items: ['undo', 'redo'] }, { name: 'custom', items: ['redactButton'] }, { name: 'code', items: ['code'] }, ], @@ -442,64 +451,7 @@ function itflowInit() { } }); - var rewordButtonApi; - - editor.ui.registry.addButton('reword', { - icon: 'ai', - tooltip: 'Reword Text', - onAction: function() { - var content = editor.getContent(); - rewordButtonApi.setEnabled(false); - editor.setProgressState(true); - - fetch('ajax.php?ai_reword&use_case=Tickets', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ text: content }), - }) - .then(response => { - if (!response.ok) throw new Error('Network response was not ok'); - return response.json(); - }) - .then(data => { - editor.setProgressState(false); - rewordButtonApi.setEnabled(true); - - // Leave the user's text alone if the reword failed - if (data.error || !data.rewordedText) { - editor.notificationManager.open({ - text: data.error || 'Could not reword the text.', - type: 'error', - timeout: 8000 - }); - return; - } - - editor.undoManager.transact(function() { - editor.setContent(data.rewordedText); - }); - editor.notificationManager.open({ - text: 'Text reworded successfully!', - type: 'success', - timeout: 3000 - }); - }) - .catch(error => { - console.error('Error:', error); - editor.setProgressState(false); - rewordButtonApi.setEnabled(true); - editor.notificationManager.open({ - text: 'An error occurred while rewording the text.', - type: 'error', - timeout: 5000 - }); - }); - }, - onSetup: function(buttonApi) { - rewordButtonApi = buttonApi; - return function() {}; - } - }); + itflowAiReword(editor, true); editor.ui.registry.addButton('redactButton', { icon: 'permanent-pen', diff --git a/setup/seed_data.php b/setup/seed_data.php index f88bfe55a..1efc3dee4 100644 --- a/setup/seed_data.php +++ b/setup/seed_data.php @@ -92,6 +92,7 @@ function seedDefaultData($mysqli) mysqli_query($mysqli, "INSERT INTO modules SET module_name = 'module_sales', module_description = 'Access to quotes, invoices and products'"); mysqli_query($mysqli, "INSERT INTO modules SET module_name = 'module_financial', module_description = 'Access to payments, accounts, expenses and budgets'"); mysqli_query($mysqli, "INSERT INTO modules SET module_name = 'module_reporting', module_description = 'Access to all reports'"); + mysqli_query($mysqli, "INSERT INTO modules SET module_name = 'module_ai', module_description = 'Using AI - rewording, summaries and drafts'"); // Add default roles mysqli_query($mysqli, "INSERT INTO user_roles SET role_id = 1, role_name = 'Accountant', role_description = 'Built-in - Limited access to financial-focused modules'"); @@ -100,12 +101,14 @@ function seedDefaultData($mysqli) mysqli_query($mysqli, "INSERT INTO user_role_permissions SET user_role_id = 1, module_id = 4, user_role_permission_level = 1"); // Read sales mysqli_query($mysqli, "INSERT INTO user_role_permissions SET user_role_id = 1, module_id = 5, user_role_permission_level = 3"); // Full financial mysqli_query($mysqli, "INSERT INTO user_role_permissions SET user_role_id = 1, module_id = 6, user_role_permission_level = 1"); // Read reports + mysqli_query($mysqli, "INSERT INTO user_role_permissions SET user_role_id = 1, module_id = 7, user_role_permission_level = 1"); // Use AI mysqli_query($mysqli, "INSERT INTO user_roles SET role_id = 2, role_name = 'Technician', role_description = 'Built-in - Limited access to technical-focused modules'"); mysqli_query($mysqli, "INSERT INTO user_role_permissions SET user_role_id = 2, module_id = 1, user_role_permission_level = 2"); // Modify clients mysqli_query($mysqli, "INSERT INTO user_role_permissions SET user_role_id = 2, module_id = 2, user_role_permission_level = 2"); // Modify support mysqli_query($mysqli, "INSERT INTO user_role_permissions SET user_role_id = 2, module_id = 3, user_role_permission_level = 2"); // Modify credentials mysqli_query($mysqli, "INSERT INTO user_role_permissions SET user_role_id = 2, module_id = 4, user_role_permission_level = 2"); // Modify sales + mysqli_query($mysqli, "INSERT INTO user_role_permissions SET user_role_id = 2, module_id = 7, user_role_permission_level = 1"); // Use AI mysqli_query($mysqli, "INSERT INTO user_roles SET role_id = 3, role_name = 'Administrator', role_description = 'Built-in - Full administrative access to all modules (including user management)', role_is_admin = 1");