mirror of
https://github.com/itflow-org/itflow
synced 2026-11-19 10:57:24 +00:00
Speed up Transactions, Reconcile and the transactions export again by reading the ledger lines first and joining each to its account with STRAIGHT_JOIN instead of letting the optimizer look lines up through a temporary index, and stop billable expenses waiting to be invoiced from slowing every page: database update 3.5.5 indexes expenses.expense_refund_of_id so the side menu count, the invoice page, Add Expenses and the expense quick look no longer check each waiting expense against every expense; 26.11.2 changelog updated
This commit is contained in:
@@ -6,11 +6,12 @@ This file documents all notable changes made to ITFlow.
|
||||
|
||||
### Upgrading
|
||||
|
||||
Update from Maintenance > Update — Queue Update hands the job to cron, which updates the files and applies this release's one database change (3.5.3 to 3.5.4) in the same pass. Nothing else is required.
|
||||
Update from Maintenance > Update — Queue Update hands the job to cron, which updates the files and applies this release's two database changes (3.5.3 to 3.5.5) in the same pass. Nothing else is required.
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Transactions: the list, its Transfers pill, Reconcile and the transactions export got slower as the books grew, taking several seconds with a few years of history, and saving a new transfer waited on the same list. They load quickly again. On a test with eight years of books, the Transfers pill went from over six seconds to about a third of a second.
|
||||
- Transactions: the list, its Transfers pill, Reconcile and the transactions export got slower as the books grew, taking several seconds with a few years of history, and saving a new transfer waited on the same list. They load quickly again. On a test with eight years of books, the Transfers pill went from over six seconds to under a fifth of a second.
|
||||
- Expenses: billable expenses waiting to go on an invoice slowed down every page, because the side menu counts them and each one was checked against every expense. With a few hundred waiting, that was about a third of a second on every page. The invoice page, Add Expenses and an expense's quick look had the same delay. All of them are quick again.
|
||||
|
||||
### Developer Updates
|
||||
|
||||
@@ -19,6 +20,8 @@ Update from Maintenance > Update — Queue Update hands the job to cron, which u
|
||||
- `agent/transactions.php` passes it for one account sorted by date, the only view that shows the balance.
|
||||
- The transactions export passes it for one account. Reconcile (`reconcileLines()` and the uncleared count in `agent/reconcile.php`) never does.
|
||||
- `agent/transactions.php` works out the Transfers summary cards in the same query as the pill counts and totals, so the Transfers pill reads the ledger twice instead of three times.
|
||||
- `accountLedgerSql()` reads its lines first and joins each to its account with `STRAIGHT_JOIN`. Left to choose, the optimizer started from the accounts and found their lines through an index it built on the lines' temporary table (`derived_with_keys`), which was two to three times slower.
|
||||
- Database 3.5.5 indexes `expenses.expense_refund_of_id`. `expenseUnbilledCount()` (the side menu, on every agent page), `expenseUnbilled()`, `expenseOnInvoice()` and `expenseRow()` sum each expense's refunds in a correlated subquery, which scanned every expense for each one. Like 3.5.4, it is safe to run again, and `db.sql` has the key.
|
||||
|
||||
## [26.11.1]
|
||||
|
||||
|
||||
26
admin/database_updates/3.5.5.php
Normal file
26
admin/database_updates/3.5.5.php
Normal file
@@ -0,0 +1,26 @@
|
||||
<?php
|
||||
|
||||
/*
|
||||
* ITFlow - Database update to version 3.5.5 (from 3.5.4)
|
||||
* Included by admin/database_updates.php - do not access directly
|
||||
*
|
||||
* Index expenses.expense_refund_of_id - the expense a refund gives back.
|
||||
* What is left to bill of an expense is its amount less its refunds, and
|
||||
* each expense waiting to be billed looked its refunds up with nothing to
|
||||
* find them by, so every one was checked against every expense. The side
|
||||
* menu counts them on every agent page (expenseUnbilledCount()), and the
|
||||
* invoice page, Add Expenses and an expense's quick look do the same.
|
||||
*
|
||||
* Re-runnable: skipped where the index is already there.
|
||||
*/
|
||||
|
||||
defined('FROM_DB_UPDATER') || die("Direct file access is not allowed");
|
||||
|
||||
$itflow_index_exists = mysqli_query($mysqli, "SELECT 1 FROM information_schema.STATISTICS
|
||||
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'expenses' AND INDEX_NAME = 'expense_refund_of_id' LIMIT 1");
|
||||
|
||||
if (!$itflow_index_exists || mysqli_num_rows($itflow_index_exists) === 0) {
|
||||
mysqli_query($mysqli, "ALTER TABLE `expenses` ADD KEY `expense_refund_of_id` (`expense_refund_of_id`)");
|
||||
}
|
||||
|
||||
unset($itflow_index_exists);
|
||||
3
db.sql
3
db.sql
@@ -1746,7 +1746,8 @@ CREATE TABLE `expenses` (
|
||||
KEY `expense_invoice_id` (`expense_invoice_id`),
|
||||
KEY `expense_import_key` (`expense_import_key`),
|
||||
KEY `expense_reimbursement_id` (`expense_reimbursement_id`),
|
||||
KEY `expense_reconciliation_id` (`expense_reconciliation_id`)
|
||||
KEY `expense_reconciliation_id` (`expense_reconciliation_id`),
|
||||
KEY `expense_refund_of_id` (`expense_refund_of_id`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;
|
||||
/*!40101 SET character_set_client = @saved_cs_client */;
|
||||
|
||||
|
||||
@@ -363,6 +363,11 @@ function accountLedgerKinds() {
|
||||
* it is only worked out with $with_balance - where it is shown. Without it,
|
||||
* txn_balance is NULL: counts, totals and lists across every account don't
|
||||
* need it.
|
||||
*
|
||||
* The lines are read first and each looks its account up (STRAIGHT_JOIN).
|
||||
* Left to choose, the optimizer starts from the few accounts and finds their
|
||||
* lines through an index it builds on the lines' temporary table, which is
|
||||
* two to three times slower.
|
||||
*/
|
||||
function accountLedgerSql($account_id = 0, $with_balance = true) {
|
||||
$account_id = intval($account_id);
|
||||
@@ -416,7 +421,7 @@ function accountLedgerSql($account_id = 0, $with_balance = true) {
|
||||
" . ($with_balance ? "accounts.opening_balance + SUM(ledger_lines.txn_amount) OVER (PARTITION BY ledger_lines.txn_account_id ORDER BY ledger_lines.txn_date ASC, ledger_lines.txn_created_at ASC,
|
||||
ledger_lines.txn_order ASC, ledger_lines.txn_id ASC)" : "NULL") . " AS txn_balance
|
||||
FROM ($rows) AS ledger_lines
|
||||
JOIN accounts ON accounts.account_id = ledger_lines.txn_account_id
|
||||
STRAIGHT_JOIN accounts ON accounts.account_id = ledger_lines.txn_account_id
|
||||
LEFT JOIN accounts AS other_account ON ledger_lines.txn_kind IN ('transfer_in', 'transfer_out') AND other_account.account_id = ledger_lines.txn_other_account_id
|
||||
LEFT JOIN account_reconciliations AS cleared ON cleared.reconciliation_id = ledger_lines.txn_reconciliation_id AND cleared.reconciliation_finished_at IS NOT NULL
|
||||
LEFT JOIN categories ON category_id = ledger_lines.txn_category_id
|
||||
|
||||
Reference in New Issue
Block a user