Nobody knew. The backup counted as working until the first attempt to restore it.

What was inside

The built-in Bitrix24 backup produced files of 115 to 120 GB. A respectable size, a fresh date, a "complete" mark. The backup had no b_user, b_crm_deal, b_im_message, b_tasks_* or b_disk_* tables. In other words, nothing a backup exists for.

The dump stopped at b_crm_access_attr_dynamic_1154. The only explanation was in the event log, under the BACKUP_ERROR filter:

Table b_crm_access_attr_dynamic_1160 is corrupted due to an internal MySQL error.

The table isn't corrupted, it doesn't exist

The Bitrix message sends you the wrong way. b_crm_access_attr_dynamic_1160 isn't damaged. It simply doesn't exist. SHOW CREATE TABLE returns error 1146, "table doesn't exist", and the MySQL data dictionary has no trace of it.

Here is how it happened:

1. At the start of a run, Bitrix builds a list of tables and works through it.
2. On a database this size, a full pass took about a week. The file itself shows it: the name says July 28, the last write was August 4.
3. During that week someone deleted a smart process. A routine operation that any CRM administrator does. Its permissions table went with it.
4. A few days later the export reached that table on its list, couldn't find it and stopped.
5. Tables are exported in alphabetical order, and the break came early. Everything that mattered never made it into the backup.

This isn't a one-off. Smart process IDs on this portal have more than fifteen gaps: they get deleted regularly, and any deletion during a week-long run breaks the backup again.

Why nobody saw it

A failed backup shows up only as an entry in the event log. The log is purged after its retention period, and there are no failure alerts. The administrator sees a file of the right size with the right date and has no reason to doubt it.

The owner sees even less. In the work plan, "backups configured" is a closed item, and it comes up again on the day a backup is really needed. That is the day you find out there is nothing to restore.

A complete backup can fail too

The next backup ran all the way through: 2,230 tables. Restoring it failed halfway:

ERROR 1062 (23000): Duplicate entry '7920399-1141386-2' for key 'b_crm_timeline_bind.PRIMARY'

A duplicate primary key means the same row was exported twice. Bitrix reads tables in chunks without a transaction while the portal keeps working. For tables without a numeric key, it reads by offset (LIMIT offset, N). A new row in the middle of the table shifts the offsets, and some rows get exported again. The export runs for hours, so the odds of a duplicate are high.

When restoring by hand you can work around it: replace INSERT with INSERT IGNORE, and the repeated row is dropped. Nothing is lost, it's the same row. But the point stands: a backup can contain every table and still not restore without manual work.

What to do about it

There is nothing to repair in the table. The options:

• Restart the backup. The table list is rebuilt and correct. But it's a week-long gamble: the next deleted smart process or pipeline breaks it again.
• Freeze deletions of smart processes, pipelines and infoblocks during the run. A workaround for one attempt. On a portal with hundreds of employees it won't hold.
• Make the run shorter. This is the real fix. On this portal, the business process log b_bp_tracking took 182 GB, while the working data (deals, tasks, messages, users) was about 15 GB. Without that log, the backup goes from a week to about an hour, and the window for this failure almost closes.

And one rule that always holds: a backup is verified only once you have restored from it. File size, date and a green mark mean nothing.

How to check your backup

Quick check: are the main tables in the dump at all? If there are no users, you can stop there:

grep -c '^DROP TABLE IF EXISTS `b_user`' dump.sql

Count the tables in the dump and compare with the live database:

LC_ALL=C grep -ohE '^DROP TABLE IF EXISTS `[^`]+`' dump.sql | sort -u | wc -l

Check that the permission tables match the smart processes that actually exist. If there are more or fewer tables than live types, the next run may trip:

SELECT ENTITY_TYPE_ID, TITLE FROM b_crm_dynamic_type ORDER BY ENTITY_TYPE_ID;

SELECT TABLE_NAME FROM information_schema.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME LIKE 'b_crm_access_attr_dynamic_%';

And once a month or once a quarter, restore a backup on a separate server. It's the only check after which you can say you have a backup.

An analysis of 7,753 product rows showed the opposite. 1C does the math correctly. The kopecks come from Bitrix24, and its documentation doesn't say a word about it.

Where the kopecks come from

Bitrix24 smart invoices don't round the price to the kopeck when they save a product row. The database holds prices like 805.23809523, 232.66285715, 3784.725. Out of 7,753 rows on this portal, 6,996 had a pre-tax price longer than two decimals. That's 90%.

How such a price appears:

• The manager enters the row total, not the price. Bitrix24 divides the total by the quantity and stores the result as is: 4 × 1625.00000001 = 6500.00.
• Percentage discounts. 7,737 of 7,753 rows had one, and almost any percentage discount leaves a tail on the price.
• The 5% VAT rate. The price with tax is divided by 1.05, which almost always gives an endless fraction.

1C can't store prices like that: a document price is in rubles and kopecks. During the exchange it truncates the price to the kopeck, and the error is multiplied by the quantity. At a quantity of 214, a single row can be off by up to 43 kopecks. The worst invoice on the portal was off by 94 kopecks.

Bitrix24 calculates one document two ways

The second cause is subtler. Within a single invoice, Bitrix24 rounds differently:

• the invoice total is calculated from the sum: add up all rows, round once at the end;
• the tax is calculated per row: round each row's VAT, then add them up.

On the 405 invoices where the two methods give different results, the tax matched the per-row calculation in 404. We had to reconstruct the formulas from the core code (CCrmProductRow::preparePrices) and verify them on all 7,753 rows: the REST documentation describes the price fields without any formulas.

Why you can't adjust 1C

The first idea is usually to tune rounding in 1C so it matches Bitrix. But here 1C follows the law.

• The Russian Tax Code (Article 168, clause 2) requires VAT to be calculated for each line item. So the per-row tax total is the correct one.
• The rules for VAT invoices (Government Resolution No. 1137) require rubles and kopecks. Rounding to whole rubles isn't allowed.

So "fixing" 1C means breaking its correct calculation to match an incorrect one.

What doesn't help

All tested, so you don't have to:

• Price rounding rules in Bitrix24. That's a product catalog feature. It doesn't apply to CRM invoice product rows.
• Precision settings and changing the column type in the database. Only works on-premise. This portal is in the cloud, with no access to files or the database.
• Rounding in 1C. Against the law, see above.

What to send to 1C

The fix belongs in the exchange:

1. Send the row total, not the price. The row total in Bitrix24 is correct. Only the price obtained by division is off.
2. Calculate the document total as the sum of rounded rows. That's not the "Amount" field (opportunity) on the invoice card: it's calculated from the sum.
3. Calculate VAT per row, as the law requires.
4. If the total is fixed, put the kopeck difference on one row, the one with the largest amount. Don't spread it across all rows. 1C:Trade Management 11 has a dedicated discount type for exactly this, "СкидкаНаценкаОкругление" (rounding adjustment).

Two traps along the way

crm.item.productrow.list returns at most 50 rows. It's paginated, and without walking the pages an invoice's rows are silently cut off at the fiftieth. Four invoices on this portal were cut this way, with up to 58 rows each. On the first pass they showed "unexplained" VAT. After loading the missing rows, they matched. If your 1C integration pulls rows with this method, check that it walks the pages. Otherwise 1C receives incomplete invoices.

The "tax included in price" flag isn't always true. One invoice had 51 rows where the flag was off but the price already included tax: 441 against a pre-tax price of 420, exactly 420 × 1.05. The receiving system will add VAT a second time, and the difference will be 5%, not a kopeck.

The log showed why: almost every incoming WhatsApp message reached the portal dated the year 58520.

What the log showed

I took five days of imopenlines.log: 57 MB, about 30,000 events. On this portal 96% of open line traffic came through Wazzup (WhatsApp).

Incoming CONNECTOR - ENTITY ADD events carry a message date. In 7,111 of 7,224 cases (98.4%) it looked like this:

[date] => 58520-02-06 18:00:01.000000

The same log also had correct dates: 1784705238 and 2026-07-20T06:45:25-06:00. So the portal handles time correctly in general, and one source breaks it.

Where the year 58520 comes from

With the integration settings this portal had, Wazzup sent the message time in milliseconds: 1784705238000, thirteen digits. The connector on the Bitrix24 side reads that number as seconds. 1,784,705,238,000 seconds after 1970 lands in the year 58520.

The correct value in seconds, 1784705238, is July 20, 2026. Three extra zeros.

Why it matters

A wrong date on a message looks harmless. In practice it switches off the cleanup.

Open line agents close and purge sessions with a simple rule: the last change happened earlier than "now minus the timeout". A session dated 58520 never matches that rule. It stays open forever, and everything attached to it stays too:

• the chat and its members;
• operators' unread counters;
• session service records, indexes, response statistics;
• a slot in every operator's recent dialogs list, which the portal walks through each time chats are opened.

Date indexes stop helping once a table holds records "from the future", and history queries read more rows than they need. The same log had 81 NO LAST SESSION errors, where the portal could not find a client's last session. Most likely the broken dates caused those too.

For scale: open lines produced 83% of all messages on this portal, about 4,000 a day, a million and a half a year. Every session that fails to close leaves its rows in the database for good.

Why nobody notices at launch

An integration gets tested like this: send a WhatsApp message, see it arrive in Bitrix24, the operator replies, the client receives the reply. Everything works, the task is closed.

Nobody checks the message date in that test. The interface barely shows it, and the damage builds up in the database for months. By the time the portal slows down, the integration has long been considered done, and the search starts elsewhere: hardware, cache, "Bitrix is just heavy".

Integrations often go this way. Teams check that data arrived. They rarely check what shape it landed in and what happens to it six months later.

What we did

We fixed it in three steps, and the order matters. Then we added a safeguard for the future.

First, stop the flow of broken dates. As long as new messages arrive from the year 58520, any cleanup is pointless: within an hour there are new sessions that will never close. So we fixed the problem where it starts, on the Wazzup side, by configuring it to send the date in the format Bitrix24 expects. The portal code stayed untouched, so a Bitrix24 update cannot roll the fix back.

Checking the result is easy. New CONNECTOR - ENTITY ADD events in imopenlines.log show today's date again, and the count of messages from the future (the first query at the end of this article) stops growing. Once the setting changed, dates started arriving in the right format.

Then close what was already stuck. New messages were fine now, but the old sessions still carried dates in the future. Auto-close would never pick them up, so we wrote a script that closed every open chat.

They have to be closed, not deleted from the database. An open line session is tied to a chat, its members, operators' unread counters and statistics. Delete the session and leave the rest, and you get orphans: a chat with no session sits in an operator's list, the counter never resets, and the portal logs errors like NOT_LOAD_SESSION_OR_CHAT whenever someone touches that dialog. The log already had a couple of those. A proper close frees the chat, removes the operators, resets their counters and records the result in statistics. That is the work auto-close should have been doing all along.

Only after that, clean up the bloated database. Carefully: the open line and message tables are large, and a careless delete on a live portal is expensive. Take a fresh copy of the database before a cleanup like this, because without one there is no way to undo a delete.

Finally, add a watchman. Someone can change the integration setting by accident, or Wazzup can update its side, and the dates will drift again. Finding out six months later because the portal is slow is not an option, so we added a simple check built on Bitrix24 lists. Every incoming message creates a list item, and items are kept for one day. If the year in a message date differs from the current year, a notification goes straight to the Bitrix24 administrator and the head of IT. Now an outage like this gets noticed right away, and there is time to react before the database grows again.

The same log showed a second, non-technical cause of hanging sessions. Staff picked up colleagues' dialogs, but their role lacked the "close other operators' dialogs" permission. Each attempt to close failed with ERROR_USER_NOT_OPERATOR, 60 to 90 times a day, and the dialog stayed open. One checkbox in the line's permissions fixes it.

How to check your portal

If your on-premise portal runs open lines through Wazzup or another messenger connector, these queries show whether you have the problem:

-- messages and sessions dated more than a year ahead
SELECT COUNT(*) FROM b_im_message          WHERE DATE_CREATE > NOW() + INTERVAL 1 YEAR;
SELECT COUNT(*) FROM b_imopenlines_session WHERE DATE_CREATE > NOW() + INTERVAL 1 YEAR;

-- how many sessions are open right now
SELECT COUNT(*) FROM b_imopenlines_session WHERE CLOSED = 'N';

If MySQL is not in strict mode, you may find 0000-00-00 instead of a future date. Look for that too.

If you have thousands of open sessions while operators handle dozens of dialogs, look at the [message][date] field of CONNECTOR - ENTITY ADD events in imopenlines.log. And check in the line settings whether staff can close dialogs they took over from colleagues.