fix: allow snapshots reset on upgraded datadirs by restricting table config#22281
Conversation
|
ReadOnly: i think we don’t need it. Non-ReadOnly allow mdbx to auto-recover from “safe_no_sync power-off” state. i think we must not use ReadOnly anywhere in Erigon tooling. Accede: we need because we don’t know all that cli flags to create db (geometry and pageSize). WithConfig - nice trick. But if we don’t use ReadOnly - then maybe we can run_migrations on db to create tables. But it’s big question - which tool is lower level “migrations” or “seg reset” (what depends on what). Also 1 long-standing issue: Capli using separated DB but same schema with ChainDB - i thin we need split Caplin schema in future. |
|
Also “read chaindata vs pass —chain flag” is very old chiken-egg problem - i think we must have good tooling for this problem. |
|
Drop legacy Table from schema - also old problem which we didn’t invest time into. It-theory it must be doable - in practice likely various erigon versions will return error - if some table will not exist (or maybe it will be not real problem if we will remove ReadOnly mode usage). Also Acceede allows us to start RPCD with Erigon in-parallel and don’t scare of race-risk (but it’a alao ephomeral risk - mecause mdbx support cross-process guarantee of 1-rwtx-at-a-time a means no race, no conflicts) |
… in snapshots reset
Head branch was pushed to by a user without write access
|
Thanks so much for the detailed review @AskAlexSharov. So ReadOnly - yes you are correct, I have dropped .Readonly(true) now. MDBX can do its auto-recovery correctly without it, if the DB wasn't shutdown cleanly (like after power cut or crash). And this is also in line with how dbCfg in snapshots_cmd.go opens the DB - just Accede(true) without Readonly. For Accede - agreed, we need it because we simply don't have the geometry, page size and map size configuration available here to build the DB from scratch. One interesting thing I noticed going through Regarding the named config - done, I extracted it to ChaindataSchemaForGenesis as you suggested and updated the tests to use the same as well. For the migrations question I think And yes, To open the DB we need the schema but to run migrations we need the chain name first. For legacy tables - actually the infrastructure is already in place. In For RPCD and Accede - yes, mostly it is about avoiding geometry mismatches. Race conditions are not really an issue with MDBX since it has native handling of the cross-process 1-RW-tx guarantee. My feeling is that we could generalise |
|
will need cherry-pick this PR to |
Issue
Basically, when upgrading Erigon (for example, from v3.4.1 to v3.5.0), we introduce new tables to the schema. The getChainNameFromChainData helper is called by the
snapshots resetcommand to automatically read the chain name from the database. It opens the database withAccede(true)andReadonly(true)options.But since it was using the default
ChainDBconfiguration, it tried to validate the existence of all ~138 tables registered in the schema. Because migrations have not run yet on the upgraded datadir, newly added tables (likeBuilderPendingPaymentsorAccountHistoryKeys) are physically missing. In Accede mode, MDBX cannot create missing tables, so the database open fails and the reset command crashes with a misleading error about the chain flag not being set.Fix
To fix this, we need to decouple this simple metadata lookup from the rest of the database schema. The helper only needs to read the genesis canonical hash and the chain config JSON.
We have modified the database open call in getChainNameFromChainData to use
WithTableCfg. This allows us to pass a minimal table configuration containing only the HeaderCanonical and ConfigTable tables. Now, it only checks for these two metadata tables and safely ignores any other execution-layer tables.Closes: #22275