Rollback safe system table additions - #5679
Open
jsdt wants to merge 5 commits into
Open
Conversation
jsdt
changed the title
gefjon
reviewed
Aug 6, 2026
gefjon
left a comment
gefjon
left a comment
Contributor
There was a problem hiding this comment.
Does this change the creation of a new on-disk database so that the initial snapshot is now at tx_offset 1, rather than zero?
Comment on lines
+236
to
+298
| /// Return the rows which describe the built-in system table schemas. | ||
| /// | ||
| /// These rows are inserted directly into committed state by | ||
| /// `CommittedState::bootstrap_system_tables`. Durable databases also write | ||
| /// them into the commit log before the first ordinary transaction so replay | ||
| /// from offset 0 can learn about system tables which are newer than the replay | ||
| /// binary's built-in catalog. | ||
| pub fn system_table_schema_rows() -> Vec<(TableId, ProductValue)> { | ||
| let schemas = system_tables(); | ||
| let mut rows = Vec::new(); | ||
|
|
||
| for schema in &schemas { | ||
| rows.push(( | ||
| ST_TABLE_ID, | ||
| ProductValue::from(StTableRow { | ||
| table_id: schema.table_id, | ||
| table_name: schema.table_name.clone(), | ||
| table_type: StTableType::System, | ||
| table_access: schema.table_access, | ||
| table_primary_key: schema.primary_key.map(Into::into), | ||
| }), | ||
| )); | ||
| } | ||
|
|
||
| for col in schemas.iter().flat_map(|schema| schema.columns()).cloned() { | ||
| rows.push((ST_COLUMN_ID, ProductValue::from(StColumnRow::from(col)))); | ||
| } | ||
|
|
||
| for constraint in schemas.iter().flat_map(|schema| &schema.constraints) { | ||
| rows.push(( | ||
| ST_CONSTRAINT_ID, | ||
| ProductValue::from(StConstraintRow { | ||
| constraint_id: constraint.constraint_id, | ||
| constraint_name: constraint.constraint_name.clone(), | ||
| table_id: constraint.table_id, | ||
| constraint_data: constraint.data.clone().into(), | ||
| }), | ||
| )); | ||
| } | ||
|
|
||
| for index in schemas.iter().flat_map(|schema| &schema.indexes).cloned() { | ||
| rows.push((ST_INDEX_ID, ProductValue::from(StIndexRow::from(index)))); | ||
| } | ||
|
|
||
| for seq in schemas.iter().flat_map(|schema| &schema.sequences) { | ||
| rows.push(( | ||
| ST_SEQUENCE_ID, | ||
| ProductValue::from(StSequenceRow { | ||
| sequence_id: seq.sequence_id, | ||
| sequence_name: seq.sequence_name.clone(), | ||
| table_id: seq.table_id, | ||
| col_pos: seq.col_pos, | ||
| increment: seq.increment, | ||
| min_value: seq.min_value, | ||
| max_value: seq.max_value, | ||
| start: seq.start, | ||
| allocated: seq.start - 1, | ||
| }), | ||
| )); | ||
| } | ||
|
|
||
| rows | ||
| } |
Contributor
There was a problem hiding this comment.
Is it possible to combine this definition with that of CommittedState::bootstrap_system_tables? I feel at least a little uncomfortable having effectively two definitions of this same function that must be kept in sync.
Comment on lines
+444
to
+447
| let mut rows_by_table = std::collections::BTreeMap::<_, Vec<_>>::new(); | ||
| for (table_id, row) in system_table_schema_rows() { | ||
| rows_by_table.entry(table_id).or_default().push(row); | ||
| } |
Contributor
There was a problem hiding this comment.
As a possible implementation of my request in previous comment, could we read the rows out of the in-memory CommittedState rather than re-computing them by calling system_table_schema_rows?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
TLDR: Write system table schema info to the commit log immediately for new databases.
Currently if we add new system tables, there is a specific case where trying to open a database with a previous version of spacetimedb can fail: if we create a new database with the added system table (and write something to that system table), we won't be able to replay the history for that database without a snapshot. In practice, this is an unlikely error case, since we always try to write a snapshot for a new database. Databases with a snapshot don't have a problem (because the new system table schema info is in the snapshot), and existing databases don't have a problem (because we create the new system tables in a transaction).
With this change, when we open a new database, we write all of the system table schema information to the commit log as the first transaction, so older versions will still be able to parse rows for those tables.
Expected complexity level and risk
Testing
This updates some existing tests to make sure that the schema information is written to the commit log, and it adds a few additional tests in relational_db. The most important one is probably
replay_from_commitlog_preserves_unknown_future_system_table, which simulates a replay with an unknown system table.