install deletes foreign tables placed between the managed MCP markers in ~/.codex/config.toml (0.10.4 → 0.11.0 upgrade)

[daceconomy]

Version

• Installed by install.sh: codebase-memory-mcp 0.11.0 (upgrading from 0.10.4)
• Codex CLI 0.154.0 / Codex Desktop present on the same machine

Platform

macOS 15 (Darwin 24.6.0), Intel (darwin-amd64)

Install channel

GitHub release archive via install.sh (bash ~/.local/bin/install.sh, installs to ~/.local/bin)

Binary variant

standard

What happened, and what did you expect?

codebase-memory-mcp install (run by install.sh during the 0.10.4 → 0.11.0 upgrade) rewrote the managed # >>> codebase-memory-mcp MCP >>> … # <<< codebase-memory-mcp MCP <<< region of ~/.codex/config.toml and deleted every table that other software had placed between those two markers: [mcp_servers.node_repl] (+ its .env table), [desktop], and all four [marketplaces.*] tables. ~/.codex/config.toml went from 5,841 bytes to 4,371 bytes. Codex Desktop lost its marketplaces, desktop settings and its bundled node_repl MCP server. Nothing was printed for Codex; the install summary reported mcp: ~/.codex/config.toml as if it had succeeded.

How the foreign tables got inside the managed region: the 0.10.4 install had appended its block at the end of the file, so the closing marker was the last line. Codex Desktop later appended its own tables, and the trailing comment line stayed last (apparently its TOML editor treats a trailing comment as attached to the document end — I did not verify that in Codex itself), which left the closing marker below Codex's tables. From that point on, the region between the markers contained one cbm-owned table and four foreign ones.

Expected: the installer should never remove a table it did not write. cbm_toml_upsert_managed_block should fail closed (as toml_find_markers already does for orphan markers, #1558) when the span between its markers contains any table header other than [mcp_servers.codebase-memory-mcp] — or, better, replace only the owned table and keep the rest of the span verbatim. The user-added key inside the owned table (startup_timeout_sec = 90, needed because the MCP server takes >10 s to start on a 1.5 GB index) was also dropped; preserving unknown keys, or at least printing a note, would be kinder.

Reproduction

The real-world trigger was simply bash ~/.local/bin/install.sh on a machine whose ~/.codex/config.toml had foreign tables between the cbm markers (state shown in the diff below).

Sandboxed steps that should reproduce it without touching a real Codex configuration (HOME, CODEX_HOME, TMPDIR and the cbm cache/runtime dirs all point at a temp directory). Caveat: I could not execute this on the affected machine — with a live cbm daemon running, install stops with activation could not reserve exclusive access; no activation was committed, because the version-cohort locks live in /private/tmp/cbm-daemon-<uid>/ and are account-wide rather than under CBM_CACHE_DIR/CBM_RUNTIME_DIR. It should run on a machine (or CI job) with no daemon. The evidence for the bug is the real diff and the code path, both below.

sandbox="$(mktemp -d)"
mkdir -p "$sandbox/.codex" "$sandbox/bin" "$sandbox/cache" "$sandbox/runtime" "$sandbox/tmp"

cat > "$sandbox/.codex/config.toml" <<'EOF'
model = "gpt-5"

[mcp_servers.codegraph]
command = "codegraph"
args = ["serve", "--mcp"]
# >>> codebase-memory-mcp MCP >>>
[mcp_servers.codebase-memory-mcp]
command = "/tmp/old/codebase-memory-mcp"
args = []
startup_timeout_sec = 90

[mcp_servers.node_repl]
args = []
command = "/Applications/ChatGPT.app/Contents/Resources/cua_node/bin/node_repl"
startup_timeout_sec = 120

[desktop]
followUpQueueMode = "steer"

[marketplaces.openai-codex]
source_type = "git"
source = "https://github.com/openai/codex-plugin-cc.git"
# <<< codebase-memory-mcp MCP <<<
EOF
cp "$sandbox/.codex/config.toml" "$sandbox/before.toml"

HOME="$sandbox" CODEX_HOME="$sandbox/.codex" XDG_CONFIG_HOME="$sandbox/.config" TMPDIR="$sandbox/tmp" \
CBM_CACHE_DIR="$sandbox/cache" CBM_RUNTIME_DIR="$sandbox/runtime" \
  codebase-memory-mcp install -y --dir="$sandbox/bin" --clients=codex

diff "$sandbox/before.toml" "$sandbox/.codex/config.toml"

Expected result (matches what happened for real): [mcp_servers.node_repl], [desktop] and [marketplaces.openai-codex] are gone; only the regenerated [mcp_servers.codebase-memory-mcp] table remains between the markers.

Logs

Diff of the real file, before → after the upgrade (values in node_repl.env elided, $HOME written as ~). Lines 133–178 were replaced by the single env_vars line:

133,178c139
< startup_timeout_sec = 90
<
< [mcp_servers.node_repl]
< args = []
< command = "/Applications/ChatGPT.app/Contents/Resources/cua_node/bin/node_repl"
< startup_timeout_sec = 120
<
< [mcp_servers.node_repl.env]
< NODE_REPL_NATIVE_PIPE_CONNECT_TIMEOUT_MS = "…"
< … (12 more env keys)
<
< [desktop]
< followUpQueueMode = "steer"
< external-agent-import-sync-item-types = "all"
< external-agent-import-sync-enabled = true
< conversationDetailMode = "STEPS_PROSE"
< sansFontSize = 14
< codeFontSize = 13
< ambient-suggestions-enabled = true
<
< [marketplaces.openai-bundled]
< source_type = "local"
< source = "~/.codex/.tmp/bundled-marketplaces/openai-bundled"
<
< [marketplaces.openai-primary-runtime]
< source_type = "local"
< source = "~/.cache/codex-runtimes/codex-primary-runtime/plugins/openai-primary-runtime"
<
< [marketplaces.claude-plugins-official]
< source_type = "git"
< source = "https://github.com/anthropics/claude-plugins-official.git"
<
< [marketplaces.openai-codex]
< source_type = "git"
< source = "https://github.com/openai/codex-plugin-cc.git"
---
> env_vars = ["CBM_CACHE_DIR", "CBM_RUNTIME_DIR"]

Installer output for the Codex section (no warning about the removed tables):

Codex CLI:
  mcp: ~/.codex/config.toml
  instructions: ~/.codex/AGENTS.md (managed activation pointer)
  ...
  hooks: SessionStart + SubagentStart (dynamic graph context)

Where it happens: src/cli/config_toml_edit.c, cbm_toml_upsert_managed_block() copies existing[0 .. begin_line.start), appends the fresh managed block, then copies existing[end_line.full_end ..). Everything between the two marker lines is discarded, and toml_managed_block_conflicts() only inspects the text outside that span, so foreign tables inside it are neither detected nor preserved.

Project scale (if relevant)

Not index related. (For context: the machine's main index is 567 k nodes / 707 k edges, 1.5 GB.)

Related observations from the same upgrade (not the main report)

• ~/.codex/hooks.json: the two hook-augment entries had been edited locally to prefix CBM_HOOK_DEADLINE_MS=20000 (see next bullet). The installer treated them as foreign, preserved them, and appended its own canonical SessionStart and SubagentStart groups, so Codex ended up with each hook twice. Matching on the binary path + hook-augment token, rather than the exact command string, would avoid the duplicate.
• Feature ask: hook-augment has a built-in 2000 ms deadline (only overridable via the CBM_HOOK_DEADLINE_MS env var). On this 1.5 GB index a hook query takes 4–6 s even with a warm daemon, so every hook silently exited empty (127 deadline_exceeded lines in logs/hook-augment-timeouts.log) until the shims were hand-edited — which then makes them "not ours" for the installer. A config set hook_deadline_ms setting honoured by the managed shims would avoid the whole conflict.

Confirmations

• I searched existing issues (Installer adds hooks to both hooks.json and config.toml #570, Codex: reinstall appends duplicate [[hooks.SessionStart]] when hook exists in inline form — "duplicate key" breaks config.toml #1432 and Installer/updater scope and safety: no per-client selector, destructive update step before an interactive prompt, hidden ui_enabled, brittle legacy-marker cleanup #1558 are about hook duplication / orphan markers; none cover foreign tables inside the managed MCP region).
• The sandboxed steps above do not modify a real Codex configuration; the real-file diff was taken from a snapshot made two minutes before the upgrade and the file has since been restored by hand.

[DeusData]

Thank you for the detailed before/after evidence and for separating the confirmed configuration loss from the additional observations. I checked current main (59a05eb): cbm_toml_upsert_managed_block replaces the entire marked span, and its conflict check excludes that same span. That confirms the preservation gap you identified when foreign tables end up inside the markers.

We are treating this as a high-priority installer data-preservation bug and keeping it open. Review needs to cover both preserving unrelated tables and handling extra keys in the owned table, without weakening the existing malformed-marker protections. The hook duplication and deadline-setting suggestions are separate concerns, not prerequisites for addressing the data loss. Please keep your restored configuration and backup; there is no need to repeat the destructive upgrade on your real settings.

Code checked:

codebase-memory-mcp/src/cli/config_toml_edit.c

Lines 1070 to 1134 in 59a05eb

	int cbm_toml_upsert_managed_block(const char *file_path, const char *begin_marker,
	const char *end_marker, const char *block) {
	size_t block_len = 0U;
	if (!toml_valid_path(file_path) || !toml_valid_marker(begin_marker) ||
	!toml_valid_marker(end_marker) || strcmp(begin_marker, end_marker) == 0 || !block ||
	toml_bounded_length(block, TOML_EDIT_MAX_BYTES, &block_len) != TOML_EDIT_OK ||
	!toml_text_is_safe(block, block_len, 1) ||
	toml_contains_marker_line(block, block_len, begin_marker, end_marker) ||
	toml_validate_lexical_strings(block, block_len) != TOML_EDIT_OK) {
	return TOML_EDIT_ERR;
	}
	char *existing = NULL;
	size_t existing_len = 0;
	toml_file_snapshot_t snapshot;
	if (toml_read_file(file_path, &existing, &existing_len, &snapshot) != TOML_EDIT_OK ||
	!toml_text_is_safe(existing, existing_len, 1)) {
	free(existing);
	return TOML_EDIT_ERR;
	}
	toml_line_t begin_line = {0}, end_line = {0};
	int has_pair = 0;
	if (toml_find_markers(existing, existing_len, begin_marker, end_marker, &begin_line, &end_line,
	&has_pair) != TOML_EDIT_OK) {
	free(existing);
	return TOML_EDIT_ERR;
	}
	size_t exclude_start = has_pair ? begin_line.start : SIZE_MAX;
	size_t exclude_end = has_pair ? end_line.full_end : SIZE_MAX;
	if (toml_managed_block_conflicts(existing, existing_len, exclude_start, exclude_end, block,
	block_len) != TOML_EDIT_OK) {
	free(existing);
	return TOML_EDIT_ERR;
	}
	toml_buffer_t output = {0};
	size_t prefix_len = has_pair ? begin_line.start : existing_len;
	if (has_pair && prefix_len == 0U && existing_len >= 3U && (unsigned char)existing[0] == 0xefU &&
	(unsigned char)existing[1] == 0xbbU && (unsigned char)existing[2] == 0xbfU) {
	prefix_len = 3U;
	}
	const char *newline = toml_newline_style(existing, existing_len);
	size_t payload_start = existing_len >= 3U && (unsigned char)existing[0] == 0xefU &&
	(unsigned char)existing[1] == 0xbbU &&
	(unsigned char)existing[2] == 0xbfU
	? 3U
	: 0U;
	if (toml_buffer_append(&output, existing, prefix_len) != TOML_EDIT_OK ||
	(!has_pair && existing_len > payload_start && existing[existing_len - 1] != '\n' &&
	toml_buffer_append_cstr(&output, newline) != TOML_EDIT_OK) ||
	toml_append_managed(&output, begin_marker, end_marker, block, newline) != TOML_EDIT_OK ||
	(has_pair && toml_buffer_append(&output, existing + end_line.full_end,
	existing_len - end_line.full_end) != TOML_EDIT_OK)) {
	toml_buffer_dispose(&output);
	free(existing);
	return TOML_EDIT_ERR;
	}
	int result =
	toml_write_atomic(file_path, existing, existing_len, output.data, output.len, &snapshot);
	toml_buffer_dispose(&output);
	free(existing);
	return result;
	}