Skip to content

fix(lsp): queued writes and committed workspace status - #65

Closed
Sunrisepeak wants to merge 4 commits into
mainfrom
fix/17-connection-writer-encoding
Closed

Sunrisepeak wants to merge 4 commits into
mainfrom
fix/17-connection-writer-encoding

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Completion messages can serialize large replies on the event loop, producer/scanner differences can rediscover an unchanged source repeatedly, and a Zed file worktree can be adopted as clangd's working directory. This PR moves connection encoding to its writer, publishes status after synchronous plan application commits, confirms valid producer source structures conservatively, and filters existing regular files from workspace roots.

It also preserves completion/state-transition timing evidence, corrects the genuine Windows fixture's std dependency, and retains first CI failures. The Termux check uses the official package source; the Zed smoke requires its independently started PATH clangd and mcppls's own initialized child. No performance budgets are relaxed.

Validation on the frozen local source now pushed in fcb3450: dev/release each pass 6 workspace members / 58 programs and 32 LSP fixtures, including the new 70-check watched-source fixture. Specification validation has zero failures (four existing pending rules). New-source CI, native Windows real-producer acceptance and unresolved performance investigation remain pending; the prior nightly failures are preserved, not superseded by a later pass.

Portable handoff and file map · Checksummed evidence and restore commands · Failure summary. Selected evidence is approximately 1 MB and can be restored without the original machine. Engine memo materials are unimplemented review proposals; the engine and payload lock remain unchanged.

Draft: do not merge or publish 0.0.17. This checkpoint synchronizes the local repairs and handoff materials. Refs #59 and #63; follow-up to #64.

Release preflight hit the unchanged 4 MiB / 2 s send check on macOS dev after PR #64 had merged. Sending still encoded each JSON frame synchronously on the caller, even though pipe writes were queued. Queue an owned JSON snapshot and encode it on the writer outside the queue lock, preserving FIFO, both framings and UTF-8 replacement. Writer exceptions become a connection failure and unsent snapshots are discarded after closing the queue.

Keep the original send/stop gates and record their actual elapsed time. Real echo and departed-peer tests cover snapshot lifetimes, ordering, shutdown drain and error cleanup. Keep the first failure and the limits of its timing evidence; no host-load attribution or OOM injection is claimed. Version 0.0.17 is still unpublished.

Refs #59, #64.
@Sunrisepeak Sunrisepeak changed the title fix(lsp): child messages are encoded on the writer fix(lsp): queued writes and committed workspace status Oct 22, 2026
The target fixture used a simulated producer, which left the design's native Windows i686/x64 production chain unproven. A separate Windows fixture configures both targets with real mcpp and follows their complete unmodified databases in one LSP session, checking actual commands and conditional module semantics. CI retains the first generated databases and logs separately from diagnostic reruns. Product and runner code remain unchanged; native acceptance is still required.
@Sunrisepeak

Sunrisepeak commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner Author

交接同步更正:原评论发布时确实没有 commit/push,这个交接缺口现已补齐。修复与四份引擎提案已推送到 fcb3450,精选证据已随分支提交,可在另一台机器恢复;下面保留的是当时的历史检查点。请以 HANDOFF.md 与证据获取说明 为当前入口。PR 仍是 draft,未合并、未发布。

交接检查点(2026-10-22 10:36 UTC)

本条用于直接通过 PR 交接。当前不再继续修改其他内容;0.0.17 暂缓发布。引擎侧仅记录问题与候选方向,不代表已经实施或获准跨仓发布。 PR 仍为 draft,远端 HEAD 是 e01e1c1ac06fec8f4c4335043e6dc5354efeb7b2;下述新增修复仍在本地工作区,尚未 commit/push,不能将现有 CI 当作它们的验证。不发 issue 评论、不变更 issue 状态。

1. 已确认的问题与本地修复

  • 重复 producer discovery:项目问题。 合法 #if 0 import 被 producer 正确排除,旧实现仍反复比较 lexical imports 与实际 graph,body-only/相同字节保存会重复 emit。旧产品 1341938 的原 28 项 fixture 有 11 次 emit(预期 6)、14 个失败检查,包含计数级联,非 14 个独立缺陷。
    本地修复在 src/orchestrator/workspace.cpp 的 capture_source_structures / confirm_source_structures / watch 分支:仅确认有效成功回答所覆盖的 run-start 结构、同代数与 producer 身份,并验证文件 stamp 未变;失败、迟到变化、A→B→A、pending reload 均不能误确认。共享 32 MiB 有界读取,失败不重复消耗额度。src/project/scan.cpp/.cppm 的预处理文本仅 opt-in 采集,避免 native index 为全项目额外长期保存文本;宏形式/未完整解析 import、header unit 与未知声明保守重查,不声称实现完整预处理器。扩展 fixture 为 70 项,含真实变化的 producer requires、条件开关、失败重试、in-flight、target/provider/manifest 正对照。不能据此断言历史 U15 13/min 已归因。
  • Zed 文件被当 workspace directory:项目兼容问题。 原始 initialize 中 workspaceFolders 指向存在的 greet.cppm,rootUri 却是正确目录;服务端优先采用文件,继而把它作为 clangd cwd,触发 engine-start-failed。本地 src/server/session.cpp/.cppm 的 workspace_folders 忽略已有普通文件,沿原 rootUri/rootPath/cwd 回退,不猜测 parent;有效及尚不存在的目录保持原语义。新增 tests/test_session.cpp。
    原 smoke 的任意 diagnostics 可由模块层错误满足,不能证明 clangd 成功启动;本地 smoke 新增实际目录初始化检查、mcppls 自身 clangd 的初始化握手和存活子进程检查,复用 lsp_frames.py,不冒充全部 BMI 已准备完成。
  • 真实 Windows fixture 预检查:fixture 问题。 新 fixture 没使用 std,实际 mcpp 不输出既有 validator 要求的 mcpp:std。已补真实 import std / 类型断言,Linux 上实际 pinned producer 的 schema 对照通过,validator 未放宽。真实 Windows i686→x64→i686 LSP 仍需原生 CI 通过。 已加 schema 阶段 producer envelope 的失败 artifact 保留。
  • Termux / Zed 下载:基础设施问题。 Termux 随机镜像 proot-distro_5.6.0_all.deb 返回 404,尚未进入产品验证;本地改官方主源与严格 apt update/install,保留原 bind/LSP 检查。Zed 默认模式 GitHub API 403,240 秒内无自己的 clangd;本地改为预装 PATH clangd,仍让 Zed 以默认设置亲自启动独立进程,保留原门槛,不再声称测试自动下载。这些修改还需实际新 CI。

2. 不得被后续 PASS 覆盖的性能失败

1341938 的 nightly 39742450508 最终 58 success / 4 failure / 4 skipped:

失败 原始观测与边界
mcpp round 2 U5 p95 2.374 s / max 8.264 s,U6 max 6.384 s;超原 1/3/3 秒门槛。server completion max 8.262 s 与客户端一致,不能只归咎 runner 接收延迟;根因未定。
mcpp round 1 U15 edits 13/min >12。逐点重算跨度 59,999,840,473 ns,严格 <60s 确实失败;支持点均在 U6 autosave,而非 U8。
xlings two-core U5 timeout 1;U7a p95/max 1.53 s >1.5 s。best effort 不等于通过。
xlings round 1 原 600 s idle CPU 0.006108 core;RSS 948.086→1188.250 MiB,+25.33% >10%。idle 前 126.419 s 有限工作单列;idle 无记录到的 engine activity,不等于无内部内存分配。

后续 e01e1c1:PR CI 39752365858 为 45 success / 1 failure / 1 skipped;nightly 39752360539 为 59 success / 3 failure / 4 skipped,失败分别是 Windows 预检查、Termux 404、Zed 默认模式。其余通过不能解释或抹去前述失败。

3. 可能的问题位置,包括引擎

慢请求尚未归因。 后续优先检查 src/server/session.cpp 主循环/计时、src/orchestrator/workspace.cpp 的同步 apply/replan、completion 处理与 handle_timers 中的保存工作。已检查并非无限 drain event 后才处理 timers;但引擎 timers 先于部分 deadline 判断,同步工作仍可能推迟预算处理。engineRepliedAt 前也有 normalization/client-view/cache 等处理,因此 engineMs 不能直接等同纯 clangd 时间。以上是排查位置,不是已证明根因。新 runner src/bin/conformance.cpp 保留原回答人口的 completionTrace、全部请求中的最慢支持点,以及已有 before/after report 的 serverCompletionEvidence;recent 样本和 journal 有截断/覆盖边界,不能拿晚期 p95 反推此前慢请求。

引擎 memo 的独立无效分配已证明,但不能解释全部 RSS 增长。 当前是 23.1.0-mcppls.4 / fork 92a4c630f4691890e6133c87d3e4591bc5fd8258。发布补丁 0030 的 ModuleMemoStore.cpp::Store::write() 先 snapshot/构造全部 JSON,后比较 WrittenVerdicts/WrittenScans,所以无变化的每 30 秒周期也有完整构造与销毁。独立 memo-only 对照(无打开文件、background index 关闭)180 秒 RSS 176.832→266.754 MiB、零 memo 写入,证明“无文件写入”不是“无分配”,不证明持续泄漏或本次 +25.3% 全由它造成。
候选方向是两个 cache 同锁内的廉价 changeCount 检查,放到周期 JSON 构造之前;仅成功持久化后推进 Written,失败仍重试;最终 flush 保留强制 snapshot,不能把 join Store worker 当成所有生产者已停止。测试应直接计 JSON 构造,旧实现必须失败,并补 race/写失败/退出对照。只有本地提案,未改引擎、未编译/运行 canary、未发布或更新 payload lock。 后续若继续排查 RSS,应保留原 idle 起点/预算并获取逐 PID/allocator 证据,不能关 persistence 来制造通过。

新一次诊断揭示另一项真实 U7a 失败。 用旧产品 + 新 trace runner 单次原四阶段(CPU、producer、cache a/a/a/b、输入和预算不变)得到 25 checks / 1 failure,168 个观测 PID/birth 身份全部退出。本次 U5/U6 max 0.861/0.707 秒,未复现历史慢请求;唯一失败是 importer diagnostics 107.247 s >33 s。
完整日志显示 src/config.cppm 在 U7a setup、保存之前发生 clangd Build AST crash;其余 7 个 importer 在保存后 0.822–5.605 秒重发,而 config 被隔离 120.216 秒,handback 后 AST 仅约 0.730 秒。对应位置:src/engine/clangd/guard.cppm/.cpp 的 FIRST_TERM=2min / put/due,以及 src/engine/clangd.cpp 的 set_aside_、sources_changed;现有 source-change 释放处理 moduleFailed 隔离,不能据相关模块重建就假定原 crash 条件消失。不要缩短冷却或无条件复活来掩盖 33 秒失败。 可审查“相关依赖成功产生不同有效闭包后一次有界重试”的方向,但尚未证明安全,不能当已成立修复。
崩溃上下文涉及 nlohmann JSON 模板实例化、GCC16 stl_uninitialized.h 和 realloc 栈。实际 clangd SHA 与 manifest/incident 一致;ELF stripped,无调试符号,四个地址符号化均为 ??。记录无明确 signal 编号/故障地址,不能仅凭 exitCode -1 称 SIGSEGV,也不能归咎上游、系统或硬件;底层原因 UNASSIGNED。历史 dev runner/cold 异常也保留未归因边界,不能因旧版本也异常就排除本 PR。

4. 当前验证、证据与接手顺序

  • 新增 focused 回归 dev/release 均通过:scan 23 cases /146 assertions、Workspace 29/399、Session 3/12。Session 首次测试的两处 string/JSON 比较编译错误已修为 .get<std::string>(),首个日志仍保留。
  • 最新本地源码的 dev/release 完整 workspace 均为 6 members /58 programs /0 failed;两种 profile 的完整 build 也通过,匹配 server/runner/mock 已冻结并记录 SHA。dev 原31组 +新增70项 fixture 共32组全部通过;release 对应32组在评论准备时仍运行,不能提前计通过。
  • 本地主要入口:.payload-cache/59-63-real-target-integration/state.json、.payload-cache/59-63-current-state.json;源码/测试/冻结二进制证据在 .payload-cache/59-63-source-watch-and-zed/(source-after-session-test-fix.json、binaries-{dev,release}.json、workspace-tests.json、fixtures-{profile}/summary.json)。当前 release fixture 会话 31682,fixture-profiles-run.log 可核实进度;不要因观察超时重启。
  • 性能原证据在 .payload-cache/59-63-real-target-integration/first134-nightly-*;新诊断在 .payload-cache/59-63-performance/mcpp-134-nightly-round2-latency-diagnostic/:terminal-analysis.json、u7a-postrun-analysis.{json,md}、完整 stack/parser context 与 u7a-symbolization-summary.json。原始大日志/缓存不上传 PR。简报 / 复现入口 的本地版本已有补充,但尚未推送;本地 engine-memory-proposal 四份材料也仅供 review。
  • 接手先读取当前工作区 diff 与以上实际终态,收齐正在执行的 release fixture;审阅并选择性提交本地修复,再跑对应源版本全部 CI/nightly、真实 Windows 生产链及新产品原完整性能验收。严格性能测量期间不并发构建/重型扫描,不改预算、fixture、CPU 或缓存谱系,不以重跑取一次 PASS 代替定位。
  • 本次仅交接,不合并、不发布。 0.0.17 尚未发布,原两个 0.0.16 升级 profile 保持原状;不提前安装候选、不重置旧升级证据。用户的 .zcodeignore 和 .agents/docs/2026-10.12-implementation-status-tmp 不要纳入提交或清理。清理仅限本项目过期、无引用的可再生临时产物,保留日志/报告/冻结二进制与当前验证环境。

…ndaries

A successful producer answer acknowledges the source structure it actually covered, so inactive imports no longer trigger discovery on every body save. Run-start stamps, generations and producer identities keep failures and concurrent changes conservative. Existing files named as workspace folders fall back to directory roots before clangd starts.

Preserve completion timing support, correct the genuine Windows fixture std dependency, and retain first CI evidence. Termux uses the official package source and the Zed smoke requires both its independent PATH clangd and mcppls own initialized child. Ship a portable, checksummed handoff bundle instead of references to one machine.

Validation: dev/release each pass six workspace members and 58 programs, plus 32 LSP fixtures including the new 70-check source-watch fixture. Specification validation reports zero failures. The 222-file evidence archive was verified and extracted independently. New-source CI, native Windows production-chain acceptance and unresolved performance investigation remain pending; no merge or release.
@Sunrisepeak

Copy link
Copy Markdown
Owner Author

上次仅写交接说明、没有同步本地改动,导致其他机器找不到,这是交接缺口。现在已实际补齐,并更正了原评论顶部。

  • 源码与测试已 commit/push:fcb3450,PR fix(lsp): queued writes and committed workspace status #65 的 head 已确认是 fcb3450e7960d093e93cfb0aca9ae2ad7f007d70。
  • 重复 discovery 修复和 70 项 fixture、Zed workspace 修复及 test_session.cpp、真实 Windows std 依赖、Termux/Zed CI 修改、四份引擎提案都已在分支中。
  • 不再依赖原机器:HANDOFF.md 是完整交接入口。精简证据包与恢复工具 随 Git 提交,约 1 MB、222 份文件,包含首次性能失败、U7a stack/parser context/符号化、两 profile 验证和源码快照,逐项 SHA-256 可核对。不是完整 PCM/构建缓存环境。
  • 已从 GitHub API 重新下载 20 个关键文件并逐字节核对,再仅使用远端下载的 archive/manifest/verifier 在新目录成功恢复全部 222 份证据,不是只检查原工作区。
  • 当前本地验证:dev/release 各 6 members /58 programs /0 failed;各 32 组 LSP /0 failed,其中新 fixture 各 70 checks /0 failed。这些是已冻结源码的本地结果;新 commit 的 CI 不能提前计通过。历史性能失败和真实 Windows 验收限制保留。

在另一台机器已有仓库中,可不改现有 release 分支,另建 worktree:

git fetch https://github.com/Sunrisepeak/mcpp-language-server.git fix/17-connection-writer-encoding
git worktree add --detach ../mcppls-pr65-handoff FETCH_HEAD
cd ../mcppls-pr65-handoff
python3 .agents/docs/2026-10-22-issues-59-63-evidence/portable-handoff/verify.py --extract --into .

预期输出 verifiedFiles: 222、extracted: true。历史 state 文件可能有旧的 live/pending 标签,以 HANDOFF 与实际终态 JSON 为准。只同步交接;PR 保持 draft,没有合并、发布 0.0.17 或修改引擎仓库。

@Sunrisepeak

Copy link
Copy Markdown
Owner Author

Continued in #67 (0.0.17 release), which carries this branch's four commits unchanged; fcb3450 and its evidence bundle stay reachable from both PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant