Skip to content

feat(mesh): upgrade to mesh-llm 0.76.0-rc3 and recommend Qwen3.8 27B - #6189

Open
micspiral wants to merge 2 commits into
mainfrom
micspiral/mesh-0.76.0-rc3
Open

feat(mesh): upgrade to mesh-llm 0.76.0-rc3 and recommend Qwen3.8 27B#6189
micspiral wants to merge 2 commits into
mainfrom
micspiral/mesh-0.76.0-rc3

Conversation

@micspiral

@micspiral micspiral commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Upgrades every mesh-llm git dependency (buzz-relay + buzz-desktop) from v0.75.1 to v0.76.0-rc3, and makes Qwen3.8 27B Q4_K_M the curated pick for 64GB-class machines.

Why this rc, specifically, for Qwen3.8

Qwen3.8 could not be recommended on 0.75.1. Two upstream changes in this rc are correctness prerequisites, not polish:

Catalog change

Because the large pick now exists in MODEL_CATALOG, the synthesized entry Buzz carried for the gemma-4-26B pick is deleted along with its _SIZE / _FILE / _DESCRIPTION constants — the entry is canonicalized from the upstream catalog instead. The retired gemma-4-26B alias is kept in canonical_curated_model_id so machines that already selected it keep resolving to a model id Mesh's OpenAI ingress accepts.

The small tier (gemma-4-E4B-it:Q4_K_M) and the 64GB rated-capacity boundary are unchanged.

Dependency resolution

Two transitive bumps were forced, both required to resolve at all:

  • iroh 1.0.2 -> 1.0.3 (the rc requires ^1.0.3).
  • libsqlite3-sys 0.30.1 -> 0.35.0 — the rc's mesh-llm-log-store needs rusqlite 0.37, and only one package in the graph may links = "sqlite3".

Verification

  • cargo check -p buzz-relay — clean.
  • cargo check --features mesh-llm (buzz-desktop) — clean.
  • cargo test --features mesh-llm (buzz-desktop, full package) — 2611 passed.
  • cargo fmt --check — clean.

Two failures predate this change; both reproduce on unmodified origin/main (verified by stashing this diff at f8692fa9b):

  1. managed_agents::global_config::tests::inherited_shared_compute_translates_to_supported_agent_transport — asserts BUZZ_AGENT_MODEL == "auto", gets "mesh".
  2. clippy -D warnings: items after a test module at managed_agents/restore.rs:528.

Scope of evidence

This is a compile-and-test-level upgrade. It is not runtime-verified against a live Qwen3.8 mesh node from this branch. I have separately verified Qwen3.8 27B Q4_K_M serving correctly (real completions, hybrid recurrent, 65 blocks, full_attention_interval 4) on a source build at mesh-llm main, but that was a mesh-llm binary, not this Buzz branch — someone should smoke a share-compute session before release.

Related open micspiral PRs


Includes #6115 (closed, folded in here)

Per Mic, #6115 fix(desktop): stop abandoning healthy mesh prefills at four minutes is closed and its content lands in this PR. Cherry-picked clean as a second commit, one file (managed_agents/relay_mesh.rs).

That fix is a genuine prerequisite for the Qwen3.8 default rather than an unrelated ride-along: buzz-agent's BUZZ_AGENT_LLM_TIMEOUT_SECS defaults to 240 s while MeshLLM's frontend gives a backend call 600 s, and mesh requests are stream: false — so a cold prefill is one silent request. Measured on an M5 Max serving Qwen3.8-27B Q4_K_M: an 88,318-token cold prompt returns HTTP 200 after 503 s. At the 240 s default the client abandons it at four minutes and retries with an escalated budget (240 -> 480 -> 960), piling load onto a box already prefilling the first attempt. Since this PR makes that exact model the 64GB-class default, shipping the upgrade without the fix would make the timeout more likely to bite, not less. Mesh is seated at 660 s; remote providers keep 240 s.

Combined-branch verification at 777426ba8: 2615 desktop tests pass (4 more than the upgrade commit alone — the fix's own tests), cargo fmt --check clean, same single pre-existing global_config failure noted above and nothing new.

Bump every mesh-llm git dependency in buzz-relay and buzz-desktop from
v0.75.1 to v0.76.0-rc3, and make Qwen3.8 27B Q4_K_M the curated pick for
64GB-class machines.

The rc carries the two changes that make Qwen3.8 safe to recommend:
Mesh-LLM/mesh-llm#1283 routes Qwen3.8 identities to the qwen35 recurrent
family (without it the identity compacts to a string matching neither
qwen35 nor qwen36, so a hybrid recurrent model is classified as dense
attention-KV), and #1342 restores recurrent shared prefixes. #1343 adds
Qwen3.8-27B-Q4_K_M to the compiled MODEL_CATALOG.

Because the large pick now exists in MODEL_CATALOG, the synthesized
catalog entry Buzz carried for the gemma-4-26B pick is removed along with
its size/file/description constants; the entry is canonicalized from the
upstream catalog instead. The retired gemma-4-26B alias is kept in
canonical_curated_model_id so machines that already selected it keep
resolving to a model id Mesh's OpenAI ingress accepts.

Dependency resolution notes: iroh moves 1.0.2 -> 1.0.3 to match the rc's
requirement, and libsqlite3-sys moves 0.30.1 -> 0.35.0 because the rc's
mesh-llm-log-store requires rusqlite 0.37 (only one package may link
sqlite3).

Verification: cargo check passes for buzz-relay and for buzz-desktop with
--features mesh-llm; 2611 buzz-desktop tests pass; cargo fmt --check is
clean. Two failures predate this change and reproduce on unmodified
origin/main: the global_config
inherited_shared_compute_translates_to_supported_agent_transport
assertion (expects "auto", gets "mesh") and an items-after-test-module
clippy lint in managed_agents/restore.rs. Not runtime-verified against a
live Qwen3.8 mesh node from this branch.

Co-authored-by: Michael Neale <michael.neale@gmail.com>
Signed-off-by: Michael Neale <michael.neale@gmail.com>
@micspiral
micspiral requested a review from a team as a code owner August 18, 2026 02:26
buzz-agent's `BUZZ_AGENT_LLM_TIMEOUT_SECS` defaults to 240s. MeshLLM's
OpenAI frontend gives a backend call 600s
(`OpenAiFrontendConfig::DEFAULT_BACKEND_TIMEOUT`), so on shared compute the
client aborts work the server is still legitimately doing.

Requests to a local mesh are `stream: false`, so a cold multi-ten-thousand
token prefill is one silent request. Measured on an M5 Max serving
Qwen3.8-27B Q4_K_M: an 88,318-token cold prompt returns HTTP 200 after
503s at 176 tok/s prefill. At the 240s default the client gives up at
four minutes, then retries with an escalated budget (240 -> 480 -> 960),
piling load onto a box that is already prefilling the first attempt.

Seat the mesh client budget at 660s, just above the frontend's 600s, so
the mesh's own error surfaces rather than a client abort racing it.
Remote providers are untouched and keep buzz-agent's 240s default. Like
the other mesh seeds this is a default, not policy: an explicit user
value survives via `insert_default_if_unset` plus the copy-forward list
in `relay_mesh_process_env`.

Co-authored-by: Michael Neale <michael.neale@gmail.com>
Signed-off-by: Michael Neale <michael.neale@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants