Skip to content

node: handshake with a transit router the catalog does not list - #667

Merged
aojea merged 2 commits into
google:mainfrom
aojea:node-transit-unlisted
Oct 11, 2026
Merged

aojea merged 2 commits into
google:mainfrom
aojea:node-transit-unlisted

Conversation

@aojea

@aojea aojea commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to the deploy failure at 14:35 today (dd17037a, no ladder load): the cold-path probe called the JS agent through its two circuit addresses and both relays answered PERMISSION_DENIED (202).

selectTransit recognized a router only through the node's catalog, read from /info at start and every fifteen minutes. A circuit through a router the catalog did not list was treated as a relay that is not a router and dialled unauthenticated, and the relay refused it. A member that starts during a router rollout reads a catalog without the routers that are restarting (their lease is withdrawn for 30 s), which is what the probe is. Before #652 every member handshook with every router at start, which hid this.

For a relay the catalog does not list the node now re-reads the routers (at most once in ten seconds) and handshakes with a relay still unlisted after that, after the listed routers, taking the relay's own address from the circuit address. A relay that is not a router admits on its own terms and its circuit is kept as before. Every transit handshake is counted in agentmesh_node_transit_handshakes_total{router="listed"|"unlisted",result} and an unlisted one is logged, so the next such failure reads from the metrics.

Tests: TestSelectTransitHandshakesWithARouterTheCatalogDoesNotList (two fake routers, node holds one, no control plane to re-read; fails on main with 0 handshakes), TestCallerTransitsARouterItsCatalogMissed (integration: router b and its provider arrive after the caller). Also refreshRouterCatalog/loadIdentityForAuth return an error instead of dereferencing a nil store, which some unit tests build.

A caller reaches a peer through a circuit at one of the peer's routers,
after handshaking there. It recognized a router by its catalog, read from
/info at start and every fifteen minutes: a circuit through a router the
catalog did not list was treated as a relay that is not a router and
dialled as is, and the relay refused it (PERMISSION_DENIED). A member that
starts during a router rollout reads a catalog without the routers that
are restarting, and the deploy's cold-path probe did on 2026-10-11: both
circuits to the SDK agent refused, nothing in the probe's log to say why.
Until the proactive mesh was removed every member handshook with every
router at start, which hid this.

For a relay the catalog does not list the node now re-reads the routers,
at most once in ten seconds, and handshakes with a relay still unlisted
after that, after the listed routers; the relay's own address comes from
the circuit address. A relay that is not a router admits what it
handshakes with on its own terms and its circuit is kept whatever the
handshake said, as before. Each handshake for transit is counted in
agentmesh_node_transit_handshakes_total{router=listed|unlisted,result},
and an unlisted one is logged.

A node without a store, as some tests build, reports that instead of
dereferencing it.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for transiting routers that are not listed in the control plane's catalog, allowing nodes to handshake with unlisted relays and periodically refresh the catalog. It also introduces a new Prometheus metric agentmesh_node_transit_handshakes_total to track these handshakes, along with corresponding tests and documentation. A critical issue was found in relayTransportAddr where calling AppendComponent on a nil multiaddr.Multiaddr interface will cause a runtime panic; using multiaddr.Join is recommended to safely construct the address.

Comment thread internal/node/mcp.go
Review on google#667. Multiaddr is a slice of components in go-multiaddr v0.16,
so the relay's address is the components before the /p2p/<relay>/p2p-circuit
tail, cloned; TestRelayTransportAddr pins the addressed, bare and direct
cases.
@aojea
aojea merged commit 8be6847 into google:main Oct 11, 2026
21 checks passed
@aojea aojea mentioned this pull request Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant