Skip to content

docs: explain proxyrunner session-aware backend pod routing when using backendReplicas #708

@yrobla

Description

@yrobla

Problem

guides-k8s/run-mcp-k8s.mdx documents the backendReplicas field and states that Redis is required when replicas > 1, but it does not explain why Redis is needed or how the proxyrunner routes sessions to specific backend pods.

When backendReplicas > 1, the proxyrunner uses a Redis-backed RoutingStorage to pin each session to the specific backend pod that initialized it. Without this explanation, users may:

  • Not understand why Redis is needed just for backend scaling (not proxy scaling)
  • Misconfigure or skip Redis for backendReplicas scenarios
  • Not understand what happens when a backend pod is restarted or replaced

What's needed

Add a short explanation (a paragraph or callout) in the run-mcp-k8s.mdx#horizontal-scaling section that covers:

  1. Why Redis is needed when scaling backends: the proxyrunner must route all requests within a session to the backend pod that initialized that session (stateful MCP connections). Redis stores the session-to-backend-pod mapping so all proxyrunner replicas share this state.
  2. What happens on backend pod restart: the session-to-pod mapping in Redis is invalidated; the next request reconnects to an available pod (sessions are not automatically migrated).
  3. The note about SessionStorageWarning: clarify that the condition only fires for replicas > 1 (proxy scaling), not for backendReplicas > 1 — but Redis is still recommended for backendReplicas > 1 since without it, session routing relies solely on client-IP affinity which is unreliable behind NAT (this is already documented but the implication for backendReplicas should be made explicit).

References

  • guides-k8s/run-mcp-k8s.mdx#horizontal-scaling — section to update
  • Epic: stacklok/stacklok-epics#263 (Horizontal Scaling for Proxyrunner, THV-0047)
  • RFC THV-0047 — describes the RoutingStorage design

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions