cache: an open circuit sheds reads and still repairs the cache

[loks0n]

The problem

Cache\Adapter\CircuitBreaker sheds every operation while its circuit is open — load() and save(). Shedding reads is the point: the dependency is sick, the read would fail anyway, and refusing it early costs nothing.

A write is not the same thing, because a cache write is the repair. Refusing it holds the miss rate at 100% for as long as the circuit stays open, so the traffic the breaker diverted keeps arriving at whatever it was diverted to well after the cache itself is healthy. A cache cannot warm up while it is forbidden to remember anything.

Measured, from the incident this came out of. A 6 s cache stall opened the circuit for its full 30 s timeout, and every request in that window resolved its project document from MySQL and threw the answer away:

	cache misses/s
before	1,295–1,620
+5 min	3,544
+10 min	5,816
+15 min	10,001 — still climbing

898 requests failed with DATABASE_UNAVAILABLE in the worst minute. The breaker did not protect the system: it moved load from an elastic dependency onto an inelastic one, and then blocked the path back.

This is the sharp end of Brooker's argument that a client-side breaker can make a partial outage worse. The fallback contract is "go to the source of truth" — not "go to the source of truth and refuse to remember the answer."

What changes

save(), saveWithLease() and touch() now reach the adapter while the circuit is open, returning their usual fallback if they fail. load(), list(), getSize() and ping() are shed as before.

This is the only behaviour — there is no flag. Two details are what make that safe rather than reckless:

While open, the write bypasses the breaker rather than reporting to it. The verdict is already made, so one more data point cannot change it, and what decides when the circuit closes should be the probes half-open schedules rather than repair traffic arriving at whatever rate the fallback path happens to generate. While the circuit is not open a write reports normally, so a failing cache can still open it — testWritesStillReportToAClosedCircuit pins that.

One failed repair ends the attempts for that open episode. This is the bound that makes unconditional repair safe. A repair is worth one timeout to learn whether the cache accepts writes, and worth nothing after that: against an adapter that is not answering, retrying on every request would add its timeout to every request — precisely the cost an open circuit exists to avoid. The first failure suppresses the rest of the episode, and the guard clears as soon as the circuit is no longer open, so a cache that refused writes is retried next time rather than written off for the life of the process.

So the worst case for a fully dead cache is one extra timeout per open episode, not one per request.

Compatibility

This is a behaviour change, deliberately and by default: an open circuit no longer discards writes. Callers see no API change — the signatures, the fallback values and the read path are all identical, and the existing CircuitBreakerTest passes unmodified.

It pairs with an adapter that fails fast and cheaply, which for the multiplexing adapter is what #152 makes true.

Tests

tests/Cache/Unit/RepairWhileOpenTest.php, 9 tests:

• an open circuit still writes through, asserted by reading it back off the adapter directly
• reads are still shed while open — repairing is not reopening
• touch() counts as a write
• a failing write still returns the fallback rather than raising something the caller did not have to handle
• one failed repair stops the rest of the episode trying, asserted on a call-counting adapter
• a healthy cache keeps being repaired
• the guard clears once the circuit is not open
• writes still report to a closed circuit, and can still open it
• a closed circuit is unaffected

Full package suite: 211 tests, 537 assertions, 4 skipped (pre-existing). bin/monorepo check cache and bin/monorepo validate pass; Vale clean.

Relationship to the other two

Independent — branched from main, and the three touch different concerns:

• cache: stop a caller's read deadline from killing the shared connection #152 stops one slow call failing every other caller on a multiplexed connection (removes the source of a correlated failure batch)
• circuit-breaker: judge health by failure rate, not a consecutive count #153 makes the breaker read such a batch correctly if one arrives, and caps the recovery herd
• this one stops the breaker's own open state from preventing recovery

🤖 Generated with Claude Code

[loks0n]

Force-pushed: repairWhileOpen is gone — repairing while open is now the default and only behaviour.

Making it unconditional changes the risk profile, so it comes with a bound. Without a flag, a fully dead cache would mean every request pays the write timeout on top of the fallback it was already paying — 100 ms per request in the environment this came from — where an open circuit previously made the write free. That is the cost an open circuit exists to avoid, so shipping the flagless version without addressing it would have traded one regression for another.

The bound is one repair attempt per open episode. The first failure suppresses the rest of that episode; the guard clears as soon as the circuit is no longer open, so a cache that refused writes is retried next time rather than written off for the life of the process. Worst case for a dead cache is one extra timeout per episode, not one per request.

Three new tests cover it: one failed repair stops the rest of the episode trying (asserted on a call-counting adapter), a healthy cache keeps being repaired, and the guard clears once the circuit is not open. Also added testWritesStillReportToAClosedCircuit, since bypassing the breaker must not make a failing write invisible to it while closed.

211 tests, 537 assertions green. Title and description updated to describe behaviour rather than an option.