Repository navigation
Let parallel stores to the persistent HTTP cache succeed - #66
FlorianRappl merged 4 commits into
Conversation
Every store rewrote the origin.txt of its origin, so parallel loads from one origin collided on it (51 of 64 parallel stores failed in a repro), and stores of one key could collide on the swap. The IOException reached the page load. origin.txt is only written when missing, as its content follows from the folder name. A store that still collides with another one is dropped and its temporary file deleted: the other entry is as good, and like a failed read, a failed store only means a cache miss later.
| @@ -1,3 +1,7 @@ | |||
| # 1.3.2 | |||
|
|
|||
| - Fixed parallel stores to the persistent HTTP cache failing the page load with an `IOException` @michaelthyregod | |||
FlorianRappl
left a comment
There was a problem hiding this comment.
There are too many and imho unnecessary changes.
If we worry about parallelism here we should put in a origin-specific queue (Task) which puts one change/write after another. The read should always work from memory. If one write is outstanding then no new write on the same origin is queued.
|
Picking up the file-system part here. I looked at where Io writes to disk: the persistent stores from All three follow the same shape: a handler with plain read/write, and a provider above it deciding when to write. So a general solution could be a small internal helper that ensures at most one write per key (origin or file) at a time, used by each provider, so handlers never have to care about parallel writes. #65 already does this for the cookie file. For #66 that would mean dropping the handler changes and letting Would you like it that way? If so, I'd rework #66 to introduce the helper and use it in |
|
I think that's a great suggestion. Let's do it like that! |
# Conflicts: # CHANGELOG.md
HttpCache now queues its handler writes per origin (WriteQueue), so handlers never see parallel writes; the changes to PersistenceHttpCacheHandler are undone.
|
Reworked as discussed: the writes now go through a small |
Every store to the persistent HTTP cache rewrote the
origin.txtof its origin, so parallel loads from one origin collided on it: 51 of 64 parallel stores failed in a repro. Stores of one key could also collide on the swap. TheIOExceptionreached the page load.WriteQueueruns the writes of one key one after another, andHttpCacheroutes all its handler writes (stores,RemoveAsync,ClearAsync) through it, keyed by origin. Writes for different origins still run in parallel. There are no locks and no thread waits: a write simply awaits the previous one of its origin.PersistenceHttpCacheHandleris unchanged, and customIHttpCacheHandlers never see two writes for one origin at the same time either.Tests: 64 parallel stores to one origin and 16 to one key all succeed and can be read back (both fail without the queue). For the queue itself: writes of one key run one after another, writes of other keys don't wait, and a failed write reports its error without stopping the next.
WriteQueueis meant to be reused, e.g., for localStorage later, in its own PR.🤖 Generated with Claude Code