BORG_STORE_CACHE: shared cache dir vs. repo-create / repo-delete (separate the store cache per repository?)

[ThomasWaldmann]

Current state

BORG_STORE_CACHE enables a borgstore writethrough cache for the packs/ namespace. The cache directory is shared by all repositories:

• BORG_STORE_CACHE=1 → $BORG_CACHE_DIR/storecache (not $BORG_CACHE_DIR/<repo id>/... like the chunks index / files cache),
• BORG_STORE_CACHE=/some/path → that path, as given.

This is intentional and documented (env var docs and the comment in Repository.__init__): packs are named by content hash, so one directory can hold packs of several repositories. BORG_PACK_CACHE_SIZE is therefore one global budget.

Problem

repo-create and repo-delete are about the primary storage, but with the cache configured they also act on the (shared) cache backend, because Repository hands cache_url to the Store for every command and Store.create() / Store.destroy() include the cache backend.

BORG_STORE_CACHE typically lives in a profile or wrapper script, so it is set for these commands too. Tested with master ec4fe8a + borgstore 0.6.3:

1. borg repo-create fails as soon as the cache directory is not empty (i.e. for the second repository of a user):

   $ export BORG_STORE_CACHE=/path/to/cache
   $ BORG_REPO=/path/repoX borg repo-create -e aes256-ocb    # rc 0
   $ BORG_REPO=/path/repoY borg repo-create -e aes256-ocb
   file:///path/repoY has no repository config.                 # rc 11
   

   Store.create() → cache_backend.create() → posixfs BackendAlreadyExists ("base path is not empty") → borg reports IncompleteRepository for the new repository, which is misleading, and an empty repoY directory is left behind. The same happens when re-creating a repository after deleting its directory manually.

   Running borg's archiver tests with the cache forced on (BORG_STORE_CACHE=1), every test that creates more than one repository fails this way (33 tests).

2. borg repo-delete removes the whole cache directory: Store.destroy() → cache_backend.destroy() → rmtree. That drops the cached packs of all other repositories sharing the directory - and everything else in there: with BORG_STORE_CACHE=/some/dir pointing to a directory that also had an unrelated file in a subdirectory, borg repo-delete deleted that file, too (borg accepts a non-empty directory when opening the cache, only create insists on an empty one).

Options

A. Keep the shared cache, keep it out of create/destroy (borg-side, small).
Do not give cache_url to the Store when creating a repository, and destroy only the primary backend in Repository.destroy() and in the cleanup paths after a failed create. repo-create and repo-delete then do what they are meant to do: create / delete the primary storage. Cached packs of a deleted repository stay until LRU eviction removes them (with a size limit) or the user clears the directory (without one) - would need a sentence in the docs.

B. Separate the store cache by repository ($BORG_CACHE_DIR/<repo id>/storecache or similar), like the rest of the borg cache.

	per-repo directory	shared directory (current)
repo-create with existing cache	works (fresh dir)	fails, rc 11
repo-delete	removes exactly this repo's cached packs	removes all repos' cached packs (and foreign files)
BORG_PACK_CACHE_SIZE	per-repo limit, total disk use = N × limit	one global budget
dedup of cached packs between related repos	lost (only matters for byte-identical packs, e.g. transferred / related repos)	kept

Complication for B: the Store (incl. cache_url) is constructed in Repository.__init__, but the repository id is only known after open() has read config/config through that store. So it needs either attaching the cache backend after the config was loaded (no borgstore API for that yet), or a directory key that is known up front (e.g. a hash of the location, which is not stable if the repo is moved or reached via different URLs). Also needs a rule for an explicitly given BORG_STORE_CACHE=/path (always append the id?).

C. borgstore-side: Store.create() tolerates an existing cache backend, Store.destroy() only invalidates the cached namespaces instead of destroying the cache backend. Changes the Store API semantics for all borgstore users.

A and B are not exclusive: even with B, not passing the cache to repo-create avoids the misleading IncompleteRepository error if a stale cache directory exists.

Questions

• shared global budget or per-repo cache - which one do we want?
• if shared: is option A acceptable (repo-delete leaves cached packs to LRU eviction)?
• should borg refuse / warn about a BORG_STORE_CACHE directory that contains things that are not a borgstore cache?