Dequeue transferred records into a chosen database #366

Overview

This task (milestone M2) makes the dequeue path run against a database that the caller names. Session contexts also get a database handle for the local database.

Background & Motivation

Dequeue merges transferred buffers into the store. The disk export runs this same dequeue inside the archive, so that the archive holds the same data as a network server. Today the dequeue code uses the default connection, the module-level backend, and the host instance counter.

Design: spec. Plan: implementation plan, Task 5.

Description & Expected Outcomes

The private dequeue function and the two legacy self-referential order functions take a required database handle. All their queries, raw SQL, transactions, and the instance counter use that handle.

Each session context has a db attribute for the local database. The default is the routed default handle. The attribute does not go into the pickled state, and it returns as the default after unpickling. The receiver dequeue operation passes the context database to the dequeue function.

Deliverables & Contracts

The task delivers:

• _dequeue_into_store(transfer_session, fsic, v2_format, self_ref_order, db), with db required.
• A required db parameter on _update_legacy_self_ref_order and _update_legacy_self_ref_order_for_model.
• SessionContext.db, with the routed default handle as its default value.

Acceptance Criteria

• Dequeue with a handle pinned to default2 writes Store rows in default2 and makes no query to default. The test runs with override_settings(DATABASE_ROUTERS=[]).
• That dequeue increments the instance counter in default2, and the host instance counter stays the same.
• A pickled and unpickled LocalSessionContext has the routed default handle.
• Existing dequeue tests pass after they pass the default handle to the new parameter.
• The characterization suite passes without edits.

Technical Pointers & Architecture

• Target Components: morango/sync/operations.py:728 and :747 (legacy self-referential order; :748 reads transfer_session.sync_session.profile, which must become a lookup by sync_session_id on the handle), :762 (_dequeue_into_store), :1102 (ReceiverDequeueOperation), morango/sync/context.py (SessionContext).
• Related Patterns: SyncDatabase from Explicit database handle for sync database access #365.
• Data Model & Schema Considerations: None.
• Resilience & Failure Modes: Dequeue keeps its transaction and partition locks, now on the named connection.

Notes & Tradeoffs

• Dependencies: Blocked by: Explicit database handle for sync database access #365 (Explicit database handle for sync database access). Blocks: Receiver helpers work against a chosen database #367.
• Testing Requirements: The test app's TestingRouter sends instance saves and foreign-key reads to default, so tests with a pinned handle clear the routers (Spike: prove that a Django-managed archive database works #362). This is a refactor. The characterization suite from Tests that pin today's async network push and pull behavior #363 and Tests that pin today's legacy network sync behavior #364 must pass without edits. DequeueBufferIntoStoreTestCase calls the private function directly, so its calls get the new argument.
• Tradeoffs & Alternatives Considered: An optional instance argument, with None for the host instance, was rejected. It hides a second job in one parameter. The database handle gives the instance instead.

Metadata

• Complexity: Medium
• Target Branch: release-v0.9.x

AI Usage

Drafted with Claude (Claude Code) from the approved design spec and implementation plan. The author reviewed the requirements, and the code references were checked against the release-v0.9.x codebase.