Pro backend updates for 0.2.0 backend (PFS edition)#96
Draft
jagerman wants to merge 137 commits into
Draft
Conversation
session-deps is a new repo that consolidates how we handle loading and doing static bundle builds of various common external dependencies across Session projects.
Carrying a libsodium fork is too much of a nuissance as updating it to the latest version is non-trivial. This drops the libsodium-internal fork in favour of using tweenacl's implementation *just* for the X25519 -> Ed25519 pubkey conversion, and using a stock libsodium for everything else. This also bumps the libsodium requirement up to 1.0.21: that version will be required for SHAKE support in future commits on this branch.
- Blake2b hashing:
- Add nicer hash::blake2b functions for simpler has computations where
you can just pass a bunch of spannables and get the hash over them,
rather than needing to do a bunch of manual C API update calls.
- Add a ""_b2b_pers for a compile-time validated blake2b
personalisation string
- Drop make_blake2b32_hasher().
- TODO: convert these to make full use of hash::blake2b(...), as
above.
- Change cleared_array to take a Char type instead of forcing unsigned
char.
- Add cleared_uchars (and cleared_bytes) for the old force-unsigned char
typedef.
- Make `to_span` work for any input convertible to a span
- Tighten up various functions taking fixed length values (like session
ids, pubkeys) to take compile-time-fixed spans instead of dynamic
extent spans. This allows these generic functions to not have to
worry about length checking.
- Add a generic random::fill(s) function that fills some spannable type
s with random bytes.
This switches to the PR branches for session-router and libquic to use session-deps. (This was required under ninja builds, in particular, to get the deduplication handling for gmp and nettle via new session-deps code to deal with that).
- Refactor manual libsodium blake2b hash calls to use simpler hash::blake2b functions instead. - Unify usage of array_uc32/33/64 and uc32/33/64: now we have just uc32/33/64 and cleared_uc32/33/64 (i.e. the "array_" prefix is gone). - Add a unsigned char array literal, which is quite useful for static hash keys.
- Including raw integer bytes would break the hashes on a non-little endian arches; this adds a helper than ensures we are always hashing the little-endian value. - Make the remaining manual hash use blake2b_pers. This didn't get autoconverted before because the `if` around part of the hash, but that if is actually completely unnecessary: if the value is empty, a blake2b hash update does nothing, and so it can just be always included (and when empty, it is still the right thing).
oxenc has buggy "constexpr" overloads that just break if invoked, and they get invoked here with a uint8_t value + unsigned char array. The issue is fixed in oxenc dev, but switching to a byte here works around it for current and older oxenc versions.
The recent commit to unify the types wasn't applied to the test suite.
hash::blake2b (and related) now take integers directly, writing the integer bytes (with byte swapping applied, if necessary), which further simplies the hashing API.
Make way for account keys to be in here as well.
If two devices rotate account keys at approximately the same time, both might end up with inconsistent "active" keys. This commit adds deterministic tie breaking (always prefering the later, with fallback to seed ordering).
Adds a "needs push" concept, along with more tracking fields to let us distinguish between various possible states.
`session::AdjustedClock` now carries an adjustable static offset and when `now()` is called it returns standard system clock timepoints with the adjustment applied. The networking code had a similar adjustment already, although it was per-network-object: this change replaces that, and makes it now global across all uses of the clock anywhere, instead of per-Network instance.
Integrates the type-safe Pro feature flags refactor, the session_protocol.cpp internal-helper dedup, and the C proof_status delegation. All three verified behaviour- and wire-preserving. Conflicts arose only because pro-flags-cleanup forked before this branch's Pro wire refactor (revocation_tag rename, ms->seconds, version removal). Resolved by keeping this branch's wire semantics while taking the cleanup's structure: - parse_pro_message()/proof_from_c() retargeted to revocation_tag + seconds (as_sys_seconds) instead of the cleanup's gen_index_hash + ms/from_epoch_ms. - proof_status delegation uses the seconds `ts` param (as_sys_seconds(ts)). - decode_for_community C entry keeps as_sys_seconds(ts) + adopts optional_uc32_from_ptr. - UserProfile keeps the sys_seconds set_pro_access_expiry signature + gains the cleanup's private set_profile_feature(ProProfileFlags) helper. Full suite green (175 test cases). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Delete the three dead [[maybe_unused]] dump_pro_* stderr helpers (several with malformed format specifiers, e.g. "%" PRId64 "zu"), and convert the remaining fprintf(stderr, "ERROR: ...") diagnostics to Catch2 INFO/UNSCOPED_INFO so they surface through the test framework instead of raw stderr. Per-error messages use UNSCOPED_INFO so they survive to the post-loop assertion. Drops the now-unused <cinttypes> include. No test logic change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the incremental `nlohmann::json j; j["k"] = v; ...` pattern in the four
*_body helpers with a single braced json literal returned directly
(`return nlohmann::json{...}.dump();`, no named temporary). Cosmetic only:
nlohmann sorts object keys on dump, so the serialized bodies are byte-identical
(the C-vs-C++ request cross-check tests confirm), and the C _request_to_json
wrappers call the same helpers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
epoch_seconds() returns int64_t, but the pfs request-hash helpers and proof_hash_internal stored it in uint64_t before folding it into the BLAKE2b digest. Use int64_t throughout, matching the signed-seconds convention (and the stable branch, which already does). Byte-identical: C++20 mandates two's complement and int64_t has no padding bits, so the little-endian serialization (oxenc reads/byteswaps the object representation) is the same bit pattern either way — signatures are unchanged. Purely a signedness/consistency fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per pro-wire-protocol.md §1 (Q13), the get-details payment-item `status` and the account `user_status` travel as stable string codes, not integers — as `payment_provider`/`plan` already do. libsession still parsed both as integers mapped to hand-synced C enums, which (a) was incompatible with the live backend (payment status has been a string on the wire) and (b) hard-failed the entire get-details parse on any unknown value. Parse both as opaque pass-through std::string (C: char[64] + _count), so unknown or future codes degrade gracefully instead of breaking old clients. Delete the SESSION_PRO_BACKEND_PAYMENT_STATUS and SESSION_PRO_BACKEND_USER_PRO_STATUS integer enums and their "must match backend" coupling. Known codes: payment status unredeemed/redeemed/expired/revoked; user_status never/active/expired. Backend shipped the matching change (session-pro-backend b3db4e3); in lockstep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the all-zero PUBKEY placeholder with the Session Pro Backend's real Ed25519 public signing key (current test deployment, expected to carry through to production). Consumers verify a proof was issued by the backend by checking its signature against this key (ProProof::verify_signature). Trivially editable if the backend signing key ever changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add session::pro_backend::URL (= https://pro.session.codes) so clients read the canonical production base URL from libsession rather than hand-carrying it, and export the Ed25519 signing pubkey to the C API (SESSION_PRO_BACKEND_PUBKEY) for C consumers (iOS); C++ already has pro_backend::PUBKEY. Both C symbols point at the single C++ definitions (no duplicated literal/key). Clients may still override the URL for dev/test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…an audit hash::detail::make_hashable must serialize integer args as fixed-width little-endian (the byte encoding under every Session Pro signed digest). Add an [endian] known-answer test asserting an integer hashes identically to its explicit little-endian bytes — cheap always-on coverage of the LE path. make_hashable's big-endian byte-swap branch is if-constexpr-discarded on LE hosts (never compiled/tested — where a past bug hid). utils/test-bigendian.sh builds + runs the suite under an emulated s390x (qemu-user) so that branch, and other serialization, run on a real big-endian host. Manual/occasional, not CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drop the privileged tonistiigi/binfmt helper container in favour of a one-time host install of qemu-user-static + binfmt-support (registers the binfmt_misc handlers with the F flag, which is what makes them work inside containers). Guard on the handler being present. Also bump the base image bookworm -> sid so the audit runs against a modern toolchain representative of what the code targets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The *_unix_ts members/params are sys_seconds / sys_ms time points, so the name baked the representation into the identifier. Rename them to *_at (as the backend did) to stop embedding the type. Pure rename: JSON wire keys and protobuf accessors are untouched, so there is no wire change. Also drop the stale expiry_unix_ts_ms grace-period doc formula (the wire is integer seconds now; no *_ms field survives). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Requests now carry {endpoint, content_type, data} instead of {endpoint, body}.
Clients relay `data` verbatim under `content_type` and must NOT assume or parse
a format -- libsession owns the wire encoding and may change it. Renames the C
mirror to match: session_pro_backend_to_json -> session_pro_backend_request
(json field -> data, plus endpoint + content_type), *_request_to_json ->
*_request_build, to_json_free -> request_free. The content type is single-sourced
("application/json") next to the endpoint strings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The C request API reimplemented the wire pipeline: a per-request _build_sigs that signed, an input struct the caller hand-filled with pubkeys + precomputed sigs, and a _request_build that re-serialized the body from them. Replace all of it with loose-arg builders that take the same inputs as the C++ *_request() functions (private keys + params) and just call them -- one wire implementation, living in C++. session_pro_backend_request now owns the built C++ ProRequest in an opaque void* and points endpoint/content_type/data into it (zero copy); request_free deletes it. Deletes the _build_sigs/_build_sig functions, the per-request input structs, the add_pro_payment_user_transaction struct, and the master_rotating_signatures/signature return structs. Tests and the dev-server integration block are rewritten to the single-call API. Also tightens the internal *_hash/*_body helpers to fixed-extent spans (span<...,32> for pubkeys, span<...,64> for sigs) so sizes are enforced at compile time rather than silently truncated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Desktop consumes the backend's X25519 pubkey as a constant, so expose it alongside PUBKEY: session::pro_backend::PUBKEY_X25519 (C++) and the C export SESSION_PRO_BACKEND_PUBKEY_X25519. A unit test asserts the hardcoded bytes equal the runtime crypto_sign_ed25519_pk_to_curve25519(PUBKEY) conversion so they cannot drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ded tags Iterated the occasional big-endian audit into a working state (validated on real s390x): use the host's qemu-user-static + binfmt-support (no privileged tonistiigi/binfmt container); debian:sid for a modern toolchain; a clean build dir each run; -DENABLE_NETWORKING_SROUTER=OFF plus libngtcp2/gnutls/ libevent from apt to satisfy the mandatory oxen::quic without building it from source; and fail loudly if any filter tag matches no tests (guards against a tag rename silently shrinking the audit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Response::status was a uint32_t whose domain changed by endpoint (0/1/2 for
most, plus 100/101 for add-payment) -- and for every response except add-payment
it just duplicated !errors.empty(). Remove it: Response is now { errors } with a
success() helper (C: response_header drops status; success == errors_count == 0),
and add-payment collapses to the same -- the backend already sends the human
reason (including already-redeemed / unknown-payment) in errors, so display is
unaffected. Deletes the AddProPaymentResponseStatus C++ enum, the C
SESSION_PRO_BACKEND_ADD_PRO_PAYMENT_RESPONSE_STATUS enum, and its magic-int
values. Parsers still read the wire "status" int internally to decide success;
they just no longer surface it. (A future machine-readable discriminator, if
needed, should be an opaque string slug on the wire like the Q13 statuses.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r (Delta session-foundation#12) Reworks the response envelope to spec §5 / Delta session-foundation#12. The wire `status` becomes a string ok/fail/error parsed into a CLOSED ResponseStatus enum -- an unrecognized value is a protocol error, never passed through (backend confirmed status is exhaustive, Q14). The `errors` array becomes a single `error` string (diagnostic/fallback, first-reason-only). A new open/additive `error_code` slug (opaque; unknown values pass through) carries the machine-readable outcome. Response is renamed ResponseBase (it's only ever a base) and is now { ResponseStatus status; optional<string> error_code; optional<string> error; } with an explicit operator bool(). The C response_header mirrors it (status enum + error_code/error string8, {NULL,0} == none). get-details account status key renamed status -> user_status (Q15). Deletes the obsolete integer status enum. A libsession-side parse failure yields a protocol error (error_code "invalid_response"). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… drop BLAKE2b Implements pro-wire-protocol.md Delta session-foundation#13 / §1.1: all five Ed25519 signatures (the offline proof §2 + the four signed requests §3.1–3.4) now sign the message bytes directly instead of a BLAKE2b-256 digest. - Add session::pro::signed_message() (src/pro_message.hpp): builds the signed message per §1.1 — 16-byte domain prefix, fixed-width raw keys/tags (self-delimiting), integers as canonical decimal ASCII (std::to_chars), strings verbatim, a single NUL between adjacent variable-length fields. - Replace ProProof::hash() with ProProof::signed_message() and delete the C session_protocol_pro_proof_hash() (the 32-byte digest no longer exists in the protocol). - Drop the now-dead hash.hpp includes; BLAKE2b is gone from the Pro signing path. - Retype/rename the 16-byte domain-prefix constants (session_protocol.hpp) from _b2b_pers spans to std::string_view, *_PERS -> *_DOMAIN; drop the C session_protocol_pro_proof_hash declaration. - Update tests to sign via signed_message() and to compare the proof's signature (not a hash) across the encode/decode round-trip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Client-facing enumeration of the user-visible purchasable payment platforms (provider-code slugs: google_play, app_store) so clients build their "purchase via …" store list at runtime instead of hardcoding two stores. Hidden mechanisms (rangeproof) are handled but never listed — they surface only for users who already hold them. C++ `visible_platforms()` returns std::span<const std::string_view>; C `session_pro_backend_visible_platforms(size_t*)` returns a static pointer-array derived from it. Both built from the existing SESSION_PRO_BACKEND_PAYMENT_PROVIDER_CODE_* constants (single source). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jagerman
marked this pull request as draft
July 21, 2026 19:44
…tion#14) Parse the closed `plan` grammar (pro-wire-protocol.md §1 / Delta session-foundation#14) in libsession rather than handing clients the raw wire code to reinterpret. get-details items now carry a parsed ProPlanPeriod{count, unit} (unit ∈ second/day/week/month/year/lifetime) instead of a raw string. - Add parse_plan_period() + ProPlanPeriod/ProPlanUnit (C++), mirrored by SESSION_PRO_BACKEND_PLAN_UNIT + plan_count/plan_unit on the C item. - The unit is preserved, never canonicalized ("12m" != "1y" as values). "lifetime" -> {0, lifetime}; invariant count == 0 iff unit == lifetime. - Closed grammar (a backend<->libsession contract, like the §5 status enum): an unrecognized code is a protocol error, not passed through raw. - Reserves a testing-only `s` (seconds) unit; single-unit only ("1y6m" invalid). Response-parse only -- no wire or signing change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Use the idiomatic `typedef struct X { ... } X;` instead of the split
`typedef struct X X; struct X { ... };`. None of these structs are self-
or mutually-referential, so the forward typedef bought nothing but extra
lines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The provider-code slugs were defined in the C header (static const char[]) and referenced from C++ — backwards for a library whose C++ side is the primary implementation. Move them to C++ constexpr std::string_view (session::pro_backend::PAYMENT_PROVIDER_*, the single source) and have the C `SESSION_PRO_BACKEND_PAYMENT_PROVIDER_CODE_*` symbols reference them as `extern const char* const`, matching how URL / PUBKEY / the endpoints already work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rework the C get-details/proof/revocations/refund response structs so each one owns the parsed C++ response object (via an opaque `void* internal_`) and its fields point into it, instead of deep-copying into fixed buffers backed by a hand-rolled bump arena. This mirrors how the request side already owns its C++ ProRequest. - Response string fields (`error_code`/`error`, item `status`/ `payment_provider`/`payment_id`, details `status`) become `const char*` views (NUL-terminated; NULL when absent); the per-item revocation `tag` becomes a `const unsigned char*` view. All the `char[N]`+`_count` pairs are gone -- including the 128-byte `payment_id` buffer that silently truncated long Google composites. - `header.internal_arena_buf_` -> `void* internal_`; each `*_response_free` is a one-liner over a `c_free_response<Owned>` template constrained to `std::derived_from<ResponseBase>` (the two item-bearing holders derive from their C++ response type). File-local `to_c()` converters (taking a non-const ref, so a temporary can't bind) replace the manual per-field fills. - Delete `arena_t` / `arena_alloc` / `arena_alloc_to_string8` (types.h, types.cpp) -- now used nowhere. - Reorder pro_backend.h: request struct + `request_free` above the response group (no longer wedged between the header and the responses); each `*_free` decl sits immediately after its struct. C-API-only change (response side); the C++ API is unchanged. iOS/nodejs glue must read the new `const char*`/pointer fields instead of sized buffers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cleans up C-isms the original Pro code sprayed through the header/impl:
- Collapse the split `typedef struct X X; struct X {...};` into the
idiomatic `typedef struct X {...} X;`.
- Move each `*_free` declaration to sit immediately after the struct it
frees (it's effectively that struct's destructor).
- Move the protocol limit/padding constants to C++ as the source of truth
(`session::` inline constexpr) with the C `SESSION_PROTOCOL_*` symbols
referencing them as `extern const int`; drop the misleading `PRO_` from
the two non-Pro ones (STANDARD_CHARACTER_LIMIT,
STANDARD_PINNED_CONVERSATION_LIMIT).
- Replace the reinvented `array_uc32_from_ptr_result` with
`std::optional` (maybe_uc32_from_ptr).
- Replace the `sizeof((T*)0)->member` null-deref idiom with
`sizeof(T::member)`.
- Rename the meaningless `utf`/`utf_size` params to `text`/`text_size`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…der constants) - Rename ProviderUrls -> ProviderURLs (initialism casing). - All five URL fields are now std::optional<std::string_view>: a provider may populate none of them (e.g. a one-shot crypto provider), so clients test the optional instead of an empty-string sentinel. The C struct's matching fields may now individually be NULL. - Replace the per-call constructed value + std::optional<ProviderURLs> return with fixed per-provider `constexpr ProviderURLs` constants and a `const ProviderURLs*` return (nullptr = no URLs at all). URLs are one-line `""sv` literals (guaranteed null-terminated) so clang-format won't reflow them mid-string. - Use designated initialisers for the provider constants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Now that every ProviderURLs field is optional, the by-value C return could no longer distinguish an unknown provider code from a recognised provider that simply has no URLs -- both produced an all-NULL struct. Add a `bool found` member (true iff the C++ provider_urls() returned non-null) so C clients get the same distinction the C++ `const ProviderURLs*` gives, without a returned pointer they'd have to reason about freeing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`string8` (`{char* data; size_t size}`) was a structural duplicate of the
existing `span_u8` (`{unsigned char* data; size_t size}`) -- same ptr+len byte
span, right down to a duplicate pair of alloc/copy_or_throw helpers. Once its
uses are mapped, none actually need a string-with-length:
- SESSION_PROTOCOL_STRINGS (24 fields) and session_protocol_pro_features_for_msg
`error` are static, null-terminated strings -> `const char*`. The `error`
string_view only ever views static literals ("..." + simdutf::error_to_string,
which returns string_view over literals), so there is nothing to free and no
dangling. SESSION_PROTOCOL_STRINGS drops the `string8_literal` macro for plain
string literals.
- session_pro_backend_request `data` is an opaque (possibly binary) payload that
genuinely needs ptr+len -> `span_u8`, the type already used for sig/msg.
Then delete the dead string8_alloc_or_throw/string8_copy_or_throw helpers (no
callers; duplicates of span_u8_*), and the `string8` struct + `string8_literal`
macro. Test helper string8_equals -> span_u8_equals, now a std::string_view
compare instead of a hand-rolled memcmp.
C-API change (clients): session_protocol_strings fields and
session_protocol_pro_features_for_msg.error are now `const char*`;
session_pro_backend_request.data is now `span_u8`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…details (Delta session-foundation#15) The old combined get-details endpoint conflated a cheap entitlement check with a heavy payment ledger; no client used the ledger beyond the latest item. Split it into two endpoints (both a signed-request flip, new domains): - get_pro_status (domain ProGetProStatus_; signed `master_pkey ‖ ts`) -- the hot "am I Pro?" path. Response: user_status, auto_renewing, expiry_ts, refund_requested_ts, grace_period_duration, error_report, and a single `latest_payment` (or none). No list, no pagination. C++ ProStatusResponse / parse_pro_status; C session_pro_backend_get_pro_status_response with a bool has_latest_payment + embedded latest_payment. - get_payment_details (domain ProGetPayDetails; signed `master_pkey ‖ ts ‖ \0 ‖ limit ‖ \0 ‖ before`) -- paginated history. Request {limit, before}; response {payments_total, items, next_cursor}, no user_status. Keyset pagination: `before`/`next_cursor` is an opaque cursor (std::optional / const char* NULL at end-of-data), echoed verbatim, never parsed. Payment-item parsing is factored into a shared parse_payment_item helper used by both `latest_payment` and `items`. No "pro details" terminology survives (identifiers, endpoints, domains, the error-report enum -> GET_PRO_STATUS_*). C-API change (clients): get_pro_details_{request_build,response,response_parse, response_free} and SESSION_PRO_BACKEND_GET_PRO_DETAILS_ENDPOINT are replaced by get_pro_status_* and get_payment_details_* equivalents. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is the parallel implementation of #95, but rebased (and in some cases rewritten) on top of PR #90.
(Needed for the PFS future branch code to be updated with the changes for session-foundation/session-pro-backend#2)