Generalize indexing schema versions

[jwils]

Why

The indexing pipeline spoke in json_schema_version. That key is specific to one ingestion format. A proto ingestion format has no JSON schemas, so it cannot supply that value.

This PR makes the shared pipeline speak in a format-neutral schema_version. Each ingestion format maps its own versioning concept onto that key. The key is also optional, so a format with no versions at all can omit it.

What

• Add SCHEMA_VERSION_KEY = "schema_version" and use it in elasticgraph-indexer, elasticgraph-warehouse_lambda, and the shared test support.
• JSONIngestion::IngestionAdapter restores the json_schema_version key before it validates an event against the JSON schemas, so the artifacts stay unchanged. That patch is private to the JSON gem, so elasticgraph-schema_artifacts gains no new public API and keeps no knowledge of the generic key.
• JSONIngestion::IndexingEventDecoder maps the publisher's json_schema_version onto schema_version while it decodes JSON Lines.
• JSONIngestion::IngestionAdapter dispatches on the presence of a schema version and selects the closest available JSON schema version.

The schema version is optional

schema_version is optional at every stage of the pipeline. An ingestion format with no versions, such as the planned proto runtime, omits it. Each adapter decides what a missing version means for its own format.

The JSON adapter uses the latest available JSON schema version. It still validates the event against that version, so a malformed event still fails, and it fails with a JSON schema message rather than a missing-version message.

The core indexer no longer requires the key either:

• Processor reads the version for its latency log instead of fetching it.
• WarehouseDumper groups by an optional version, and writes the fixed S3 key segment unversioned in place of v<version>. The segment count stays the same, so a reader that splits the key keeps working.

Backward compatibility

Publishers keep sending json_schema_version. The JSON schema artifacts keep the json_schema_version const. This PR adds no breaking change:

• JSONIngestion::IngestionAdapter claims and reads the legacy json_schema_version envelope key. A publisher, or an in-process caller of Indexer#processor.process, needs no edit.
• Indexer::TestSupport::Converters.upsert_event_for accepts __schema_version, the legacy __json_schema_version, or neither. A project generated before this change keeps working with no edit to its shared_factories.rb.
• The ElasticGraphIndexingLatencies log and the DumpedToWarehouseFile log emit schema_version and also the deprecated alias json_schema_version. An existing dashboard or monitor keeps working.
• The ElasticGraphMissingJSONSchemaVersion log keeps its JSON-specific field names requested_json_schema_version and selected_json_schema_version, which match its JSON-specific message type.

Review fixes

Self review of the stack found five defects, all fixed here:

• elasticgraph-indexer/README.md and Indexer::IndexingEventDecoder::Interface promised that a missing version selects the latest available version. An earlier revision of this PR had removed that behaviour, which made both texts wrong. The behaviour is back, and both texts are now format neutral: the core indexer states that the key is optional, and the JSON gem documents what it does with a missing version.
• This PR had added a second "Indexing Event Decoder" section to elasticgraph-json_ingestion/README.md. It duplicated the section from Add configurable indexing event decoder #1220 and named a class that does not exist, ElasticGraph::JSONIngestion::IndexingEventDecoder::JSONLines. Removed.
• JSONIngestion::IndexerExtension#ingestion_adapters memoized into @ingestion_adapters while it called super, which assigned the same name. It now uses @json_ingestion_adapters, so a second format gem can copy the pattern safely.
• select_schema_version sorted the available versions on every event. The adapter now sorts once and memoizes.

Known limitation

handles_event? cannot tell one format from another. The JSON adapter claims an event that carries a schema version under either key. An event with no version reaches the sole available adapter through the existing fallback in Operation::Factory, which covers every single-format deployment.

Two live formats need a real discriminator, because a version-less proto event and a version-less JSON event look the same. I plan to have each decoder stamp its format on the decoded event, and have each adapter dispatch on that stamp. That belongs in the PR that adds the proto runtime.

Verification

• elasticgraph-indexer, elasticgraph-json_ingestion, elasticgraph-warehouse_lambda, elasticgraph-indexer_lambda, elasticgraph-local and elasticgraph-lambda_support unit suites: 556 examples, 0 failures
• script/run_gem_specs elasticgraph-json_ingestion: 282 examples, 100% line and branch coverage
• script/run_gem_specs elasticgraph-warehouse_lambda: 23 examples, 100% line and branch coverage
• script/lint, script/spellcheck, script/type_check: all green
• bundle exec rake schema_artifacts:check: artifacts up to date

Stack

Current PR is marked with ->.

• #1301 Extract an ingestion adapter seam inside elasticgraph-indexer
• #1302 Move JSON ingestion into elasticgraph-json_ingestion via an indexer extension
• #1220 Add configurable indexing event decoder
• #1351 Extract indexing field metadata behind a format-neutral value object
• -> #1284 Generalize indexing schema versions

[jwils]

Pushed d3ff605, which changes the design in one substantive way and fixes five defects. I rewrote the PR description to match. Summary of the delta for anyone who already reviewed:

schema_version is now optional. The earlier revision required it and failed an event that omitted it. That forced every ingestion format to have an integer version, which protobuf does not: it achieves compatibility through field numbers, which #1304 already pins in a sidecar artifact. The key is now optional at every stage, and each adapter decides what a missing version means. The JSON adapter uses the latest available version and still validates the event against it, so a malformed event still fails early enough to be useful.

The breaking changes are gone. The earlier revision broke three things that I had not called out:

• An existing generated project's shared_factories.rb defines __json_schema_version, so Converters.upsert_event_for raised KeyError.
• An event carrying only json_schema_version failed validation, which affects in-process callers of Indexer#processor.process, not just the SQS path.
• Three operator-facing log fields were renamed, which breaks dashboards silently.

All three now accept the legacy name, and the logs emit both the new key and the deprecated alias.

Defects fixed. The README and the decoder interface both promised a default to the latest version that the earlier revision had removed; the behaviour is back and both texts are now format neutral. This PR had also added a duplicate decoder section to the JSON gem README that named a class which does not exist. The indexer extension memoized into a name its own super call assigns. Version selection re-sorted the available versions on every event.

One limitation stays open. handles_event? cannot tell one format from another, so two live formats still need a real discriminator. I describe the plan in the description; it belongs in the proto runtime PR rather than here.