docs(openapi): Autofix OpenAPI spec validation errors - #2953
Draft
Pijukatel wants to merge 1 commit into
Draft
Conversation
Error: SOFT_FAIL {"errors":[{"errorCode":"type.openapi.validation","message":"must be array","path":"/response/data/items/<N>/categories"}],"method":"GET","msg":"Response OpenAPI validation error","statusCode":200.0,"url":"/v2/store?limit=1000&offset=<OFFSET>&sortBy=lastUpdate"}
Files: apify-api/openapi/components/schemas/store/StoreListActor.yaml:29
Root cause: `categories` is nullable in stored Actor documents, and `GET /v2/store` returns it
unchanged. The write API explicitly accepts `null` for the field - both `CreateActorRequest.categories`
and `UpdateActorRequest.categories` are already documented as `type: [array, "null"]` - and
`updateActor` persists the modifier with a plain `$set`, so a submitted `null` is stored verbatim
(the `Act2Schema` `type: Array` check does not reject it, because simpl-schema skips type checks for
null values on optional keys). The Algolia consolidation daemon copies the field with
`_.pick(actor, ACTOR_INDEX_FIELDS)`, preserving the `null`, and the store search route returns the
hit unchanged via `toPublicSearchItem`'s spread. Every other consumer normalizes the field
defensively (`actor.categories ?? []` in the Recombee sync, `$ifNull: ['$categories', []]` in the
console search service, whose comment records that the stored data is not clean), but the public API
route does not, so the raw `null` reaches the response and fails the `type: array` assertion.
Widening the response schema to `[array, "null"]` documents the values the endpoint actually returns;
it still rejects any other non-array type, so the assertion is not weakened beyond null.
Reference: https://github.com/apify/apify-core/tree/912cf52953c2b5ef841ebdfbacce20a0bf0b5cfe/src/api/src/routes/store/search.ts#L179
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWepzDer9U43JgyXjACWYv
Contributor
|
✅ Preview for this PR (commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Autogenerated OpenAPI fixes suggestions based on validation errors generated from running API integration tests with OpenAPI validator turned on.
Error log: not applicable for this iteration - no
api.logwas generated. The input was a productionSOFT_FAILlog excerpt (16 lines,Sep 6 11:01-Sep 7 09:36) supplied directly, and the reporter confirmed the errors are not reproducible in the PR test environment. All 16 lines deduplicate to a single unique error.apify-core version: https://github.com/apify/apify-core/tree/912cf52953c2b5ef841ebdfbacce20a0bf0b5cfe
Stop reason: No new fixes are possible - the supplied log contains exactly one unique error, and it is fixed here.
Detailed changes description
Error fixes
categoriescan benullin the store search responseFiles:
apify-api/openapi/components/schemas/store/StoreListActor.yaml:29Error:
All 16 log lines are the same error on
GET /v2/store, differing only in item index, offset and search parameters. They span everysortByvalue, bothresponseFormatvariants and bothincludeUnrunnableActorssettings, which is consistent with a per-Actor data property rather than a query-shape problem.Root cause:
categoriesis nullable in stored Actor documents, andGET /v2/storereturns it unchanged.The write side already documents
nullas an accepted value -CreateActorRequest.categoriesandUpdateActorRequest.categoriesare bothtype: [array, "null"].updateActorpersists the modifier with a plain$set, so a submittednullis stored verbatim: theAct2Schemacategories: { type: Array, optional: true }check does not reject it, because simpl-schema skips type checks fornullon optional keys (verified against the pinnedsimpl-schema@3.2.0:nullvalidates, while a string or number is rejected withexpectedType).From there the
nullpropagates untouched to the response:_.pick(actor, ACTOR_INDEX_FIELDS), which preserves anullvalue.toPublicSearchItem's...itemspread - thefullresponse format applies no normalization tocategories.Every other consumer of the field coalesces it defensively -
actor.categories ?? []in the Recombee sync, and$ifNull: ['$categories', []]in the console search service, whose inline comment records that the stored data is not clean - but the public API route does not. So the rawnullreaches the response and fails thetype: arrayassertion.Widening the response schema to
[array, "null"]documents the values the endpoint actually returns. It does not weaken the assertion beyondnull: verified with ajv 2020 (the engine express-openapi-validator uses) against the bundled schema,nulland string arrays now pass while a bare string and a number are still rejected.Reference:
src/api/src/routes/store/search.ts#L179- response passthroughactors_store_index.server.ts#L356- Algolia record buildactors.both.ts#L571-Act2Schemadefinitionrecombee_store_sync.ts#L530- defensive normalization in the Recombee syncactor_search.service.ts#L91- defensive normalization in the console search serviceRefactoring
None. The fix is confined to the one field named in the error.
Unfixed errors
Potential API bugs
GET /v2/storereturnsnullforcategorieswhere every other surface returns[]Status: fix written in
apify-core, not yet reviewed or deployed.Root cause: The Recombee sync and the console search service both normalize a missing or
nullcategoriesto[]; the public store search route was the only surface passing the raw value through, so clients had to handle two representations of "no categories".The source-level fix now exists on the
apify-corebranchclaude/openapi-store-validation-errors-3ginx5(commitfe01e5e8). It normalizes in all three response shapers -toPublicSearchItemplus both agent-format shapers, since those emitcategoriestoo and would otherwise keep the bug reachable viaresponseFormat=agent- and widensStoreActor.categories/PublicApiStoreActor.categoriesto includenullso the types match what the backends return.Sequencing: land and deploy the
apify-corechange first, then narrow this schema totype: arrayin a follow-up (it could also becomerequired, since the field is then always present). This PR remains correct in the meantime, because a nullable type still accepts arrays.Reference:
src/api/src/routes/store/search.ts#L179Actor.categorieshas the same latent gapRoot cause:
apify-api/openapi/components/schemas/actors/Actor.yaml:118also declarescategories: type: array, while the same nullable stored value is served by the Actor read endpoints viagetPublicActor, which returns the projected document unchanged. No error for those endpoints appears in the supplied log, so it is left unchanged here per the minimum-fix rule, but it will surface the samemust be arrayerror whenever an Actor with a storednullis fetched. Theapify-corefix above is deliberately scoped to the store search route and does not cover this path.Reference:
actors.both.ts#L571False positives
None encountered. No nullable
date-timefield and no multi-type definition was touched, and the change is a genuine semantic widening rather than a syntax swap:nullpreviously failed validation and now passes.Validation
The errors are not reproducible in the PR test environment, so the regression check was done against the bundled spec instead:
pnpm openapi:lint(Redocly + Spectral + YAML) passes with no errors. The 6 Redocly warnings are pre-existing and all concern single-schemaallOfinactor-buildspaths, untouched here.openapi.jsonbefore and after shows exactly 4 leaf changes, all insideStoreListActor.properties.categories(thetypewidening plus the addeddescription). Nothing else in the spec changed.StoreListActoris referenced only byListOfStoreActors→ListOfActorsInStoreResponse→ theGET /v2/store200 response, so no other endpoint's validation is affected.typeinto a superset of the original, it cannot make a previously passing response fail - a response-type widening can only remove validation errors, never add them.Issues
Partially implements: #2286
🤖 Generated with Claude Code
https://claude.ai/code/session_01TWepzDer9U43JgyXjACWYv