Rendering updates

AGENTS.md

@@ -1,6 +1,6 @@
 # AGENTS.md
 
-AI agent context for `github.com/pb33f/libopenapi` — a Go library for parsing, indexing, mutating, bundling, diffing, overlaying, rendering, and executing OpenAPI/OAS-adjacent documents. Optimize for code-first maintenance: trust implementation and tests over README or published docs when they drift.
+`github.com/pb33f/libopenapi` is a Go library for parsing, indexing, mutating, bundling, diffing, overlaying, rendering, and mocking OpenAPI/OAS-adjacent documents. It is the engine behind vacuum, wiretap, openapi-changes, printing press, and the pb33f platform. When code, tests, and external docs disagree, code is canonical.
 
 ## Context
 
@@ -29,25 +29,25 @@ This repo is a library, not an app. The root package exposes the public entry po
 
 | Path | Purpose |
 |---|---|
-| [`document.go`](/Users/dashanle/work/libopenapi/libopenapi/document.go) | Root orchestration layer; keep it thin |
-| [`index/doc.go`](/Users/dashanle/work/libopenapi/libopenapi/index/doc.go) | Best summary of `index` subsystem boundaries and invariants |
-| [`index/index_model.go`](/Users/dashanle/work/libopenapi/libopenapi/index/index_model.go) | `SpecIndex`, config, caches, release lifecycle |
-| [`index/spec_index_build.go`](/Users/dashanle/work/libopenapi/libopenapi/index/spec_index_build.go) | Index construction/build sequencing |
-| [`index/rolodex.go`](/Users/dashanle/work/libopenapi/libopenapi/index/rolodex.go) | Cross-document lookup ownership and lifecycle |
-| [`index/extract_refs.go`](/Users/dashanle/work/libopenapi/libopenapi/index/extract_refs.go) | Reference discovery entry point |
-| [`index/find_component_entry.go`](/Users/dashanle/work/libopenapi/libopenapi/index/find_component_entry.go) | Component lookup entry path |
-| [`index/search_index.go`](/Users/dashanle/work/libopenapi/libopenapi/index/search_index.go) | Reference search flow, cache usage, schema-id lookup |
-| [`index/resolver_entry.go`](/Users/dashanle/work/libopenapi/libopenapi/index/resolver_entry.go) | Circular detection and destructive resolution entry point |
-| [`datamodel/document_config.go`](/Users/dashanle/work/libopenapi/libopenapi/datamodel/document_config.go) | Canonical config surface for documents/index/bundler behavior |
-| [`datamodel/spec_info.go`](/Users/dashanle/work/libopenapi/libopenapi/datamodel/spec_info.go) | Spec parsing, version detection, JSON conversion, `$self` handling |
-| [`datamodel/low/v3/create_document.go`](/Users/dashanle/work/libopenapi/libopenapi/datamodel/low/v3/create_document.go) | V3 document/index/rolodex assembly |
-| [`datamodel/low/model_builder.go`](/Users/dashanle/work/libopenapi/libopenapi/datamodel/low/model_builder.go) | Reflection-driven low-model population |
-| [`datamodel/high/node_builder.go`](/Users/dashanle/work/libopenapi/libopenapi/datamodel/high/node_builder.go) | High-model re-rendering/mutation path |
-| [`bundler/bundler.go`](/Users/dashanle/work/libopenapi/libopenapi/bundler/bundler.go) | Public bundling entry points/config |
-| [`bundler/bundler_composer.go`](/Users/dashanle/work/libopenapi/libopenapi/bundler/bundler_composer.go) | Composed bundling and component lifting |
-| [`what-changed/model/document.go`](/Users/dashanle/work/libopenapi/libopenapi/what-changed/model/document.go) | Unified change model and compare flow |
-| [`what-changed/model/breaking_rules.go`](/Users/dashanle/work/libopenapi/libopenapi/what-changed/model/breaking_rules.go) | Default/custom breaking-change policy |
-| [`.github/workflows/build.yaml`](/Users/dashanle/work/libopenapi/libopenapi/.github/workflows/build.yaml) | CI shape: Linux + Windows `go test ./...`, coverage upload |
+| `document.go` | Root orchestration layer; keep it thin |
+| `index/doc.go` | Best summary of `index` subsystem boundaries and invariants |
+| `index/index_model.go` | `SpecIndex`, config, caches, release lifecycle |
+| `index/spec_index_build.go` | Index construction/build sequencing |
+| `index/rolodex.go` | Cross-document lookup ownership and lifecycle |
+| `index/extract_refs.go` | Reference discovery entry point |
+| `index/find_component_entry.go` | Component lookup entry path |
+| `index/search_index.go` | Reference search flow, cache usage, schema-id lookup |
+| `index/resolver_entry.go` | Circular detection and destructive resolution entry point |
+| `datamodel/document_config.go` | Canonical config surface for documents/index/bundler behavior |
+| `datamodel/spec_info.go` | Spec parsing, version detection, JSON conversion, `$self` handling |
+| `datamodel/low/v3/create_document.go` | V3 document/index/rolodex assembly |
+| `datamodel/low/model_builder.go` | Reflection-driven low-model population |
+| `datamodel/high/node_builder.go` | High-model re-rendering/mutation path |
+| `bundler/bundler.go` | Public bundling entry points/config |
+| `bundler/bundler_composer.go` | Composed bundling and component lifting |
+| `what-changed/model/document.go` | Unified change model and compare flow |
+| `what-changed/model/breaking_rules.go` | Default/custom breaking-change policy |
+| `.github/workflows/build.yaml` | CI shape: Linux + Windows `go test ./...`, coverage upload |
 
 ## Commands
 
@@ -72,18 +72,53 @@ This repo is a library, not an app. The root package exposes the public entry po
 
 ## Rules & Patterns
 
-- Keep [`document.go`](/Users/dashanle/work/libopenapi/libopenapi/document.go) thin. Parsing/version detection belongs in `datamodel`, indexing/lookup/resolution in `index`, and diff logic in `what-changed`.
+- Keep `document.go` thin. Parsing/version detection belongs in `datamodel`, indexing/lookup/resolution in `index`, and diff logic in `what-changed`.
 - Trust code and tests before README or `pb33f.io` docs.
 - Prefer existing `index` seams over adding more orchestration: `extract_refs*` for discovery, `find_component*`/`search_*` for lookup, `resolver_*` for resolution, `rolodex*` for external docs, `schema_id*` for JSON Schema `$id`.
 - Preserve ownership boundaries: one `SpecIndex` owns one parsed document; `Rolodex` owns shared file/remote lookup and cross-document indexes.
 - Treat lifecycle work carefully. `Document.Release()` intentionally does not release the underlying `SpecIndex`; `SpecIndex.Release()` and `Rolodex.Release()` are separate cleanup steps for long-lived processes.
 - Protect hot paths in `index` and schema resolution. The package explicitly optimizes direct component lookup, caches, pooled nodes, and reduced JSONPath usage on common paths.
 - Add focused regression tests beside the behavior you change. This repo has a strong “surgical tests + high coverage” culture; preserve it.
-- If you touch sibling refs, merge semantics, quick-hash behavior, or schema proxy resolution, run both [`tests/`](/Users/dashanle/work/libopenapi/libopenapi/tests) and the relevant `what-changed` coverage/tests because these behaviors interact.
+- If you touch sibling refs, merge semantics, quick-hash behavior, or schema proxy resolution, run both `tests/` and the relevant `what-changed` coverage/tests because these behaviors interact.
 - High-level models are mutable render facades over low-level YAML-backed models. Rendering or mutation fixes usually need checks in both `datamodel/high/*` and `datamodel/low/*`.
 - `bundler` mutates models and depends on precise rolodex/index semantics. Ref rewrite or composition changes need bundler-specific tests, especially around discriminator mappings and external refs.
 - `what-changed` is intentionally unified across OAS2 and OAS3+. Preserve both default breaking rules and override/config validation behavior.
-- Use realistic fixtures from [`test_specs/`](/Users/dashanle/work/libopenapi/libopenapi/test_specs) and package-local fixture dirs instead of inventing toy specs when reproducing parser/indexer bugs.
+- Use realistic fixtures from `test_specs/` and package-local fixture dirs instead of inventing toy specs when reproducing parser/indexer bugs.
+
+## Common Failure Modes
+
+- **Hash contract**: every schema field must appear in `Schema.hash()` (`datamodel/low/base/schema_hash.go`). A missing field means equality and diff silently ignore it. Call `ClearSchemaQuickHashMap()` between document lifecycles or the global `sync.Map` cache returns stale hashes.
+- **Circular refs**: `resolver_circular.go` detects loops by comparing `FullDefinition` strings. If ref rewriting (bundler, resolver) changes these inconsistently, loops go undetected and the resolver hangs or overflows the depth limit (500).
+- **Reference cache staleness**: `index.cache` (`sync.Map`) is never cleared after bundler mutations. Lookups after bundling can return stale pre-rewrite refs pointing to external files that no longer apply.
+- **Bundler irreversibility**: `BundleDocument` / `BundleDocumentComposed` mutates the model in-place permanently. Never compare, re-bundle, or re-index a document after bundling — parse fresh from the rendered output instead.
+- **Sibling ref idempotency**: `CreateAllOfStructure()` in `datamodel/low/base/sibling_ref_transformer.go` is not idempotent. Running it twice (e.g., bundle then re-index) produces nested `allOf` wrappers that break schema validity.
+- **Resolver state leak**: `IgnorePoly` and `IgnoreArray` flags on the resolver persist between parses. Reusing a resolver across documents causes the second document's polymorphic circular refs to be silently missed.
+
+## Mutation & Rendering
+
+The library uses a dual-model architecture:
+
+- **Low-level models** (`datamodel/low/`): YAML-backed structs that preserve line numbers, column numbers, comments, raw `*yaml.Node` references, and `$ref` metadata. These are the source of truth for document structure.
+- **High-level models** (`datamodel/high/`): Mutable Go structs that wrap a low model. Every high model stores a `low` field and exposes `GoLow()` to access it.
+
+**Mutation flow**:
+
+1. Modify fields on the high-level model (e.g., `doc.Info.Title = "New Title"`)
+2. Call `Render()` or `MarshalYAML()` on the model
+3. `MarshalYAML()` creates a `NodeBuilder(highModel, lowModel)` — the builder uses reflection to read high-model field values and low-model line numbers for ordering
+4. `NodeBuilder.Render()` sorts fields by original line number, then calls `AddYAMLNode()` recursively to build a `*yaml.Node` tree
+5. `yaml.Marshal()` serializes the node tree to bytes
+
+**Key rendering modes**:
+
+- Default (`Resolve = false`): references render as `$ref: ...` strings
+- Inline (`Resolve = true`): references are inlined at point of use
+- `RenderingModeBundle`: inlines refs but preserves `$ref` inside discriminator `oneOf`/`anyOf` for bundling compatibility
+- `RenderingModeValidation`: fully inlines everything for JSON Schema validation
+
+**`RenderAndReload()`** is destructive — it renders to bytes, then re-parses and rebuilds the entire document model from scratch. The old model is invalid after this call.
+
+**`renderer/` is separate**: the `renderer` package generates mock/example data from schemas (for documentation and testing). It does not serialize models to YAML — that is handled by `NodeBuilder` and `MarshalYAML()`.
 
 ## Environment
 

bundler/bundler.go

@@ -28,6 +28,28 @@ import (
 // ErrInvalidModel is returned when the model is not usable.
 var ErrInvalidModel = errors.New("invalid model")
 
+type invalidModelBuildError struct {
+	cause error
+}
+
+func (e *invalidModelBuildError) Error() string {
+	if e == nil || e.cause == nil {
+		return ErrInvalidModel.Error()
+	}
+	return e.cause.Error()
+}
+
+func (e *invalidModelBuildError) Unwrap() error {
+	if e == nil {
+		return nil
+	}
+	return e.cause
+}
+
+func (e *invalidModelBuildError) Is(target error) bool {
+	return target == ErrInvalidModel
+}
+
 // buildV3ModelFromBytes is a helper that parses bytes and builds a v3 model.
 // Returns the model and any build errors. The model may be non-nil even when err is non-nil
 // (e.g., circular reference warnings), allowing bundling to proceed with warnings.
@@ -39,7 +61,7 @@ func buildV3ModelFromBytes(bytes []byte, configuration *datamodel.DocumentConfig
 
 	v3Doc, buildErr := doc.BuildV3Model()
 	if v3Doc == nil {
-		return nil, errors.Join(ErrInvalidModel, buildErr)
+		return nil, &invalidModelBuildError{cause: buildErr}
 	}
 	// Return both model and error - caller decides how to handle warnings/errors
 	return &v3Doc.Model, buildErr
@@ -76,7 +98,7 @@ func BundleBytesComposed(bytes []byte, configuration *datamodel.DocumentConfigur
 
 	v3Doc, err := doc.BuildV3Model()
 	if err != nil {
-		return nil, errors.Join(ErrInvalidModel, err)
+		return nil, &invalidModelBuildError{cause: err}
 	}
 
 	bundledBytes, e := compose(&v3Doc.Model, compositionConfig)
@@ -93,7 +115,7 @@ func BundleBytesComposedWithOrigins(bytes []byte, configuration *datamodel.Docum
 
 	v3Doc, err := doc.BuildV3Model()
 	if err != nil {
-		return nil, errors.Join(ErrInvalidModel, err)
+		return nil, &invalidModelBuildError{cause: err}
 	}
 
 	result, e := composeWithOrigins(&v3Doc.Model, compositionConfig)

bundler/bundler_composer.go

@@ -155,6 +155,13 @@ func isOpenAPIRootKey(key string) bool {
 	return openAPIRootKeys[key]
 }
 
+func rootSupportsPathItemComponents(rootIdx *index.SpecIndex) bool {
+	if rootIdx == nil || rootIdx.GetConfig() == nil || rootIdx.GetConfig().SpecInfo == nil {
+		return true
+	}
+	return rootIdx.GetConfig().SpecInfo.VersionNumeric >= 3.1
+}
+
 // processReference will extract a reference from the current index, and transform it into a first class
 // top-level component in the root OpenAPI document.
 func processReference(model *v3.Document, pr *processRef, cf *handleIndexConfig) error {
@@ -163,6 +170,7 @@ func processReference(model *v3.Document, pr *processRef, cf *handleIndexConfig)
 	var err error
 
 	delim := cf.compositionConfig.Delimiter
+	supportsPathItemComponents := rootSupportsPathItemComponents(cf.rootIdx)
 
 	if model.Components != nil {
 		components = model.Components
@@ -205,7 +213,11 @@ func processReference(model *v3.Document, pr *processRef, cf *handleIndexConfig)
 			case v3low.CallbacksLabel:
 				location = handleFileImport(pr, v3low.CallbacksLabel, delim, components.Callbacks)
 			case v3low.PathItemsLabel:
-				location = handleFileImport(pr, v3low.PathItemsLabel, delim, components.PathItems)
+				if supportsPathItemComponents {
+					location = handleFileImport(pr, v3low.PathItemsLabel, delim, components.PathItems)
+				} else {
+					cf.inlineRequired = append(cf.inlineRequired, pr)
+				}
 			}
 		} else {
 			// the only choice we can make here to be accurate is to inline instead of recompose.
@@ -275,10 +287,12 @@ func processReference(model *v3.Document, pr *processRef, cf *handleIndexConfig)
 					}
 
 				case v3low.PathItemsLabel:
-					if len(location) > 2 && components.PathItems != nil {
+					if supportsPathItemComponents && len(location) > 2 && components.PathItems != nil {
 						return checkReferenceAndCapture(location[2], cf.compositionConfig.Delimiter,
 							v3low.PathItemsLabel, pr, idx, components.PathItems, buildPathItem, cf.origins)
 					}
+					cf.inlineRequired = append(cf.inlineRequired, pr)
+					return nil
 				}
 			}
 		} else {
@@ -356,11 +370,13 @@ func processReference(model *v3.Document, pr *processRef, cf *handleIndexConfig)
 							return checkReferenceAndCapture(pr.name, delim, v3low.CallbacksLabel, pr, idx, components.Callbacks, buildCallback, cf.origins)
 						}
 					case v3low.PathItemsLabel:
-						if components.PathItems != nil {
+						if supportsPathItemComponents && components.PathItems != nil {
 							pr.name = checkForCollision(componentName, delim, pr, components.PathItems)
 							pr.location = []string{v3low.ComponentsLabel, v3low.PathItemsLabel, pr.name}
 							return checkReferenceAndCapture(pr.name, delim, v3low.PathItemsLabel, pr, idx, components.PathItems, buildPathItem, cf.origins)
 						}
+						cf.inlineRequired = append(cf.inlineRequired, pr)
+						return nil
 					}
 				}
 			}