diff --git a/.claude/skills/debug-bson.md b/.claude/skills/debug-bson.md index f5a7fd5b9..76bb2f7db 100644 --- a/.claude/skills/debug-bson.md +++ b/.claude/skills/debug-bson.md @@ -37,12 +37,14 @@ Note the exact error code (CE0463, CE0642, etc.) and which widget triggers it. ### Step 2: Get a Known-Good Reference -Create a working example in Studio Pro and update it: +Create a working example in Studio Pro and update it. **Do this on a COPY** — both +commands below mutate the project, and `update-widgets` converts an MPRv2 project to +single-file v1 (deletes `mprcontents/`): ```bash -# Convert project to latest format and update widget definitions -reference/mxbuild/modeler/mx convert -p /path/to/app.mpr -reference/mxbuild/modeler/mx update-widgets /path/to/app.mpr +# Convert project to latest format and update widget definitions (mutates in place) +reference/mxbuild/modeler/mx convert -p /path/to/copy.mpr +reference/mxbuild/modeler/mx update-widgets /path/to/copy.mpr ``` Then extract the widget's BSON to compare against your generated output. @@ -323,7 +325,7 @@ for t, props in crash_props.items(): **Investigation methodology used for v0.10 CE0463 fixes** — see [WIDGET_BSON_VERSION_COMPATIBILITY.md](../../docs/03-development/WIDGET_BSON_VERSION_COMPATIBILITY.md) for the full case study and version-resilience model. -**Quick workaround** (if you can't fix the root cause): Run `mx update-widgets` after creating pages. +**Quick workaround** (if you can't fix the root cause): normalize with `mxcli docker check`/`build`, which run the widget update **and preserve MPRv2 storage** (they snapshot/restore `.mpr` + `mprcontents/`). Do **not** run bare `mx update-widgets` on a v2 project you care about — it converts to single-file v1 and deletes `mprcontents/` (corrupts git, breaks `mxcli run --local`). Raw `mx update-widgets` is fine only on a throwaway diagnostic copy or a v1 project. ### CE0642: Property X Is Required @@ -388,7 +390,7 @@ for _, pm := range a.ParameterMappings { 2. **Mode-dependent properties must be consistent**: When changing a mode-switching property (e.g., `showContentAs`), all dependent properties must be updated to match. -3. **`mx update-widgets` is the safety net**: Running this post-processing step normalizes all widget Objects to match mpk definitions. Use it as a fallback. +3. **Widget normalization is the safety net — via `mxcli docker check`/`build`**: they run the update-widgets normalization *and* preserve MPRv2 storage (snapshot/restore). Bare `mx update-widgets` does the same normalization but rewrites a v2 project to v1 and deletes `mprcontents/` — only use it on a v1 project or a throwaway diagnostic copy. 4. **The mpk is the source of truth**: The XML schema defines property types/defaults, the editorConfig.js defines visibility rules. Together they specify the complete expected Object structure. diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 57b7fb52b..ca0cac36b 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -79,6 +79,8 @@ to the symptom table below, so the next similar issue costs fewer reads. | CE0463 on a pluggable widget whose TextTemplate property is *conditionally hidden* by an enum/boolean toggle (VideoPlayer `videoUrl`/`posterUrl` when `type=expression`; Timeline `title`/`description`/`timeIndication` when `customVisualization=true`) — engine clones the template's populated `ClientTemplate` for a property Studio Pro hides and nulls | Engine has no per-property visibility metadata; the hide rules live in the widget's compiled `editorConfig.js` (`hidePropertyIn`/`hidePropertiesIn`), not `widget.xml` | `mdl/executor/widget_defs.go` `widgetVisibilityRules` table + `mdl/backend/mpr/widget_builder.go` `ApplyPropertyVisibility` | Extract the widget's `.mpk` `*.editorConfig.js` (`unzip` + grep `hidePropertiesIn`), transcribe the rule into `widgetVisibilityRules[widgetID]` as `{propertyKey, hiddenWhen:{propertyKey, operator eq/ne/truthy/falsy, value}}`; the engine nulls hidden TextTemplate-typed props at build time. Bump `WidgetDefGeneratorVersion` so stale project `.def.json` auto-refresh. Issue #574 | | CE0463 on a `datagrid` whose column uses `ColumnWidth: manual` + `Size: N` — Studio Pro resets the column `size` to `1` | The MDL `ColumnWidth:` keyword isn't mapped to the schema `width` enum, so `width` stays at its `autoFill` default; `size` only applies when `width=manual`, so the value is inconsistent. Regression from the Stream B keyword-path consolidation (the deleted `datagrid_builder.go` did `colPropString(col.Properties, "ColumnWidth", "autoFill")`) | `mdl/executor/widget_defs.go` `itemPropertyAliases` | Add the MDL→schema alias under `[datagrid]["columns"]`: `"width": {"ColumnWidth"}`. Bump `WidgetDefGeneratorVersion` so stale `.def.json` regenerate. General rule when a column/object-list property's MDL keyword differs from the `.mpk` schema key (not just case), add it to `itemPropertyAliases`; cross-check against the pre-B3 `datagrid_builder.go` `colProp*` calls for any other dropped mappings | | CE0463 on an MDL-created **Combobox** (or other platform widget) at `mxcli docker build/check` time — but the SAME widget passes `mx check` on a project whose installed `.mpk` matches mxcli's embedded template (Mendix 11.6 / combobox 2.5.0). NOT the old incomplete-template bug (#112, fixed — a matching-version combobox is clean) | Widget-VERSION mismatch: mxcli emits the embedded 2.5.0-shaped PropertyTypes, but the project has a NEWER combobox (e.g. 2.8.1) that reorders/regroups properties. `augmentFromMPK` patches presence + enum values but can't restructure the baseline; `GenerateFromMPK` is less faithful still (fails even on a matching version). The designed remediation is `mx update-widgets` (docker build/check runs it before `mx check`) | `cmd/mxcli/docker/check.go` + `build.go` (`updateWidgetsPathArg`) — the update-widgets *invocation*, not the widget emission | The real trap was that `mx update-widgets ` **crashed** (`AddProjectDirAsAllowedPath` → `Path.GetDirectoryName("app.mpr")` = "" → `System.ArgumentNullException`) and some mx builds exit 0 after printing it, so the migration silently no-op'd and CE0463 survived. Pass an **absolute** path to update-widgets (always has a directory component). `mx check` is unaffected. Diagnosis: run the bundled `mx update-widgets ` yourself — if it throws ArgumentNullException on AddProjectDirAsAllowedPath, the path lacks a dir. Faithful multi-version widget emission is the larger fix (#529). Issue #112; repro `mdl-examples/bug-tests/112-combobox-enum-ce0463-widget-version.mdl` | +| **CE0463 "widget definition changed" at an `Image` widget** on Mendix 11.7+ (`mx check` without `update-widgets`) — a plain mxcli-authored Image, no custom config. `update-widgets` clears it (and on v2 destroys `mprcontents/`, so it's a data-loss trap) | NOT a version stamp, Type `$ID`s, property order, or missing properties (all ruled out empirically). The embedded `image.json` carried a **spurious default value**: a `WidgetValue.Image` pointing at `Atlas_Core.Content.Mendix` (Atlas's Mendix logo), captured when the template was extracted from a project that had it set. The installed 11.12 Image widget expects that field empty, so MxBuild flags the definition as changed | `modelsdk/widgets/templates/mendix-11.6/image.json` + `sdk/widgets/templates/mendix-11.6/image.json` (line ~63) | Clear the stale default: `"Image": "Atlas_Core.Content.Mendix"` → `"Image": ""` in both engine templates. **Diagnosis method for this whole CE0463 class**: dump the widget BSON, `mx update-widgets` on a COPY, dump again, diff the `CustomWidgets$CustomWidget` subtree **order-independently** (canonicalise key order + mask `$ID`/`TypePointer` blobs). Reordering and generic instance chrome (`LabelTemplate`, `Appearance.DesignProperties`) are cosmetic — a *passing* widget gets reordered/those-added too; the real cause is whatever value/structure survives that normalisation. Other hand-extracted templates likely hide similar stale defaults (audit with the same diff). Repro `mdl-examples/bug-tests/image-ce0463-stale-default.mdl`. DataGrid2 custom-content CE0463 (#600) is a *separate*, more complex delta — same method, own fix | +| `mxcli docker check`/`build` (or a bare `mx update-widgets`) **silently deletes `mprcontents/`** and rewrites an MPRv2 project into single-file v1 — `check` reports **0 errors** and looks successful, but the git working tree diverges from tracked files, a running `mxcli run --local` loop breaks (it watches `mprcontents/`), and Studio Pro may crash on open (`LibGit2RepositoryProvider.WriteBaseFile`). Triggered by following the CE0463 remediation on a `mxcli new` (always v2) project | The pre-check `mx update-widgets` step (run to suppress false CE0463) **performs the conversion**: it inlines every unit into the `.mpr` (`Unit.Contents` column) and deletes `mprcontents/`. The `check` itself is read-only; `update-widgets` is the mutator. docker check/build invoked it with no storage-format protection | `cmd/mxcli/docker/check.go` (`snapshotStorageFormat` / `copyFile` / `copyDir`) + `build.go` | Snapshot `.mpr` + `mprcontents/` to a temp dir before `update-widgets`, `defer restore()` after the check (restore removes the post-conversion single-file `.mpr` residue and puts the v2 tree back); MPRv1 projects need no protection. The check still runs against the widget-normalized model, so CE0463 stays suppressed — only the on-disk format is preserved. **Never tell an agent to run bare `mx update-widgets` on a v2 project** — the synced skills (`create-page.md`, `custom-widgets.md`, `migrate-design-prototype.md`, `download-marketplace-content.md`) + dev `debug-bson.md` route to `mxcli docker check`/`build` (v2-safe) instead. Issue #763 / PR #764 | | Nightly `mx check` reports `CE0117 "Error(s) in expression." at Log message activity 'Log message (warning)'` on Mendix 10.24.19+ but not 10.24.16 or 11.x | Mendix 10.24.19 tightened expression validation: `toString()` is now a type error (toString expects a non-string input). An example called `toString($OrderNumber)` where `$OrderNumber` was already a string parameter | The offending `log warning ... with ({1} = toString($stringVar))` — find via `~/.mxcli/mxbuild/{ver}/modeler/mx check`, then bisect with `drop microflow ...` until CE0117 disappears | Remove the redundant `toString()` wrapper around already-string values. Only wrap non-string values (integers, decimals, dates, enums) in `toString()`. The Mendix 11.x parser is more lenient and lets this slide, but 10.24.19+ rejects it | | A page-level property can't be set — `ALTER PAGE X { SET PopupWidth = 800; }` (or any page-level prop other than Title/Url) fails with `unsupported page-level property: …` | The page-level SET handler only special-cased a couple of properties; everything else fell through to the default error | `mdl/backend/pagemutator/mutator.go` → `applyPageLevelSetMut` (shared by both engines) | Add a `case` writing the field at the top level of the Forms$Page doc via `dSetOrAppend` with the on-disk BSON type (int64 for PopupWidth/PopupHeight, bool for PopupResizable — verify against a Studio Pro page with `mxcli bson dump --format bson`). Page-level prop names are **case-sensitive**. For DESCRIBE roundtrip, emit the values back in the CREATE PAGE header. CREATE-time support: add a generic `IDENTIFIER COLON propertyValueV3` to `pageHeaderPropertyV3` (regen grammar), recognize the keys in `parsePageHeaderV3` (`applyGenericPageHeaderProp`, error on unknown), carry `*int`/`*bool` on `CreatePageStmtV3`, default to 600/600/false in `buildPageV3`, and have both writers honour `page.Popup*` (legacy `sdk/mpr/writer_pages.go` int64; codec `mdl/backend/modelsdk/page_write.go` int32 via gen — tolerated by mx check). The MCP backend has its **own** `mcpPageMutator` (pg content tree, not raw BSON) — page-level SET there reaches `SetWidgetProperty("")`; map it or reject honestly (it rejects, pending a `pg_read_page` probe of the pop-up keys). Issue #661 | | `PopupWidth: 0` / `PopupHeight: 0` rejected ("must be a positive number") on CREATE or ALTER PAGE; user can't make an auto-size pop-up | 0 is actually Studio Pro's **default** for pop-up dimensions (auto-size) — verified live on 11.12: a pg-created PopupLayout page stores 0/0 and `mx check` = 0 errors. Two validators rejected ≤0, and both writers coerced ≤0→600, so even an allowed 0 became 600 | `mdl/visitor/visitor_page_v3.go` (`popupDimensionValue`) + `mdl/backend/pagemutator/mutator.go` (`coercePopupDimension`) + `mdl/executor/cmd_pages_builder_v3.go` (builder default) + `sdk/mpr/writer_pages.go` & `mdl/backend/modelsdk/page_write.go` (`popupDimension`) + `mdl/executor/cmd_pages_describe.go` | Relax both validators to reject only **negative**; default the builder to **0** (not 600, matching Studio Pro); drop the `≤0→600` coercion in both writers (clamp only negatives to 0); have DESCRIBE suppress only the real default 0 (emit an explicit 600). No `*int` needed — 0 is a valid stored value. Bug-test `mdl-examples/bug-tests/713-popup-zero-dimensions.mdl`. Issue #713 | diff --git a/.claude/skills/mendix/alter-page.md b/.claude/skills/mendix/alter-page.md index 2f691c8ab..4865f46c5 100644 --- a/.claude/skills/mendix/alter-page.md +++ b/.claude/skills/mendix/alter-page.md @@ -134,10 +134,22 @@ insert after txtName { insert before btnSave { actionbutton btnPreview (caption: 'Preview', action: microflow Module.ACT_Preview) } + +-- Insert INTO a container — append as its last child (works on an EMPTY container) +insert into ctnToolbar { + actionbutton btnNew (caption: 'New', action: nothing, buttonstyle: primary) +} ``` Inserted widgets use the same syntax as `create page`. Multiple widgets can be inserted in a single block. +`insert into ` appends as the last child of the named container — the +only way to fill an **empty** container, and handy for adding to a container/dataview +without needing a sibling to anchor to. Widgets inserted into a dataview take that +dataview's entity as their context. Supported on simple containers (container, +dataview, groupbox, scroll-container region); for a layout grid or tab container, +insert relative to a widget inside the target column/tab instead. + ### DROP - Remove Widgets ```sql diff --git a/.claude/skills/mendix/create-page.md b/.claude/skills/mendix/create-page.md index 46ce76e34..ec5e3400e 100644 --- a/.claude/skills/mendix/create-page.md +++ b/.claude/skills/mendix/create-page.md @@ -831,9 +831,16 @@ Notes: PieChart use the same `series` shape. - **CE0463 at `mx check`**: charts can report "widget definition changed" from widget-version drift (embedded template vs the installed `Charts.mpk`), even for a - chart with no series. `mxcli docker check`/`build` fix this automatically by - running `mx update-widgets` first; if you invoke `mx check` directly, run - `mx update-widgets ` (absolute path) beforehand. + chart with no series. Clear it with **`mxcli docker check`/`build`**, which normalize + the widgets and preserve your storage format. + **Do NOT run bare `mx update-widgets` on an MPRv2 project** (one with an + `mprcontents/` folder — everything `mxcli new` creates). `mx update-widgets` rewrites + the project into the single-file v1 format and **deletes `mprcontents/`**, which + corrupts a git working tree, breaks a running `mxcli run --local` loop, and can leave + Studio Pro unable to open the project. `mxcli docker check`/`build` snapshot and + restore the v2 files around the normalization, so they are safe; raw + `mx update-widgets` is only safe on a v1 project (or on a throwaway copy used purely + for diagnosis). - LineChart/BubbleChart/HeatMap (the `line`/`scalecolor` object-lists) are not yet authorable via MDL — use Studio Pro for those. diff --git a/.claude/skills/mendix/custom-widgets.md b/.claude/skills/mendix/custom-widgets.md index fc57b32db..e45d6e0f2 100644 --- a/.claude/skills/mendix/custom-widgets.md +++ b/.claude/skills/mendix/custom-widgets.md @@ -123,10 +123,16 @@ MDL032). **CE0463 "update this widget" is EXPECTED after generating charts.** mxcli writes the WidgetType from an embedded 11.6 baseline; the installed Charts.mpk is a -different version, so Studio Pro/mxbuild flags drift. Clear it with -`mx update-widgets` (or just use `mxcli docker check`/`build`, which run it first) -before `mx check`. The whole `mdl-examples/doctype-tests/34-chart-widget-examples.mdl` -builds **0 errors** after update-widgets. +different version, so Studio Pro/mxbuild flags drift. Clear it with **`mxcli docker +check`/`build`** (they normalize the widgets and preserve your storage format). The +whole `mdl-examples/doctype-tests/34-chart-widget-examples.mdl` builds **0 errors** +after normalization. +**Do NOT run bare `mx update-widgets` on an MPRv2 project** (an `mprcontents/`-folder +project — what `mxcli new` creates): it converts the project to single-file v1 and +**deletes `mprcontents/`**, corrupting git, breaking a running `mxcli run --local` +loop, and sometimes making the project unopenable in Studio Pro. `mxcli docker +check`/`build` snapshot/restore the v2 files around the normalization; raw +`mx update-widgets` is only safe on a v1 project or a throwaway diagnostic copy. **DESCRIBE round-trips** series/line/scalecolor object-lists (item names are synthesized, e.g. `series1`); a Pie/HeatMap's widget-level `SeriesName`/datasource diff --git a/.claude/skills/mendix/download-marketplace-content.md b/.claude/skills/mendix/download-marketplace-content.md index 7917103bd..e5e87a105 100644 --- a/.claude/skills/mendix/download-marketplace-content.md +++ b/.claude/skills/mendix/download-marketplace-content.md @@ -60,7 +60,7 @@ mxcli marketplace install -p app.mpr [--version X.Y.Z] | Content type | Behavior | |---|---| -| **Widget** | Copied into `widgets/` (overwrites on update). Reload in Studio Pro or run `mx update-widgets`. | +| **Widget** | Copied into `widgets/` (overwrites on update). Reload in Studio Pro, or run `mxcli docker check`/`build` to normalize (v2-safe). Do **not** run bare `mx update-widgets` on an `mprcontents/` project — it converts to v1 and deletes `mprcontents/`. | | **Module** (new) | Imported via `mx module-import` — needs a matching mxbuild (`mxcli setup mxbuild -p app.mpr`). | | **Module** (already present) | **Reported, not modified** — see the caveat below. | | Theme / Starter App / Sample | Downloaded with import instructions (import via Studio Pro). | diff --git a/.claude/skills/mendix/migrate-design-prototype.md b/.claude/skills/mendix/migrate-design-prototype.md index 997587044..ad15fd51a 100644 --- a/.claude/skills/mendix/migrate-design-prototype.md +++ b/.claude/skills/mendix/migrate-design-prototype.md @@ -256,7 +256,9 @@ too — each `series`/`line` binds its own OQL-view datasource + X/Y attributes; Pie/HeatMap bind at the widget level (`ValueAttribute:`, Pie needs `SeriesName:`). See **[Custom & Pluggable Widgets → Charts](custom-widgets.md)** for the chart-type → id table, per-chart required-property gotchas (TimeSeries needs a datetime X, -Bubble needs a size attribute), and the **CE0463 → `mx update-widgets`** step. +Bubble needs a size attribute), and the **CE0463 → `mxcli docker check`/`build`** step +(these normalize widgets *and* preserve MPRv2 storage — never run bare +`mx update-widgets` on a `mxcli new` project; it deletes `mprcontents/`). `mdl-examples/doctype-tests/34-chart-widget-examples.mdl` is the full showcase. **Pluggable-widget gotchas:** @@ -502,8 +504,9 @@ the fast index so a design migration doesn't rediscover them. (`Charts.mpk`: column/bar/line/area/pie)** now author via MDL — each `series` (an object-list item inside the chart) binds a datasource plus X/Y attributes: `series s1 (staticDataSource: database from Module.View, staticXAttribute: "X", staticYAttribute: "Y")` - (a per-series OQL view works too). `mxcli docker check`/`build` run `mx update-widgets`, which - clears the widget-version-drift CE0463. Still lighter when the design allows: a **CSS-background + (a per-series OQL view works too). `mxcli docker check`/`build` clear the + widget-version-drift CE0463 (they normalize the widgets and preserve MPRv2 storage — + do not run bare `mx update-widgets`, which deletes `mprcontents/`). Still lighter when the design allows: a **CSS-background SVG** container (or `HTMLElement`) for sparklines/trends — no datasource — and `ProgressCircle` (`type: expression`, `expressionCurrentValue: '$currentObject/Rate'`, min `'0'` / max `'100'`, `labelType: percentage`) for a single-value gauge. diff --git a/CLAUDE.md b/CLAUDE.md index d6f0eb6f9..60b4d2d65 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -417,7 +417,7 @@ go build -o bin/mxcli ./cmd/mxcli | **Diff local** | `mxcli diff-local -p app.mpr --ref head` | Git diff for MPR v2 projects | | **Diff revisions** | `mxcli diff-local -p app.mpr --ref main..feature` | Compare two arbitrary git revisions | | **OQL** | `mxcli oql -p app.mpr "select ..."` | Query running Mendix runtime | -| **Widgets** | `show widgets`, `update widgets set ...` | Widget discovery and bulk updates (experimental) | +| **Widgets** | `show widgets`, `update widgets set ...`, `mxcli widget describe ` | Widget discovery, bulk updates (experimental), and inspecting a widget's discovered properties + dynamic rules | | **External SQL** | `sql connect`, `sql select ...`, `mxcli sql` | Direct SQL queries against PostgreSQL, Oracle, SQL Server (credential isolation) | | **Data import** | `import from query '...' into Module.Entity map (...)` | Import from external DB into Mendix app PostgreSQL (batch insert with ID generation) | | **Connector gen** | `sql generate connector into [tables (...)] [views (...)] [exec]` | Auto-generate Database Connector MDL from discovered schema | diff --git a/cmd/mxcli/cmd_widget_describe.go b/cmd/mxcli/cmd_widget_describe.go new file mode 100644 index 000000000..76912fc59 --- /dev/null +++ b/cmd/mxcli/cmd_widget_describe.go @@ -0,0 +1,415 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "encoding/json" + "fmt" + "path/filepath" + "sort" + "strings" + + "github.com/mendixlabs/mxcli/mdl/executor" + "github.com/mendixlabs/mxcli/mdl/types" + mwidgets "github.com/mendixlabs/mxcli/modelsdk/widgets" + mmpk "github.com/mendixlabs/mxcli/modelsdk/widgets/mpk" + "github.com/spf13/cobra" +) + +var widgetDescribeCmd = &cobra.Command{ + Use: "describe ", + Short: "Inspect a pluggable widget's discovered format", + Long: `Show the format mxcli has discovered for a pluggable widget: its expected +properties (key, type, caption, category, required, default, enum options) and the +dynamic property rules (which properties the widget's editor hides under which +configuration) lifted from the widget package's editorConfig. + +The widget can be named by its MDL keyword (e.g. COMBOBOX, DATAGRID2) or its full +widget id (e.g. com.mendix.widget.web.combobox.Combobox). + +With -p, the properties and dynamic rules come from the widget package actually +installed in the project (widgets/*.mpk) — the version-accurate "discovered" format, +including marketplace widgets mxcli has no built-in knowledge of. Without -p, they +come from mxcli's embedded template for that widget.`, + Example: ` mxcli widget describe COMBOBOX + mxcli widget describe DATAGRID2 -p app.mpr + mxcli widget describe com.mendix.widget.web.combobox.Combobox -p app.mpr --format json`, + Args: cobra.ExactArgs(1), + RunE: runWidgetDescribe, +} + +func init() { + widgetDescribeCmd.Flags().StringP("project", "p", "", "Path to .mpr project file (use the project's installed widget version)") + widgetDescribeCmd.Flags().String("format", "text", "Output format: text or json") + widgetCmd.AddCommand(widgetDescribeCmd) +} + +// describedProperty is one property of a widget's discovered format. +type describedProperty struct { + Key string `json:"key"` + Type string `json:"type"` + Caption string `json:"caption,omitempty"` + Category string `json:"category,omitempty"` + Required bool `json:"required"` + Default string `json:"default,omitempty"` + System bool `json:"system,omitempty"` + Enum []string `json:"enum,omitempty"` + Children []describedProperty `json:"children,omitempty"` +} + +// describedRule is one dynamic (visibility) rule of a widget's discovered format. +type describedRule struct { + Property string `json:"property"` + HiddenWhen string `json:"hiddenWhen"` +} + +// widgetDescription is the full inspection result (also the JSON shape). +type widgetDescription struct { + WidgetID string `json:"widgetId"` + MDLName string `json:"mdlName,omitempty"` + Name string `json:"name,omitempty"` + Version string `json:"version,omitempty"` + Source string `json:"source"` // "project .mpk" | "embedded template" + Kind string `json:"kind,omitempty"` + Properties []describedProperty `json:"properties"` + Rules []describedRule `json:"dynamicRules"` + RuleCoverage string `json:"ruleCoverage,omitempty"` +} + +func runWidgetDescribe(cmd *cobra.Command, args []string) error { + arg := args[0] + projectPath, _ := cmd.Flags().GetString("project") + format, _ := cmd.Flags().GetString("format") + + registry, err := executor.NewWidgetRegistry() + if err != nil { + return fmt.Errorf("failed to create widget registry: %w", err) + } + if projectPath != "" { + _ = registry.LoadUserDefinitions(projectPath) + } + + // Resolve the target widget id + optional built-in definition. + widgetID, def := resolveWidgetTarget(registry, arg) + if widgetID == "" { + return widgetNotFoundError(registry, arg) + } + + desc := widgetDescription{WidgetID: widgetID} + if def != nil { + desc.MDLName = def.MDLName + desc.Kind = def.WidgetKind + } + if desc.Kind == "" { + desc.Kind = "pluggable" + } + + // Properties + version: prefer the project's installed .mpk (version-accurate, + // includes marketplace widgets); else fall back to mxcli's embedded template. + if projectPath != "" { + if dir := projectDirOf(projectPath); dir != "" { + if mpkPath, ferr := mmpk.FindMPK(dir, widgetID); ferr == nil && mpkPath != "" { + if wd, perr := mmpk.ParseMPKForWidget(mpkPath, widgetID); perr == nil && wd != nil { + desc.Name = wd.Name + desc.Version = wd.Version + desc.Source = "project .mpk" + desc.Properties = propsFromMPK(wd) + desc.Rules, desc.RuleCoverage = rulesFromProject(mpkPath, widgetID) + } + } + } + } + if desc.Source == "" { + // Embedded template fallback. + tmpl, terr := mwidgets.GetTemplate(widgetID) + if terr != nil || tmpl == nil { + return fmt.Errorf("no installed .mpk and no embedded template for %q — try -p to inspect a project widget", arg) + } + desc.Name = tmpl.Name + desc.Version = tmpl.Version + desc.Source = "embedded template" + desc.Properties = propsFromTemplate(tmpl.Type) + if def != nil { + desc.Rules = rulesFromDef(def.PropertyVisibility) + } + } + + if strings.EqualFold(format, "json") { + enc := json.NewEncoder(cmd.OutOrStdout()) + enc.SetIndent("", " ") + return enc.Encode(desc) + } + printWidgetDescription(cmd, desc) + return nil +} + +// resolveWidgetTarget maps a CLI argument (MDL keyword or widget id) to a widget id +// and, when known, the built-in WidgetDefinition. A dotted argument is treated as a +// widget id directly. +func resolveWidgetTarget(registry *executor.WidgetRegistry, arg string) (string, *executor.WidgetDefinition) { + if strings.Contains(arg, ".") { + if def, ok := registry.GetByWidgetID(arg); ok { + return arg, def + } + return arg, nil // unknown to the registry, but a valid id to look up in the project + } + upper := strings.ToUpper(arg) + if def, ok := registry.Get(upper); ok { + return def.WidgetID, def + } + // Well-known widgets that are special-cased in the executor (no .def.json in the + // registry) but that users still name by keyword. + if id, ok := builtinWidgetAliases[upper]; ok { + def, _ := registry.GetByWidgetID(id) + return id, def + } + return "", nil +} + +// builtinWidgetAliases maps MDL keywords for executor-special-cased widgets (which +// have no .def.json registry entry) to their widget ids, so `widget describe` can +// resolve them by the same friendly names users write in MDL. +var builtinWidgetAliases = map[string]string{ + "DATAGRID": "com.mendix.widget.web.datagrid.Datagrid", + "DATAGRID2": "com.mendix.widget.web.datagrid.Datagrid", +} + +// widgetNotFoundError builds a helpful error listing the known MDL names. +func widgetNotFoundError(registry *executor.WidgetRegistry, arg string) error { + var names []string + for _, d := range registry.All() { + if d.MDLName != "" { + names = append(names, d.MDLName) + } + } + for alias := range builtinWidgetAliases { + names = append(names, strings.ToLower(alias)) + } + sort.Strings(names) + return fmt.Errorf("unknown widget %q — use an MDL keyword (%s) or a full widget id (com.mendix.widget…). Run `mxcli widget list` to see all", + arg, strings.Join(names, ", ")) +} + +// projectDirOf returns the directory containing widgets/ for a project path +// (accepts either the .mpr file or its directory). +func projectDirOf(projectPath string) string { + if strings.EqualFold(filepath.Ext(projectPath), ".mpr") { + return filepath.Dir(projectPath) + } + return projectPath +} + +// propsFromMPK builds described properties from a parsed .mpk definition, in the +// widget's declared order (regular + system interleaved). +func propsFromMPK(wd *mmpk.WidgetDefinition) []describedProperty { + order := wd.AllTopLevel + if len(order) == 0 { + order = wd.Properties + } + out := make([]describedProperty, 0, len(order)) + for _, p := range order { + out = append(out, describedPropFromMPK(p)) + } + return out +} + +func describedPropFromMPK(p mmpk.PropertyDef) describedProperty { + dp := describedProperty{ + Key: p.Key, + Type: p.Type, + Caption: p.Caption, + Category: p.Category, + Required: p.Required, + Default: p.DefaultValue, + System: p.IsSystem, + } + if dp.System && dp.Type == "" { + dp.Type = "system" + } + for _, ev := range p.EnumValues { + dp.Enum = append(dp.Enum, ev.Key) + } + for _, c := range p.Children { + dp.Children = append(dp.Children, describedPropFromMPK(c)) + } + return dp +} + +// propsFromTemplate walks an embedded template's Type map (ObjectType.PropertyTypes) +// to build described properties. Used when no project .mpk is available. +func propsFromTemplate(typ map[string]any) []describedProperty { + objType, _ := typ["ObjectType"].(map[string]any) + pts, _ := objType["PropertyTypes"].([]any) + var out []describedProperty + for _, pt := range pts { + m, ok := pt.(map[string]any) + if !ok { + continue // leading array marker + } + out = append(out, describedPropFromTemplate(m)) + } + return out +} + +func describedPropFromTemplate(m map[string]any) describedProperty { + dp := describedProperty{ + Key: asString(m["PropertyKey"]), + Caption: asString(m["Caption"]), + Category: asString(m["Category"]), + } + vt, _ := m["ValueType"].(map[string]any) + if vt != nil { + dp.Type = asString(vt["Type"]) + dp.Default = asString(vt["DefaultValue"]) + if r, ok := vt["Required"].(bool); ok { + dp.Required = r + } + if evs, ok := vt["EnumerationValues"].([]any); ok { + for _, ev := range evs { + if em, ok := ev.(map[string]any); ok { + if k := asString(em["_Key"]); k != "" { + dp.Enum = append(dp.Enum, k) + } + } + } + } + if nested, ok := vt["ObjectType"].(map[string]any); ok { + if npts, ok := nested["PropertyTypes"].([]any); ok { + for _, npt := range npts { + if nm, ok := npt.(map[string]any); ok { + dp.Children = append(dp.Children, describedPropFromTemplate(nm)) + } + } + } + } + } + dp.System = isSystemPropKey(dp.Key) + return dp +} + +func isSystemPropKey(key string) bool { + switch key { + case "Label", "Visibility", "Editability", "Name", "TabIndex": + return true + } + return false +} + +// rulesFromProject extracts dynamic rules from the project's installed .mpk editor +// config, returning the rules and a coverage note (recognized / total hide-calls). +func rulesFromProject(mpkPath, widgetID string) ([]describedRule, string) { + rules, recognized, total := executor.ExtractWidgetVisibilityStats(mpkPath, widgetID) + coverage := "" + if total > 0 { + coverage = fmt.Sprintf("%d of %d editor hide-rules recognized", recognized, total) + } + return rulesToDescribed(rules), coverage +} + +func rulesFromDef(rules []types.WidgetVisibilityRule) []describedRule { + return rulesToDescribed(rules) +} + +func rulesToDescribed(rules []types.WidgetVisibilityRule) []describedRule { + out := make([]describedRule, 0, len(rules)) + for _, r := range rules { + if r.HiddenWhen == nil { + continue + } + out = append(out, describedRule{Property: r.PropertyKey, HiddenWhen: conditionText(r.HiddenWhen)}) + } + sort.Slice(out, func(i, j int) bool { return out[i].Property < out[j].Property }) + return out +} + +// conditionText renders a visibility condition as readable English. +func conditionText(c *types.WidgetVisibilityCondition) string { + switch c.Operator { + case "eq": + return fmt.Sprintf("%s = %q", c.PropertyKey, c.Value) + case "ne": + return fmt.Sprintf("%s ≠ %q", c.PropertyKey, c.Value) + case "truthy": + return fmt.Sprintf("%s is set", c.PropertyKey) + case "falsy": + return fmt.Sprintf("%s is not set", c.PropertyKey) + default: + return fmt.Sprintf("%s %s %q", c.PropertyKey, c.Operator, c.Value) + } +} + +func asString(v any) string { + s, _ := v.(string) + return s +} + +func printWidgetDescription(cmd *cobra.Command, d widgetDescription) { + out := cmd.OutOrStdout() + title := d.Name + if title == "" { + title = d.WidgetID + } + fmt.Fprintf(out, "Widget: %s", title) + if d.MDLName != "" { + fmt.Fprintf(out, " (%s)", d.MDLName) + } + fmt.Fprintln(out) + fmt.Fprintf(out, " ID: %s\n", d.WidgetID) + if d.Version != "" { + fmt.Fprintf(out, " Version: %s\n", d.Version) + } + fmt.Fprintf(out, " Kind: %s\n", d.Kind) + fmt.Fprintf(out, " Source: %s\n", d.Source) + + fmt.Fprintf(out, "\nProperties (%d):\n", countProps(d.Properties)) + printProps(out, d.Properties, 0) + + fmt.Fprintf(out, "\nDynamic property rules (%d):\n", len(d.Rules)) + if len(d.Rules) == 0 { + fmt.Fprintln(out, " (none discovered)") + } + for _, r := range d.Rules { + fmt.Fprintf(out, " %-40s hidden when %s\n", r.Property, r.HiddenWhen) + } + if d.RuleCoverage != "" { + fmt.Fprintf(out, " — %s\n", d.RuleCoverage) + } +} + +func countProps(props []describedProperty) int { + n := 0 + for _, p := range props { + n++ + n += countProps(p.Children) + } + return n +} + +func printProps(out interface{ Write([]byte) (int, error) }, props []describedProperty, depth int) { + indent := strings.Repeat(" ", depth+1) + for _, p := range props { + req := "" + if p.Required { + req = " required" + } + sys := "" + if p.System { + sys = " [system]" + } + line := fmt.Sprintf("%s%-34s %-13s", indent, p.Key, p.Type) + extra := strings.TrimRight(req+sys, " ") + if p.Default != "" { + extra = strings.TrimSpace(extra + " default=" + p.Default) + } + if len(p.Enum) > 0 { + extra = strings.TrimSpace(extra + " {" + strings.Join(p.Enum, "|") + "}") + } + if p.Category != "" { + extra = strings.TrimSpace(extra + " (" + p.Category + ")") + } + fmt.Fprintf(out, "%s %s\n", strings.TrimRight(line, " "), extra) + if len(p.Children) > 0 { + printProps(out, p.Children, depth+1) + } + } +} diff --git a/cmd/mxcli/cmd_widget_describe_test.go b/cmd/mxcli/cmd_widget_describe_test.go new file mode 100644 index 000000000..c7b3cfa17 --- /dev/null +++ b/cmd/mxcli/cmd_widget_describe_test.go @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "encoding/json" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/executor" + "github.com/mendixlabs/mxcli/mdl/types" +) + +// TestWidgetDescribe_EmbeddedCombobox runs `widget describe COMBOBOX --format json` +// against mxcli's embedded template (no project) and checks the discovered format. +func TestWidgetDescribe_EmbeddedCombobox(t *testing.T) { + var out strings.Builder + cmd := widgetDescribeCmd + cmd.SetOut(&out) + cmd.SetErr(&out) + if err := runWidgetDescribe(cmd, []string{"COMBOBOX"}); err != nil { + t.Fatalf("describe COMBOBOX: %v", err) + } + // Re-run with --format json by setting the flag. + out.Reset() + _ = cmd.Flags().Set("format", "json") + defer cmd.Flags().Set("format", "text") + if err := runWidgetDescribe(cmd, []string{"COMBOBOX"}); err != nil { + t.Fatalf("describe COMBOBOX json: %v", err) + } + var d widgetDescription + if err := json.Unmarshal([]byte(out.String()), &d); err != nil { + t.Fatalf("unmarshal json: %v\n%s", err, out.String()) + } + if d.WidgetID != "com.mendix.widget.web.combobox.Combobox" { + t.Errorf("widgetId = %q", d.WidgetID) + } + if d.Source != "embedded template" { + t.Errorf("source = %q, want embedded template", d.Source) + } + if len(d.Properties) == 0 { + t.Fatal("no properties described") + } + // The declared order must place the system properties mid-list (after + // selectAllButtonCaption), not at the end — the ComboBox order fix. + idx := map[string]int{} + for i, p := range d.Properties { + idx[p.Key] = i + } + for _, k := range []string{"Label", "Visibility", "Editability", "customEditability"} { + if _, ok := idx[k]; !ok { + t.Errorf("expected property %q in described format", k) + } + } + if idx["Label"] < idx["selectAllButtonCaption"] || idx["Label"] > idx["customEditability"] { + t.Errorf("Label at %d not between selectAllButtonCaption (%d) and customEditability (%d)", + idx["Label"], idx["selectAllButtonCaption"], idx["customEditability"]) + } +} + +// TestWidgetDescribe_UnknownWidget reports a helpful error. +func TestWidgetDescribe_UnknownWidget(t *testing.T) { + reg, err := executor.NewWidgetRegistry() + if err != nil { + t.Fatalf("registry: %v", err) + } + id, _ := resolveWidgetTarget(reg, "NOPE") + if id != "" { + t.Errorf("resolveWidgetTarget(NOPE) = %q, want empty", id) + } + // DATAGRID2 resolves via the builtin alias even without a .def.json entry. + if id, _ := resolveWidgetTarget(reg, "datagrid2"); id != "com.mendix.widget.web.datagrid.Datagrid" { + t.Errorf("resolveWidgetTarget(datagrid2) = %q", id) + } +} + +// TestConditionText renders the four operators as readable English. +func TestConditionText(t *testing.T) { + cases := []struct { + op, val, want string + }{ + {"eq", "None", `itemSelection = "None"`}, + {"ne", "Multi", `itemSelection ≠ "Multi"`}, + {"truthy", "", "itemSelection is set"}, + {"falsy", "", "itemSelection is not set"}, + } + for _, c := range cases { + got := conditionText(&types.WidgetVisibilityCondition{PropertyKey: "itemSelection", Operator: c.op, Value: c.val}) + if got != c.want { + t.Errorf("op %s: got %q, want %q", c.op, got, c.want) + } + } +} diff --git a/cmd/mxcli/syntax/features_page.go b/cmd/mxcli/syntax/features_page.go index 5bab9f14e..5a4762a0d 100644 --- a/cmd/mxcli/syntax/features_page.go +++ b/cmd/mxcli/syntax/features_page.go @@ -89,7 +89,7 @@ func init() { "set property", "insert widget", "drop widget", "replace widget", "popup width", "popup height", "popup resizable", }, - Syntax: "ALTER PAGE Module.Name {\n SET property = value ON widgetName;\n SET (prop1 = val1, prop2 = val2) ON widgetName;\n SET Title = 'New Title'; -- page-level (case-sensitive)\n SET Class = 'css-class'; -- page-level CSS class / style\n SET Style = 'css: rule';\n SET PopupWidth = 800; -- page-level pop-up dimensions\n SET PopupHeight = 480;\n SET PopupResizable = true;\n INSERT AFTER widgetName { };\n INSERT BEFORE widgetName { };\n DROP WIDGET name1, name2;\n REPLACE widgetName WITH { };\n};", + Syntax: "ALTER PAGE Module.Name {\n SET property = value ON widgetName;\n SET (prop1 = val1, prop2 = val2) ON widgetName;\n SET Title = 'New Title'; -- page-level (case-sensitive)\n SET Class = 'css-class'; -- page-level CSS class / style\n SET Style = 'css: rule';\n SET PopupWidth = 800; -- page-level pop-up dimensions\n SET PopupHeight = 480;\n SET PopupResizable = true;\n INSERT AFTER widgetName { };\n INSERT BEFORE widgetName { };\n INSERT INTO containerName { };\n DROP WIDGET name1, name2;\n REPLACE widgetName WITH { };\n};", Example: "ALTER PAGE Module.EditPage {\n SET (Caption = 'Save & Close', ButtonStyle = Success) ON btnSave;\n INSERT AFTER txtName {\n TEXTBOX txtMiddleName (Label: 'Middle Name', Binds: MiddleName)\n };\n DROP WIDGET txtUnused;\n};", SeeAlso: []string{"page.create", "page.show", "snippet.alter"}, }) @@ -152,7 +152,7 @@ func init() { Keywords: []string{ "alter snippet", "modify snippet", "update snippet", }, - Syntax: "ALTER SNIPPET Module.Name {\n SET property = value ON widgetName;\n INSERT AFTER widgetName { };\n INSERT BEFORE widgetName { };\n DROP WIDGET name1, name2;\n REPLACE widgetName WITH { };\n};", + Syntax: "ALTER SNIPPET Module.Name {\n SET property = value ON widgetName;\n INSERT AFTER widgetName { };\n INSERT BEFORE widgetName { };\n INSERT INTO containerName { };\n DROP WIDGET name1, name2;\n REPLACE widgetName WITH { };\n};", Example: "ALTER SNIPPET Module.NavSnippet {\n REPLACE navItem1 WITH {\n ACTIONBUTTON btnHome (Caption: 'Home', Action: SHOW_PAGE Module.HomePage)\n };\n DROP WIDGET txtOldField;\n INSERT AFTER txtName {\n TEXTBOX txtNewField (Label: 'New Field', Binds: NewAttr)\n };\n};", SeeAlso: []string{"snippet", "page.alter"}, }) diff --git a/docs-site/src/SUMMARY.md b/docs-site/src/SUMMARY.md index 7537125cf..a744de98d 100644 --- a/docs-site/src/SUMMARY.md +++ b/docs-site/src/SUMMARY.md @@ -332,6 +332,7 @@ - [Capabilities Overview](reference/capabilities.md) - [Customizing AI Generation](guides/customizing-ai.md) +- [Pluggable Widgets Across Versions](guides/pluggable-widgets.md) - [Marketplace Content](guides/marketplace.md) --- diff --git a/docs-site/src/appendixes/quick-reference.md b/docs-site/src/appendixes/quick-reference.md index 2fdbd5bb5..d8c108a3e 100644 --- a/docs-site/src/appendixes/quick-reference.md +++ b/docs-site/src/appendixes/quick-reference.md @@ -421,6 +421,7 @@ Modify an existing page or snippet's widget tree in-place without full `CREATE O | Page-level set | `SET Title = 'New Title'` | No ON clause for page properties | | Insert after | `INSERT AFTER widgetName { widgets }` | Add widgets after target | | Insert before | `INSERT BEFORE widgetName { widgets }` | Add widgets before target | +| Insert into | `INSERT INTO containerName { widgets }` | Append as the container's last child (fills an empty container) | | Drop widgets | `DROP WIDGET name1, name2` | Remove widgets by name | | Replace widget | `REPLACE widgetName WITH { widgets }` | Replace widget subtree | | Pluggable prop | `SET 'showLabel' = false ON cbStatus` | Quoted name for pluggable widgets | diff --git a/docs-site/src/examples/alter-page.md b/docs-site/src/examples/alter-page.md index 5fbee57ab..af4461984 100644 --- a/docs-site/src/examples/alter-page.md +++ b/docs-site/src/examples/alter-page.md @@ -20,6 +20,18 @@ ALTER PAGE CRM.Customer_Edit { }; ``` +## Add a Widget Into a Container + +Use `INSERT INTO` to append a widget as a container's last child — the only way to fill an empty container. + +```sql +ALTER PAGE CRM.Customer_Overview { + INSERT INTO ctnToolbar { + ACTIONBUTTON btnNew (Caption: 'New Customer', Action: NOTHING, ButtonStyle: Primary) + } +}; +``` + ## Remove Widgets ```sql diff --git a/docs-site/src/guides/pluggable-widgets.md b/docs-site/src/guides/pluggable-widgets.md new file mode 100644 index 000000000..6b3ce3f86 --- /dev/null +++ b/docs-site/src/guides/pluggable-widgets.md @@ -0,0 +1,159 @@ +# Pluggable Widgets Across Mendix Versions + +Modern Mendix UI is built from **pluggable widgets** — DataGrid 2, Combo box, Gallery, +the data-grid filters, and any widget you install from the Marketplace. mxcli can create +and configure these widgets in a page, and it does so in a way that stays correct as the +installed widget versions change from project to project. This guide explains how that +works and how to inspect what mxcli has discovered about a widget. + +## The problem: "the definition of this widget has changed" (CE0463) + +Every pluggable-widget instance stored in an `.mpr` embeds a copy of the widget's +**definition** — the full list of properties, their types, default values, captions, +categories, and declared order. When you open the app, Studio Pro (and `mx check`) +compares that embedded definition against the widget package (`.mpk`) actually installed +in the project. If they differ in any way, you get: + +> **CE0463** — "The definition of this widget has changed. Update this widget by +> right-clicking it and selecting 'Update widget'…" + +Different Mendix versions — and different Marketplace releases of the same widget — ship +slightly different definitions: a property is added, an enumeration option is renamed, a +category moves, the property order changes. A tool that emitted one fixed definition would +trip CE0463 the moment the project's widget version didn't match. + +## How mxcli stays version-correct + +mxcli never hard-codes one widget shape. It reconciles a known-good template against the +widget package **installed in your project**, so the definition it writes matches the +version you actually have — no `mx update-widgets` step required. + +1. **Embedded template.** mxcli ships a known-good template for each built-in widget + (extracted from Studio Pro). This provides the correct nested BSON structure that is + hard to build from scratch. + +2. **Reconcile against the project `.mpk`.** When you pass a project (`-p`), mxcli finds + the widget's `.mpk` in the project's `widgets/` folder, parses its definition, and + updates the template to match: it adds and removes properties, rewrites enumeration + option sets, fixes categories/captions/defaults, fills in per-property metadata, and — + importantly — **re-orders the properties to the package's declared order**, including + the system properties (Label / Visibility / Editability) at their real position. This + is exactly what Studio Pro's own "Update widget" does, derived generically from the + package with no widget-specific code. + +3. **Apply the widget's own dynamic rules.** Which properties a widget shows depends on + its *configuration* — a Combo box in enumeration mode hides the association properties; + a DataGrid 2 with selection off hides the selection labels. That logic lives in each + widget's compiled `editorConfig.js`. mxcli statically extracts those **dynamic property + rules** and applies them, so a freshly-created widget carries exactly the defaults + Studio Pro would give it. + +The result: widgets created by mxcli open cleanly across Mendix 10.x and 11.x with the +bundled widget packages, and against Marketplace-updated packages, without manual +"Update widgets" fix-ups. + +> For the internal mechanics (template extraction, BSON cross-references, the augment +> pipeline), see [Widget Template System](../internals/widget-templates.md) and +> [Pluggable Widget Engine](../internals/widget-engine.md). + +## Inspecting the discovered widget format + +Two commands let you see exactly what mxcli knows about the widgets available to a project. + +### List available widgets + +```bash +mxcli widget list # built-in widget definitions +mxcli widget list -p app.mpr # also loads project-specific definitions +``` + +### Describe one widget + +`mxcli widget describe` shows a widget's **expected properties** and its **dynamic property +rules**. Name the widget by its MDL keyword (`COMBOBOX`, `DATAGRID2`, `GALLERY`, …) or its +full widget id. + +```bash +# mxcli's built-in view of the widget: +mxcli widget describe COMBOBOX + +# the version-accurate view from the package installed in the project: +mxcli widget describe COMBOBOX -p app.mpr + +# machine-readable: +mxcli widget describe DATAGRID2 -p app.mpr --format json +``` + +With `-p`, the properties and rules come from the widget package actually installed in the +project (`widgets/*.mpk`) — the version-accurate "discovered" format, and the only way to +inspect a Marketplace widget mxcli has no built-in knowledge of. Without `-p`, they come +from mxcli's embedded template. + +Example (abridged): + +```text +Widget: Combo box (combobox) + ID: com.mendix.widget.web.combobox.Combobox + Version: 2.5.0 + Kind: pluggable + Source: project .mpk + +Properties (58): + source enumeration required default=context {context|database|static} (General::Data source) + optionsSourceType enumeration required default=association {association|enumeration|boolean} (General::Data source) + attributeEnumeration attribute required (General::Data source) + … + selectAllButtonCaption textTemplate required (General::Multiple-selection (reference set)) + Label system [system] + Visibility system [system] + Editability system [system] + customEditability enumeration required default=default {default|never|conditionally} (General::Editability) + … + +Dynamic property rules (10): + itemSelectionMethod hidden when itemSelection = "None" + keepSelection hidden when itemSelection ≠ "Multi" + loadMoreButtonCaption hidden when pagination ≠ "loadMore" + … + — 7 of 23 editor hide-rules recognized +``` + +The property list is in the widget's **declared order** (system properties appear where the +package declares them, not appended at the end), each with its type, whether it's required, +its default, enumeration options, and category. Object-list widgets (e.g. DataGrid 2 +columns) show their nested item properties indented. The dynamic rules read as +"property *X* is hidden when *Y*", and the trailing coverage line reports how many of the +widget's editor hide-rules mxcli recognized. + +### `--format json` + +Both the properties and the rules are available as JSON for scripting: + +```json +{ + "widgetId": "com.mendix.widget.web.combobox.Combobox", + "version": "2.5.0", + "source": "project .mpk", + "properties": [ + { "key": "source", "type": "enumeration", "required": true, "default": "context", + "enum": ["context", "database", "static"], "category": "General::Data source" } + ], + "dynamicRules": [ + { "property": "loadMoreButtonCaption", "hiddenWhen": "pagination ≠ \"loadMore\"" } + ] +} +``` + +## Marketplace and custom widgets + +`mxcli widget describe -p app.mpr ` works for **any** widget installed in the +project, including Marketplace and custom widgets mxcli has no embedded template for — it +reads the definition and editor rules straight from the installed `.mpk`. To teach the +page builder to author a custom widget from MDL, extract a definition for it first: + +```bash +mxcli widget extract --mpk widgets/MyWidget.mpk # writes a .def.json +mxcli widget init -p app.mpr # extract defs for all project widgets +``` + +See [Marketplace Content](marketplace.md) for downloading and installing packages. diff --git a/docs-site/src/internals/widget-engine.md b/docs-site/src/internals/widget-engine.md index f90513c96..1ea1f3fbe 100644 --- a/docs-site/src/internals/widget-engine.md +++ b/docs-site/src/internals/widget-engine.md @@ -271,10 +271,21 @@ CREATE PAGE MyModule.ReviewPage (...) { } ``` +## Object-Lists and DataGrid2 Columns + +A widget property can be an **object-list**: a repeated, structured child object rather than a single value or a flat widget slot. DataGrid 2 columns are the canonical example — each column is its own object with a header, an attribute (or custom content), a filter, sortable/width settings, and so on. The engine models this with `ObjectListMapping` (`mdl/executor/widget_engine.go`): + +- **`ItemProperties`** — scalar sub-properties of each item (attribute, caption/header, `showContentAs`, sortable, width, …), each with its own operation and MDL aliases. +- **`ItemSlots`** — *nested widget* slots inside each item, with `AcceptedChildTypes`. This is how a per-column **filter** widget (`textfilter`/`numberfilter`/`datefilter`/`dropdownfilter`) or a custom-content widget lands in the right slot of its column. + +**DataGrid 2 is built entirely through this generic path when building a page.** `datagrid` has no dedicated `case` in `buildWidgetV3`; it resolves through the widget registry (a `datagrid.def.json` generated on demand from the project's installed `.mpk`, with columns emitted as an `ObjectListMapping`) and builds via `buildPluggable`. Columns are serialized by `Builder.SetObjectList` → `buildObjectListItemBSON`, walking the template's PropertyType order (the CE0463-safe ordering) and applying the same empty-`ClientTemplate`/null conventions per column kind. There is no DataGrid-specific full-page builder — Gallery, by contrast, needs only child slots (its items are a single repeated template), while DataGrid needs the object-list machinery because each column is a distinct heterogeneous object carrying a nested filter. + ## What Stays Hardcoded **Native Mendix widgets** (TextBox, DataView, ListView, LayoutGrid, Container, etc.) use `Forms$TextBox`, `Forms$DataView`, etc. -- NOT `CustomWidgets$CustomWidget`. These stay as hardcoded builders because they have fundamentally different BSON structures and don't use the template system. +**ALTER PAGE DataGrid 2 column insert/replace.** The one part of DataGrid 2 that is *not* on the generic engine is the in-place column mutation used by `alter page … insert/replace column` — `Mutator.InsertColumns`/`ReplaceColumn` build a single column's BSON via the hand-coded `BuildDataGrid2Column` (`mdl/backend/widgetobj/datagrid_column.go`) and a bespoke `BuildFilterWidget`, because they splice into an already-serialized grid object rather than building a fresh widget. This duplicates the column conventions the object-list engine already applies; re-expressing it on `SetObjectList` semantics is a tracked follow-up (see `datagrid_column.go`). The full-page build path does **not** use this code. + ## Key Source Files | File | Purpose | diff --git a/docs-site/src/internals/widget-templates.md b/docs-site/src/internals/widget-templates.md index 78ea12f54..dd500541e 100644 --- a/docs-site/src/internals/widget-templates.md +++ b/docs-site/src/internals/widget-templates.md @@ -2,6 +2,8 @@ Pluggable widgets (DataGrid2, ComboBox, Gallery, etc.) require embedded template definitions for correct BSON serialization. This page explains how the template system works, from extraction through runtime loading and property mapping. +> For the user-facing view — how mxcli keeps widget definitions version-correct across Mendix releases, and the `mxcli widget describe` command for inspecting a widget's discovered properties and dynamic rules — see [Pluggable Widgets Across Versions](../guides/pluggable-widgets.md). + ## Why Templates Are Needed Pluggable widgets in Mendix are defined by two BSON components stored inside each widget instance: diff --git a/docs-site/src/language/alter-page.md b/docs-site/src/language/alter-page.md index 6252d19d1..677b34261 100644 --- a/docs-site/src/language/alter-page.md +++ b/docs-site/src/language/alter-page.md @@ -110,8 +110,17 @@ ALTER PAGE Module.EditPage { ACTIONBUTTON btnPreview (Caption: 'Preview', Action: MICROFLOW Module.ACT_Preview) } }; + +-- Insert INTO a container — append as its last child (works on an empty container) +ALTER PAGE Module.EditPage { + INSERT INTO ctnToolbar { + ACTIONBUTTON btnNew (Caption: 'New', Action: NOTHING, ButtonStyle: Primary) + } +}; ``` +`INSERT INTO ` appends widgets as the container's last children — the only way to fill an **empty** container, and handy for adding to a container or data view without a sibling to anchor to. Widgets inserted into a data view take that data view's entity. Supported on simple containers; for a layout grid or tab container, insert relative to a widget inside the target column/tab instead. + The inserted widgets use the same syntax as in `CREATE PAGE`. Multiple widgets can be inserted in a single block. ### DROP WIDGET -- Remove Widgets diff --git a/docs-site/src/language/snippets.md b/docs-site/src/language/snippets.md index 31407a0c6..04fb1c533 100644 --- a/docs-site/src/language/snippets.md +++ b/docs-site/src/language/snippets.md @@ -97,6 +97,9 @@ ALTER SNIPPET MyModule.CustomerCard { SET Caption = 'View Details' ON btnEdit; INSERT AFTER txtEmail { DYNAMICTEXT txtPhone (Content: '{1}', Attribute: Phone) + }; + INSERT INTO ctnActions { -- append as the container's last child + ACTIONBUTTON btnMore (Caption: 'More', Action: NOTHING) } }; ``` diff --git a/docs-site/src/reference/page/alter-page.md b/docs-site/src/reference/page/alter-page.md index 274be7f9d..a7dcae7a0 100644 --- a/docs-site/src/reference/page/alter-page.md +++ b/docs-site/src/reference/page/alter-page.md @@ -29,6 +29,9 @@ SET 'propertyName' = value ON widgetName; INSERT BEFORE widgetName { widget_definitions }; INSERT AFTER widgetName { widget_definitions }; +-- Insert widgets as the last children of a container +INSERT INTO containerName { widget_definitions }; + -- Remove widgets DROP WIDGET widgetName1, widgetName2; @@ -68,6 +71,12 @@ For pluggable widget properties, use quoted property names (e.g., `'showLabel'`) Inserts new widgets immediately before or after a named widget within its parent container. The new widgets use the same syntax as in `CREATE PAGE`. +### INSERT INTO + +Appends new widgets as the **last children** of a named container. This is the only way to fill an **empty** container, and the natural way to add a widget to a container or data view without needing a sibling to anchor to. Widgets inserted into a data view take that data view's entity as their context. + +Supported on simple containers (container/DivContainer, data view, group box, scroll-container region). A layout grid (rows/columns) and tab container have no single child list — insert relative to a widget inside the target column or tab instead. + ### DROP WIDGET Removes one or more widgets by name. The widget and all its children are removed from the tree. diff --git a/docs-site/src/tools/bootstrap-prompt.md b/docs-site/src/tools/bootstrap-prompt.md index 6c0331741..49e573c63 100644 --- a/docs-site/src/tools/bootstrap-prompt.md +++ b/docs-site/src/tools/bootstrap-prompt.md @@ -12,7 +12,7 @@ can seed the model from a design prototype in the same session — nothing to ma ## The prompt -```text +````text This is an empty repo. Provision it as a Mendix app developed with mxcli: 1. Ensure `mxcli` is available. It should be pre-installed by the environment; if @@ -43,7 +43,7 @@ This is an empty repo. Provision it as a Mendix app developed with mxcli: the app answers HTTP 200 at http://localhost:8080/ and report. (Optional) Seed the domain model, pages, and microflows from this prototype: . -``` +```` ## Which mxcli version gets installed diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index 5fa90a8cd..0f09a5190 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -982,6 +982,7 @@ Modify an existing page or snippet's widget tree in-place without full `create o | Widget dynamic classes | `set DynamicClasses = 'expr' on widgetName` | Runtime-computed classes on a widget — the surgical alternative to a bulk `update widgets` | | Insert after | `insert after widgetName { widgets }` | Add widgets after target | | Insert before | `insert before widgetName { widgets }` | Add widgets before target | +| Insert into | `insert into containerName { widgets }` | Append as the container's last child (fills an empty container; dataview children take its entity) | | Drop widgets | `drop widget name1, name2` | Remove widgets by name | | Replace widget | `replace widgetName with { widgets }` | Replace widget subtree | | Pluggable prop | `set 'showLabel' = false on cbStatus` | Quoted name for pluggable widgets | diff --git a/docs/05-mdl-specification/01-language-reference.md b/docs/05-mdl-specification/01-language-reference.md index 70023ae83..4f94a3ded 100644 --- a/docs/05-mdl-specification/01-language-reference.md +++ b/docs/05-mdl-specification/01-language-reference.md @@ -838,6 +838,7 @@ set title = 'New Title'; -- Insert widgets insert after widgetName { }; insert before widgetName { }; +insert into containerName { }; -- append as the container's last child -- Remove widgets drop widget name1, name2; diff --git a/docs/11-proposals/PROPOSAL_multi_version_pluggable_widgets.md b/docs/11-proposals/PROPOSAL_multi_version_pluggable_widgets.md index 1cebae95a..3bdec53c6 100644 --- a/docs/11-proposals/PROPOSAL_multi_version_pluggable_widgets.md +++ b/docs/11-proposals/PROPOSAL_multi_version_pluggable_widgets.md @@ -47,6 +47,23 @@ Two real, narrow gaps remain: builder applied `attribute: Country` on top without resetting them → an Object inconsistent with the schema → `CE0463`. The fix that worked (commit `827bffd4b`) was simply swapping in the legacy template's **clean/neutral** Object defaults. + **A second confirmed instance (commit `549c44f`): `image.json` carried a baked-in + static image `WidgetValue.Image = "Atlas_Core.Content.Mendix"` (Atlas's Mendix logo, + captured from a configured instance) where a fresh Image widget has `""`.** Any + Image authored on 11.7+ tripped `CE0463`; the documented remedy (`mx update-widgets`) + then destroyed MPRv2 storage (#763) — so a dirty default was silently funnelling + users into data loss. Isolated by the reusable method below and fixed by clearing + the value; verified `mx check --no-update-widgets` = 0 errors. The dirty-default + class is therefore broader than datasources: it includes **image-asset refs, page + refs, and configured client actions**, not just entity/attribute/flow bindings. + + **Reusable diagnosis method (this whole `CE0463` class).** Dump the widget BSON, + `mx update-widgets` on a **copy**, dump again, and diff the + `CustomWidgets$CustomWidget` subtree **order-independently** (canonicalise key order + + mask `$ID`/`TypePointer` blobs). Confirms the spike's tolerance result *per case*: + reordering and generic instance chrome (`LabelTemplate`, `Appearance.DesignProperties`) + are cosmetic — a *passing* widget gets those too; the real cause is whatever + value/structure survives that normalisation. For Image it was exactly one field. - **No cross-version guarantee.** Nothing tests that creation stays `CE0463`-free as Mendix minors and widget versions move; regressions surface in the field. @@ -154,8 +171,8 @@ changes. Applies to both engines. | File | Change | |------|--------| -| `sdk/widgets/templates/`, `modelsdk/widgets/templates/` | Audit every embedded template's Object for instance bindings; re-extract any dirty one from a fresh Studio-Pro widget (as done for ComboBox in `827bffd4b`) | -| `modelsdk/widgets/` (+ `sdk/widgets`) | New dirty-template guard: a test/`widget init` check that rejects Objects with populated `AttributeRef`/`DataSource`/`EntityRef` | +| `sdk/widgets/templates/`, `modelsdk/widgets/templates/` | Audit every embedded template's Object for instance bindings; re-extract any dirty one from a fresh Studio-Pro widget (ComboBox `827bffd4b`; **Image `549c44f` — done**) | +| `modelsdk/widgets/dirty_template_test.go` | **Built + broadened (this cycle).** Dirty-template guard (`dirtyBindings` + `TestEmbeddedTemplates_NoDirtyBindings`) rejects Objects with concrete entity/attribute/flow refs **and now image-asset / page / configured-action bindings** — the class the original guard missed, which is how the Image bug shipped. Scans **both** engines' template sets; proven non-vacuous (detect-dirt + clean-is-clean cases) and proven to catch the real Image regression. Follow-up: promote `dirtyBindings` from test-only to a runtime `widget init` check | | `modelsdk/widgets/loader.go`, `sdk/widgets/loader.go` | Consolidate so both engines share one augment path | | `sdk/widgets/augment.go` / `modelsdk/widgets/augment.go` | Reconcile to a single implementation | | `mdl/backend/widgetobj/builder.go` | Ensure the builder fully overrides every Object slot it sets (no bleed-through) | @@ -164,8 +181,13 @@ changes. Applies to both engines. ### Phasing -1. **Audit embedded templates for dirty Objects** + add the guard. ComboBox is - already clean (`827bffd4b`); confirm DataGrid2/Gallery/the rest. +1. **Audit embedded templates for dirty Objects** + add the guard. **In progress:** + ComboBox clean (`827bffd4b`); Image cleaned (`549c44f`); the guard is built and + broadened to the image/page/action class over both engines. Remaining: a broader + any-node scan surfaces a **configured delete `Forms$ActionButton`** + (`Forms$DeleteClientAction` + `Atlas_Core.Atlas.trash-can`) baked into + `datagrid.json` — nested-Forms-widget dirt, out of the WidgetValue-scoped guard's + safe reach (see Open Question 1). Confirm/clean it as part of the DataGrid2 pass. 2. Consolidate the two engines' widget loaders / augment onto one path. 3. Cross-version validation matrix. 4. (Optional) Evaluate whether a fresh-instance extraction could *generate* clean @@ -198,15 +220,287 @@ regenerate gen from the target version's reflection-data. ## Open Questions -1. **Per-widget-version templates for object-list widgets.** The cross-version - matrix (`scripts/widget-version-matrix.sh`) answered part of this concretely: - ComboBox and DataGrid2 reconcile across versions via augment (flat PropertyKeys), - but **Gallery `3.0.1`@10.24 produces CE0463 on both engines** — the 11.6-extracted - template's *nested object-list* sub-schema (items/content) doesn't match the - 10.24-installed widget, and augment's flat-key add/remove doesn't reach it. So - object-list widgets likely need a clean template **per widget-version**, not one - shared 11.6 base. Open: root-cause the Gallery nested diff, then decide - per-version template vs deeper (nested) augment. +1. **Within-key PropertyType drift on object-list widgets — root-caused.** Verified + directly against cached mxbuild (10.24.19 / 11.10.0 / 11.12.1): + - **DataGrid2 is clean with the *bundled* Data Widgets** (3.0.0@10.24, 3.4.0@11.10/ + 11.12) across attribute columns (the exact `#600` repro), custom content, filters, + and selection — all **0 CE0463**. The old-v0.11.0 mxcli-side bugs (NestedKeyOrder + `f12aba2`, custom-content `58508d4`) are resolved on HEAD. **But DataGrid2 drifts + exactly like Gallery when the installed Data Widgets is *updated from the + marketplace* past what the 11.6 template reconciles with.** `#600`'s latest report + (3 days ago: mxcli **v0.16.0**, Mendix **11.12.0**, "still present") is this case — + the original reporter noted **Data Widgets 3.10.0**, newer than the 3.4.0 bundled + with 11.12.0. Confirmed material facts: mxcli's widget code is **identical + v0.16.0→HEAD** (the only deltas are the Image fix + guard), so **no mxcli upgrade + fixes it** — it is a widget-version drift needing the deeper-augment fix below. + (An earlier note that DataGrid2 custom-content "still reproduces on 11.12.1" was a + **test-syntax artifact** — `Content: showContentAs` injected a spurious + TextTemplate — not a real defect; the real driver is the updated `.mpk`.) + - **Gallery custom content produces CE0463 on 10.24** (clean on 11.10 / 11.12). + The before/after-`update-widgets` subtree diff shows the cause is **NOT** a dirty + default and **NOT** merely nested: the 11.6 template and the 10.24-installed + Gallery share the same PropertyType *keys* but differ in their *definitions* — + `pagingPosition` enum values changed (`bottom`/`top` → `below`/`above`, default + `bottom`→`below`), a `Category` was renamed (`General::Pagination` → + `General::Items`), and a property's `Type`/order shifted. `augmentFromMPK` + reconciles key *presence* (add/remove) but **not within-key definition changes**, + so the emitted Type ≠ the installed widget → CE0463. + + **This is the general object-list drift, not a Gallery quirk** — DataGrid2 with a + marketplace-updated Data Widgets (3.10.0, the #600 case) drifts the same way. + Root-caused via the key-indexed PropertyType diff into **two independent axes:** + + - **Axis 1 — within-key PropertyType metadata drift. FIXED (`8b65f06`).** + `augment` reconciled key presence and enum option sets but left the rest of a + matched PropertyType stale: a `DefaultValue` outside the reconciled enum set + (Gallery `pagingPosition` options → `{below,above}` but default stayed `bottom`), + a `Category` rename (`General::Pagination`→`General::Items`), a `Caption` change. + `reconcilePropertyMetadata` now overwrites Category/Caption/DefaultValue from the + `.mpk` for every matched key. Verified: after it, **every Gallery@10.24 + PropertyType matches `update-widgets`**; no regression on 11.12. (An earlier draft + claimed this was "the likely complete fix for #600" — **disproven**; see the + large-drift measurement below, which shows the #600 DataGrid2 case needs more.) + *(modelsdk engine; the legacy sdk `augment` lacks even `reconcileEnumValues` and is + behind — consolidate per item 3 above.)* + - **Axis 2 — datasource structure + PropertyType order. FIXED (`10b5bcf`).** + Gallery@10.24 *additionally* drifted two ways: (a) mxcli emitted `Forms$GridSortBar` + without `SortItems` and `CustomWidgets$CustomWidgetXPathSource` without + `SourceVariable` (the 10.24 widget expects both, empty/null) — fixed with codec + `TypeDefaults` in `widget_write.go` (`MandatoryListMarkers: SortItems=2`, + `NullFields: SourceVariable`); and (b) the top-level PropertyType **order** differed + — the 3.x widget moved `pagingPosition` ahead of `showTotalCount`, and augment kept + the template order. **The WidgetType's PropertyType order IS checked by CE0463** + (unlike the WidgetObject's Properties order, which the spike proved tolerated); + `reorderPropertyTypes` now sorts the Type's PropertyTypes to the `.mpk` declaration + order (refs are by `$ID`, so it's safe). + + **Result (moderate drift): Gallery@10.24 passes `mx check` with 0 errors and NO + `update-widgets`.** Regression clean — DataGrid2 (attribute + filter + custom-content) + and Gallery both 0 errors on 10.24 / 11.10 / 11.12.0 / 11.12.1 with the *bundled* + Data Widgets. *(modelsdk engine; the legacy sdk `augment` remains behind — item 3.)* + + **The three axes are necessary but NOT sufficient for large version jumps — measured + against the #600 reporter's exact stack.** Downloaded Data Widgets **3.10.0** from the + marketplace, swapped it into a Mendix 11.12.0 project (over the bundled 3.4.0), and + authored a DataGrid2 with the fixed binary: **still CE0463.** Isolating that one grid + from the Atlas template pages (which legitimately drift and are a separate "update + widgets" concern), the installed 3.10.0 schema differs from the 11.6-era template in + **far more than the three axes reconcile** — an aggregated field diff over the + `CustomWidgetType`: + + | Differing `ValueType` field | # properties | + |---|---| + | `AllowUpload` (null vs false) | 79 | + | `Type` (property type changed) | 15 | + | `Translations` | 14 | + | `DefaultValue` | 13 | + | `Required` | 7 | + | `EnumerationValues` | 7 | + | `AllowedTypes` | 4 | + | + `Category`, `Description`, `ReturnType`, nested `ObjectType.PropertyTypes` | | + + So **augment's key-presence + metadata/enum/order reconciliation scales to a *moderate* + drift (Gallery@10.24: Category + one DefaultValue + order + datasource) but not to a + *large* one (DataGrid2 11.6-era → 3.10.0).** This directly disproves the earlier + working assumption that Axis 1 alone would close #600. The Phase-1 measurement that + "augment already delivers version-correct schemas (56=56 for ComboBox 2.4.3)" held only + because that was a *small* version delta; a big jump exposes per-field schema evolution + (`Type`, `Required`, `AllowedTypes`, `Translations`) that key-level augment never touches. + + **RESOLVED (definition side) — the `WidgetType` IS generically reproducible; an + earlier "only mxbuild can produce the envelope" conclusion was WRONG.** `update-widgets` + is fully generic (no per-widget knowledge), so everything it emits is derivable from the + package + the generic metamodel/spec defaults — the "not in `Datagrid.xml`" fields are + generic defaults, not widget-specific computation: + - `AllowUpload` (on **all 105** ValueTypes) is a generic `WidgetValueType` metamodel + default (`false`) — emitted for every ValueType, widget-agnostic. + - `Required` "computed 3→54" is just the **pluggable-widget spec default**: `required` + defaults to **true** when the attribute is absent. The `.mpk` parser had the wrong + default (missing→false); fixed. + - `Type`/`DefaultValue` drift was augment cloning new keys from **wrong-typed exemplars** + (our bug), plus stale template values — fixed by reconciling each matched ValueType's + `Type`/`Required`/`DefaultValue`/`AllowedTypes`/`IsList`/`ReturnType`/`Translations` + from the `.mpk` and normalizing the mutually-exclusive type-specific fields. + - `Translations`/`ReturnType`/nested `Category`/nested order — all parsed from the + widget XML (``, ``, propertyGroup captions) and emitted. + + **Committed (`c7fe714`): the emitted `CustomWidgetType` is byte-identical (canonical, + `$ID`/`TypePointer`-masked) to `update-widgets` output for the #600 stack (DW 3.10.0 / + Mendix 11.12.0) — definition diff 326 → 0 — with no regression on bundled DW across + 11.12.0 / 11.10.0 / 10.24.** The generic-reconciliation thesis holds for the definition. + + **NOT resolved (instance side) — the last-mile Object default-template instantiation is + config-conditional applicability that lives in the widget's editor code, not the + declarative package.** With the definition matching, the residual CE0463 is entirely in + the `WidgetObject`: a handful of `textTemplate` properties whose default template + (`Forms$ClientTemplate` with the shipped caption translations) mxbuild instantiates in + the instance. Even a **minimal** DataGrid2 (Selection:None, one column) reproduces it — + definition identical, only ~9 `textTemplate` default-templates differ, `update-widgets` → + 0. **Which textTemplates get a default template is config-dependent**: aria/status labels + are always instantiated, but `clearSelectionButtonLabel` / `loadMoreButtonCaption` / + `singleSelectionColumnLabel` stay `null` because their feature is off. Empirically: an + "always-populate" rule closes 9→3 but over-populates the 3 feature-gated ones (still + CE0463); a `Required`-gate closes 3→9 the other way. Neither declarative rule matches, + because the rule isn't declarative. + + **CONFIRMED mechanism — the widget's compiled `editorConfig.js` decides applicability.** + The `.mpk` ships `Datagrid.editorConfig.js` (16 KB) alongside `Datagrid.xml`. Its + `getProperties(values, defaultProperties)` function calls `hidePropertyIn` (24×) / + `hidePropertiesIn` (6×) / `changePropertyIn` conditionally on the *instance's* current + values. The exact hides for the three null properties are present verbatim: + + ```js + "Multi" !== r && hidePropertiesIn(e, t, ["selectionCounterPosition","clearSelectionButtonLabel","enableSelectAll"]) + "loadMore" !== e.pagination && hidePropertyIn(t, e, "loadMoreButtonCaption") + hidePropertyIn(e, t, "singleSelectionColumnLabel") // conditional + ``` + + This maps 1:1 onto the measurements: our minimal grid has `Selection:None` (≠ "Multi") → + `clearSelectionButtonLabel` hidden → `null`; `Pagination:buttons` (≠ "loadMore") → + `loadMoreButtonCaption` hidden → `null`. The always-populated properties + (`selectRowLabel`, `cancelExportLabel`) appear **0×** in `editorConfig.js` — never hidden, + so their default is instantiated. **A hidden property does not get its default template + instantiated in the Object.** The applicability graph is imperative JS keyed on the + instance config, not declarative XML — which is why no `.mpk` parsing reproduces it, and + why the null-vs-populated property *definitions* in `Datagrid.xml` are byte-identical. + + **Object-from-definition rebuild — tried, REJECTED (regresses the common case).** + Regenerating the whole `WidgetObject` from the reconciled definition (one WidgetValue per + PropertyType, defaults built from each ValueType) makes the count match (127=127) and is + architecturally clean, but it **discards the byte-exact extracted template Object** — and + because it can't replicate the editor-applicability rule, it **regressed bundled DataGrid2 + from 0 → 1 CE0463 on both 11.12.0 and 10.24**. The extracted template Object is the best + available source for the no-drift case; a rebuild trades that away for an approximation. + Reverted. + + **RESOLVED (instance side) — a static editorConfig extractor closes it, natively in + mxcli. No mxbuild / update-widgets dependency needed for DataGrid2.** The codebase + already had the scaffold — `WidgetVisibilityRule` + `Builder.ApplyPropertyVisibility` + (nulls a hidden textTemplate's ClientTemplate), fed by a *hand-transcribed* + `widgetVisibilityRules` table (VideoPlayer/Timeline only; the comment: "Until the JS + extractor lands (#574 Phase 2)"). The three landed commits automate that table and wire + it through: + + - **Extractor** (`4b8c4f5`, `mdl/executor/editorconfig_extract.go`) — a static analyzer + lifts the dominant `getProperties` idioms from the compiled `editorConfig.js` + (`"V"===/!==ref && hide`, `ref && hide`, `ref || hide`, `ref ? hide : …`) into + `WidgetVisibilityRule`s, with **scoped** alias resolution (`var r=e.itemSelection`) so + minified single-letter identifiers don't leak across functions, and a boundary check + that **skips compound/ternary-nested and object-list-nested guards** rather than emit a + partial (wrong) rule — degrading safely to "not hidden". Coverage on the real DW 3.10.0 + `Datagrid.editorConfig.js`: **9/28 hide-calls lifted** (12 object-list-nested, 7 + compound — safely skipped, honestly counted), including the three that drive #600. + - **Wiring** — built-in widgets (DataGrid2/Gallery), which the `.def.json` generator + skips, resolve rules on the fly from the project's installed `.mpk` editorConfig at + build time (`resolveWidgetVisibilityRules`); `ApplyPropertyVisibility` then nulls the + hidden textTemplates. Object-side textTemplate defaults are populated with the `.mpk`'s + shipped caption translations (fixes CE4899 required-textTemplate) and the visibility + pass nulls the hidden ones afterward. + - **Selection-value fix** (`666a65b`) — conditions keyed on a Selection-typed property + (`itemSelection` = None/Single/Multi) must read the WidgetValue's `Selection` field, not + `PrimitiveValue`; reading the wrong field mis-fired under `Selection:Single`. + + **Result: DataGrid2 on Mendix 11.12.0 + Data Widgets 3.10.0 (the exact #600 stack) → + 0 errors, minimal AND full (Selection:Single + column textfilters).** No regression on + bundled DW across 11.12.0 / 11.10.0 / 10.24. + + **Phase 2 landed** (`458c52a`): extraction moved into `.def.json` **generation** + (generated defs now carry `propertyVisibility`, superseding the hand table), and the rules + are wired into **`check`** as **MDL-WIDGET10** — a config-aware warning when the user sets + a property the widget hides under the current config (e.g. `ClearSelectionButtonLabel` + with `Selection:None`). Conservative: only warns when the property is explicitly set and + the condition value is determinable. + + **Phase 3 landed** (`da30bab`) — all 9 Data Widgets, plus the filter build path. + The DW 3.10.0 package ships **9 widgets**, all with an `editorConfig.js`: DataGrid2, + Gallery, DropdownSort, SelectionHelper, TreeNode (the outline tree), and the 4 filters + (Date/Dropdown/Number/Text). The extractor recognized DataGrid only; the rest used guard + forms it skipped, so three generalisations were needed — strip **any** `.` + namespace (the minifier names it `D.`/`M.`/`j.`/`A.` per widget, not just `_.`), strip a + leading `return` (getProperties' first statement is `return && hide…`), and handle + grouping parens `cond && ( hide, … )`. Coverage after (was 0 for everything but DataGrid): + + | Widget | lifted | Widget | lifted | + |---|---|---|---| + | DataGrid2 | 9/28 | DropdownFilter | 5/13 | + | Gallery | 5/13 | DateFilter | 3/5 | + | TreeNode (outline tree) | 4/7 | Number/TextFilter | 2–3 | + | SelectionHelper | 1/2 | DropdownSort | 0/0 (no hides) | + + Remaining skips are grouped-subsequent and compound guards — safely skipped, never + mis-lifted (DataGrid matrix + bundled stay 0). The filter **build path** was separate + (`BuildFilterWidget` never applied visibility → a filter's hidden textTemplate stayed + populated → CE0463); the nulling is now factored into engine-agnostic + `widgetobj.ApplyVisibilityRules`, threaded through `FilterWidgetSpec`, and resolved by the + executor. **Text/Number/Date filters → 0.** + + **RESOLVED — ComboBox definition drift (`#112`), two generic causes (`bf6c577`).** The + earlier "~3351-line ComboBox drift / large version jump" framing was an artifact of an + **ID-unmasked** diff. Re-measured with `$ID`/pointers masked, a ComboBox on a fresh 11.12.1 + project (which ships ComboBox **2.5.0 — the same version as the embedded template**, so no + version jump) drifted only **243 definition lines**, from two generic causes augment didn't + yet cover: + 1. **System-property order.** ComboBox declares `` Label/Visibility/ + Editability **inline mid-list** (its Editability group even mixes the systemProperty + ahead of a regular property). mxbuild emits the WidgetType's PropertyTypes in that + declared order and CE0463 checks it, but the `.mpk` parser split system properties into a + separate list and `reorderPropertyTypes` pushed them to the **end**. DataGrid2/Gallery + declare **no** inline system properties, so they were unaffected — precisely why they + passed and ComboBox didn't. Fixed by preserving the `.mpk`'s full declared order (regular + + system interleaved) in `mpk.WidgetDefinition.AllTopLevel` (built via a document-ordered + custom `UnmarshalXML` on `propertyGroup`) and ranking `reorderPropertyTypes` by it → + 243 → 9 lines. + 2. **``.** ComboBox's `staticDataSourceValue` + expression derives its return type from another property; the parser read only + `` and emitted a null `ReturnType` where mxbuild emits + `WidgetReturnType{Type:None, AssignableTo}`. Fixed by parsing/emitting `assignableTo` → + 9 → **0**. + + After both (generic, no ComboBox-specific code) the emitted `CustomWidgetType` is + **byte-identical (ID-masked) to `mx update-widgets`** and `mx check` reports **0 errors** — + no `update-widgets` needed. The residual 210-line **Object** diff is pure WidgetValue + *ordering*, which Studio Pro tolerates (0 errors confirms it). + + **Also RESOLVED on bundled DW — DropdownFilter (`ddf`).** The prior ~1529-line `ddf` + definition drift was measured against a *marketplace-swapped* Data Widgets 3.10.0. On the + **bundled** DW in 11.12.1 a DataGrid2 with a `dropdownfilter` now reports **0 CE0463** — the + system-property-order fix closes it there too. The 3.10.0-marketplace nested filter-options + `ObjectType` axis remains the only open definition-reconciliation follow-up (needs a project + with that exact package to re-measure). TreeNode's + remaining 3/7 visibility skips are its "dynamic structure" guards (`transformGroupsIntoTabs`, + nested `advancedMode`/`headerType` ternaries, `e.hasChildren||hide([…children…])`) — the ones + the latest DW release expanded; a JS AST would be needed to lift them. + + **Superseded options** (kept for the record): (c) v2-safe `update-widgets` via #764 — no + longer needed for DataGrid2 (mxcli now closes it natively); still the fallback for widgets + whose editorConfig the static extractor can't fully lift. (d) in-process `goja` execution + — moot: static lifting suffices for the common cases *and* yields declarative rules that + `check`/lint/an LLM can consume, which JS execution wouldn't. (b) per-version templates — + rejected. + + **Verified against `mdl-examples/` widget scripts.** `make check-mdl` is green for every + widget script (the lone unrelated FAIL is `view-entity-derived-string-length.mdl`, + MDL031). Running the pluggable scripts through `mx check` on fresh 11.12.1 projects: + **all 9 report 0 CE0463** — `pluggable-smoke`, `189-datagrid2-column-textfilter`, + `628-datagrid2-selection`, `datagrid2-custom-content-column`, + `datagrid-numberfilter-array-marker`, `bug8-datagrid-gallery-sort-desc`, + `datagrid-columnwidth-manual-size`, `datagrid2-associated-attribute-column`, and + `112-combobox-enum-ce0463-widget-version` (now closed by `bf6c577`). + + Remaining (follow-ups, not blockers): (1) definition-drift reconciliation for the one + remaining axis — the Dropdown filter's nested object-list `ObjectType` structure against a + *marketplace-swapped* Data Widgets 3.10.0 (ComboBox's drift is now closed by `bf6c577`, and + `ddf` is clean on bundled DW); (2) a real JS AST to lift the compound/ternary and + object-list-nested guards (incl. + TreeNode's dynamic structures) the regex extractor skips; (3) the same rules could feed + LLM "property cards"; (4) fold the matrix into `scripts/widget-version-matrix.sh` as a + standing gate (feasible now — `mxcli marketplace download` fetches any widget version) + asserting the DW widgets across versions stay at 0. + + **Also noted (dirty-template audit):** `datagrid.json`'s Object carries a configured + delete `Forms$ActionButton` (`Forms$DeleteClientAction` + `Atlas_Core.Atlas.trash-can`) + — a separate nested-Forms dirty default the flat WidgetValue-scoped guard deliberately + does not flag; benign on the tested versions but worth cleaning during the object-list pass. 2. **Long-tail marketplace widgets.** Built-ins can be hand-extracted clean. For arbitrary marketplace widgets with no embedded template, the only correct Object source is a fresh extraction — can `widget init` extract from a freshly-dropped diff --git a/go.mod b/go.mod index 649da2947..c7ca28503 100644 --- a/go.mod +++ b/go.mod @@ -27,7 +27,7 @@ require ( go.mongodb.org/mongo-driver/v2 v2.6.0 go.starlark.net v0.0.0-20260102030733-3fee463870c9 go.uber.org/zap v1.28.0 - golang.org/x/sync v0.20.0 + golang.org/x/sync v0.21.0 golang.org/x/term v0.43.0 gopkg.in/yaml.v3 v3.0.1 modernc.org/sqlite v1.51.0 @@ -72,7 +72,7 @@ require ( golang.org/x/crypto v0.52.0 // indirect golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect golang.org/x/sys v0.45.0 // indirect - golang.org/x/text v0.37.0 // indirect + golang.org/x/text v0.39.0 // indirect modernc.org/libc v1.72.3 // indirect modernc.org/mathutil v1.7.1 // indirect modernc.org/memory v1.11.0 // indirect diff --git a/go.sum b/go.sum index 71dc5927e..cd91f4f4a 100644 --- a/go.sum +++ b/go.sum @@ -174,37 +174,28 @@ go.uber.org/zap v1.28.0 h1:IZzaP1Fv73/T/pBMLk4VutPl36uNC+OSUh3JLG3FIjo= go.uber.org/zap v1.28.0/go.mod h1:rDLpOi171uODNm/mxFcuYWxDsqWSAVkFdX4XojSKg/Q= go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= -golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI= -golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q= golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988= golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc= golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 h1:mgKeJMpvi0yx/sU5GsxQ7p6s2wtOnGAHZWCHUM4KGzY= golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546/go.mod h1:j/pmGrbnkbPtQfxEe5D0VQhZC6qKbfKifgD0oM7sR70= -golang.org/x/mod v0.34.0 h1:xIHgNUUnW6sYkcM5Jleh05DvLOtwc6RitGHbDk4akRI= -golang.org/x/mod v0.34.0/go.mod h1:ykgH52iCZe79kzLLMhyCUzhMci+nQj+0XkbXpNYtVjY= -golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM= -golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA= -golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs= +golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ= +golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0= golang.org/x/net v0.54.0 h1:2zJIZAxAHV/OHCDTCOHAYehQzLfSXuf/5SoL/Dv6w/w= -golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= -golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/net v0.54.0/go.mod h1:Sj4oj8jK6XmHpBZU/zWHw3BV3abl4Kvi+Ut7cQcY+cQ= +golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM= +golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211110154304-99a53858aa08/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ= -golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY= golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4= golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk= -golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg= -golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164= -golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc= -golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38= -golang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s= -golang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0= -golang.org/x/tools v0.44.0 h1:UP4ajHPIcuMjT1GqzDWRlalUEoY+uzoZKnhOjbIPD2c= +golang.org/x/text v0.39.0 h1:UbZz4pLOvn600D6Oh6GGEI6VAmndrEBLv8/6BEXzyus= +golang.org/x/text v0.39.0/go.mod h1:3UwRclnC2g0TU9x8PZiyfOajCd1zaUNHF9cvqcQZ+ZM= +golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q= +golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI= google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos= diff --git a/mdl-examples/bug-tests/112-combobox-enum-ce0463-widget-version.mdl b/mdl-examples/bug-tests/112-combobox-enum-ce0463-widget-version.mdl index 66ec7be26..6bee3a2db 100644 --- a/mdl-examples/bug-tests/112-combobox-enum-ce0463-widget-version.mdl +++ b/mdl-examples/bug-tests/112-combobox-enum-ce0463-widget-version.mdl @@ -8,34 +8,42 @@ -- [CE0463] "The definition of this widget has changed. Update this widget -- by right-clicking it and selecting 'Update widget'…" at Combo box 'cbClass'. -- --- Root cause (refined): +-- Root cause (RESOLVED): -- #112 originally blamed an incomplete template (58 PropertyTypes / 52 --- WidgetProperties). That is fixed — a combobox emitted against a project whose --- installed com.mendix.widget.web.Combobox.mpk matches mxcli's embedded template --- (Mendix 11.6 / widget 2.5.0) now passes `mx check` with 0 errors. +-- WidgetProperties). That was fixed first. -- --- The RESIDUAL CE0463 is a widget-VERSION mismatch: mxcli emits the embedded --- 2.5.0-shaped PropertyTypes, but a project may have a NEWER combobox (e.g. --- 2.8.1) whose schema reorders / regroups properties. `augmentFromMPK` patches --- presence + enum values from the project .mpk but cannot restructure the --- 2.5.0 baseline into the newer shape, and full generation-from-mpk is less --- faithful still (fails even on a matching version). +-- The residual CE0463 was NOT a version jump — a fresh 11.12.x project ships +-- ComboBox 2.5.0, the same version as mxcli's embedded template — but two +-- GENERIC definition-drift causes the augment reconciliation didn't yet cover: -- --- The DESIGNED remediation is `mx update-widgets`, which migrates the stored --- widget to the installed version and clears CE0463. `mxcli docker build/check` --- run it automatically — but it was crashing on a bare .mpr path --- (AddProjectDirAsAllowedPath → ArgumentNullException), so the migration never --- ran. That path-arg crash is fixed (docker: pass an absolute path). +-- 1. System-property ORDER. ComboBox declares Label / +-- Visibility / Editability INLINE in the middle of its property list (its +-- Editability group even mixes the systemProperty ahead of a regular +-- property). mxbuild emits the WidgetType's PropertyTypes in that declared +-- order and CE0463 checks it, but the .mpk parser split system properties +-- into a separate list and reorderPropertyTypes pushed them to the END. +-- (DataGrid2 / Gallery declare no inline system properties, so they were +-- unaffected — which is why they passed and ComboBox didn't.) Fixed by +-- preserving the .mpk's full declared order (regular + system interleaved, +-- mpk.WidgetDefinition.AllTopLevel) and reordering to it. +-- 2. . ComboBox's +-- staticDataSourceValue expression derives its return type from another +-- property; the parser read only and emitted a null +-- ReturnType where mxbuild emits a WidgetReturnType{Type:None, AssignableTo}. +-- Fixed by parsing assignableTo and emitting the return type. -- --- Verification: --- - On a project whose combobox.mpk matches the template version, this script --- passes `mx check` with 0 errors (the incomplete-template regression is gone). --- - On a project with a newer combobox, `mxcli docker check -p app.mpr` now runs --- update-widgets successfully and `mx check` reports 0 errors. +-- Both fixes are generic (no ComboBox-specific code) and live in +-- modelsdk/widgets/{mpk/mpk.go,augment.go}. After them the emitted +-- CustomWidgetType is byte-identical (ID-masked) to `mx update-widgets` output. +-- +-- Verification (Mendix 11.12.1, bundled ComboBox 2.5.0): +-- - `mxcli exec … -p app.mpr` then `mx check` → 0 errors (was 1 CE0463 on cbClass). +-- - Definition (WidgetType) diff vs `mx update-widgets` → 0 lines (ID-masked). +-- - No regression: DataGrid2 / Gallery / filters (incl. DropdownFilter) stay at 0. -- -- Usage: -- mxcli exec mdl-examples/bug-tests/112-combobox-enum-ce0463-widget-version.mdl -p app.mpr --- mxcli docker check -p app.mpr # runs update-widgets (now crash-free), then mx check +-- mx check app.mpr # expect 0 errors, no update-widgets needed -- ============================================================================ create module CBBug112; diff --git a/mdl-examples/bug-tests/189-datagrid2-column-textfilter.mdl b/mdl-examples/bug-tests/189-datagrid2-column-textfilter.mdl index 40196db41..8d6edd465 100644 --- a/mdl-examples/bug-tests/189-datagrid2-column-textfilter.mdl +++ b/mdl-examples/bug-tests/189-datagrid2-column-textfilter.mdl @@ -3,10 +3,14 @@ -- 1. "Upgrade Widgets" prompt in Studio Pro (wrong widget structure) -- 2. Duplicated filter search (filter appeared in content AND filter row) -- --- Fix: in buildDataGridV3, filter-type children (TEXTFILTER, NUMBERFILTER, --- DATEFILTER, DROPDOWNFILTER) of a COLUMN are routed through BuildFilterWidget --- and stored in DataGridColumnSpec.FilterWidget, which is then serialized --- into the column's "filter" property slot in both BSON construction paths. +-- Fix: filter-type children (TEXTFILTER, NUMBERFILTER, DATEFILTER, +-- DROPDOWNFILTER) of a COLUMN are routed into the column's "filter" slot rather +-- than its custom-content slot. On the page-build path this is the generic +-- pluggable engine matching the filter keyword against the column's filter +-- item-slot (ItemSlots AcceptedChildTypes); on the ALTER PAGE column +-- insert/replace path it is buildColumnSpecFromAST → BuildFilterWidget stored in +-- DataGridColumnSpec.FilterWidget. Both serialize the filter into the column's +-- "filter" property slot. CREATE ENTITY Bug189.MyEntity ( Attribute: String, Attribute_2: String ); diff --git a/mdl-examples/bug-tests/chart-series-customseriesoptions-json.mdl b/mdl-examples/bug-tests/chart-series-customseriesoptions-json.mdl new file mode 100644 index 000000000..c8d613eef --- /dev/null +++ b/mdl-examples/bug-tests/chart-series-customseriesoptions-json.mdl @@ -0,0 +1,41 @@ +-- ============================================================================ +-- Bug: chart series customSeriesOptions emitted as " " → runtime JSON.parse error +-- ============================================================================ +-- +-- Symptom: +-- A ColumnChart (or any chart with a `series`/`line` object-list) passed +-- `mx check` (0 errors) but failed to RENDER at runtime with a client-side +-- `JSON.parse('Unexpected end of JSON input')` in the chart render path. A Pie +-- chart over the same data (widget-level datasource, no series object-list) +-- rendered fine. +-- +-- Root cause: +-- The object-list-item builder emitted an unset String property as a single +-- space " " (not ""). The chart client feeds customSeriesOptions / +-- customLayout / customConfigurations straight to JSON.parse with an empty-guard +-- `value !== "" ? value : "{}"`. A lone space passes that guard, so the client +-- runs `JSON.parse(" ")` and throws. (mx check does not execute the client, so +-- it stayed green.) The top-level string path already emitted "". +-- +-- Fix: +-- createDefaultWidgetValue (object-list-item path) leaves an unset String empty, +-- matching Studio Pro and the top-level path, so the client's empty-guard turns +-- it into "{}". +-- +-- Verify: +-- mxcli exec mdl-examples/bug-tests/chart-series-customseriesoptions-json.mdl -p app.mpr +-- # the series' customSeriesOptions is now "" (was " "); the ColumnChart renders. +-- mx check app.mpr # still 0 errors +-- Requires: Charts .mpk installed in the project's widgets/. +-- ============================================================================ + +create module ColJSON; +/ +create entity ColJSON.Sales ( Region: String, Total: Decimal ); +/ +create or replace page ColJSON.Home ( layout: Atlas_Core.Atlas_Default ) { + pluggablewidget 'com.mendix.widget.web.columnchart.ColumnChart' col1 { + series s ( dataSet: static, staticDataSource: database from ColJSON.Sales, + staticXAttribute: Region, staticYAttribute: Total, staticName: 'Revenue' ) + } +} diff --git a/mdl-examples/bug-tests/chart-widget-ce0463-v2.mdl b/mdl-examples/bug-tests/chart-widget-ce0463-v2.mdl new file mode 100644 index 000000000..e81065bb3 --- /dev/null +++ b/mdl-examples/bug-tests/chart-widget-ce0463-v2.mdl @@ -0,0 +1,56 @@ +-- ============================================================================ +-- Bug: Chart widgets (Pie / Column / …) trip CE0463 in MPR v2 projects +-- ============================================================================ +-- +-- Symptom: +-- A PieChart or ColumnChart authored via MDL persisted its series properties +-- correctly but every instance failed `mx check` with: +-- [CE0463] "The definition of this widget has changed. Update this widget…" +-- The only thing that cleared it was `mx update-widgets`, which downgrades an +-- MPR v2 project to v1 (drops mprcontents/). So charts could not ship in v2. +-- By contrast the Image widget (which has an embedded template) authored cleanly. +-- +-- Root cause: +-- Charts have no embedded template, so they are built by GenerateFromMPK. That +-- path (a) omitted the widget's declared entries (Name, +-- TabIndex, Visibility) that mxbuild emits into the WidgetType; (b) left each +-- ValueType's schema-derived fields (enum option sets, expression returnType, +-- selectionTypes, multiline) empty; and (c) parsed helpUrl/studioCategory as XML +-- attributes when Mendix declares them as elements. All three are checked by +-- CE0463, so the generated WidgetType drifted from the installed widget. +-- +-- Fix: +-- GenerateFromMPK now emits system PropertyTypes in their declared position and +-- runs the same .mpk reconciliation augment uses on template-based widgets; the +-- .mpk parser reads helpUrl/studioCategory as elements and parses multiline + +-- . Result: the emitted WidgetType is byte-identical to +-- update-widgets output — no CE0463, MPR v2 preserved. +-- +-- Verify: +-- mxcli exec mdl-examples/bug-tests/chart-widget-ce0463-v2.mdl -p app.mpr +-- mx check app.mpr # 0 errors (v2 preserved) +-- Requires: Charts .mpk installed in the project's widgets/. +-- ============================================================================ + +create module ChartBug; +/ +create entity ChartBug.Sales ( Region: String, Total: Decimal ); +/ +create or replace page ChartBug.Home ( layout: Atlas_Core.Atlas_Default ) { + -- Pie chart: single implicit series (widget-level DataSource/ValueAttribute). + pluggablewidget 'com.mendix.widget.web.piechart.PieChart' pie1 ( + DataSource: database from ChartBug.Sales, + ValueAttribute: Total, + SeriesName: 'Sales by Region' + ) + -- Column chart: an explicit `series` object-list. + pluggablewidget 'com.mendix.widget.web.columnchart.ColumnChart' col1 { + series s ( + DataSet: 'static', + DataSource: database from ChartBug.Sales, + StaticXAttribute: Region, + StaticYAttribute: Total, + StaticName: 'Revenue' + ) + } +} diff --git a/mdl-examples/bug-tests/image-ce0463-stale-default.mdl b/mdl-examples/bug-tests/image-ce0463-stale-default.mdl new file mode 100644 index 000000000..5829a1f1a --- /dev/null +++ b/mdl-examples/bug-tests/image-ce0463-stale-default.mdl @@ -0,0 +1,55 @@ +-- ============================================================================ +-- Image widget CE0463 "widget definition changed" from a stale template default +-- ============================================================================ +-- +-- Symptom (before fix): authoring an Image widget on any Mendix 11.7+ project +-- and running `mx check` (without `mx update-widgets`) reports: +-- +-- [CE0463] "The definition of this widget has changed. Update this widget..." +-- at Image 'img' +-- +-- The documented remedy was `mx update-widgets`, which on an MPRv2 project +-- rewrites storage to v1 and deletes mprcontents/ (issue #763) — so this stale +-- default quietly funnelled users into data loss. +-- +-- Root cause: NOT a version stamp, Type $IDs, property order, or missing +-- properties (all ruled out empirically). The embedded template +-- `image.json` carried a spurious default value — a `WidgetValue.Image` +-- pointing at `Atlas_Core.Content.Mendix` (Atlas's Mendix logo) — captured +-- when the template was extracted from a project that happened to have it set. +-- The installed 11.12 Image widget expects that field empty (""), so MxBuild +-- flagged the definition as changed. Clearing the default to "" makes the +-- widget pass with 0 errors and no update-widgets. +-- +-- Diagnosis method (reusable for this whole CE0463 class): dump the widget +-- BSON, run `mx update-widgets` on a COPY, dump again, and diff the +-- CustomWidget subtree ORDER-INDEPENDENTLY (canonicalise key order + mask +-- $ID/TypePointer blobs). Reordering and generic instance chrome +-- (LabelTemplate, Appearance/DesignProperties) are cosmetic — a passing +-- widget gets those too. The real cause is whatever value/structure survives +-- that normalisation. +-- +-- Fix: modelsdk/widgets/templates/mendix-11.6/image.json (+ sdk/widgets/...): +-- "Image": "Atlas_Core.Content.Mendix" -> "Image": "" +-- +-- Verify: +-- mxcli exec image-ce0463-stale-default.mdl -p <11.12 app.mpr> +-- mxcli docker check -p --no-update-widgets -> 0 errors +-- (before the fix: 1 error, CE0463 at Image 'img') +-- ============================================================================ + +create page MyFirstModule.ImgPage ( + Title: 'Image CE0463 repro', + Layout: Atlas_Core.Atlas_Default +) { + layoutgrid lg { + row r { + column c (DesktopWidth: 12) { + pluggablewidget 'com.mendix.widget.web.image.Image' img ( + ImageType: 'imageUrl', + imageUrl: 'https://example.com/x.png' + ) + } + } + } +} diff --git a/mdl-examples/bug-tests/pluggable-child-slot-container.mdl b/mdl-examples/bug-tests/pluggable-child-slot-container.mdl new file mode 100644 index 000000000..4de5d4709 --- /dev/null +++ b/mdl-examples/bug-tests/pluggable-child-slot-container.mdl @@ -0,0 +1,50 @@ +-- ============================================================================ +-- Bug: pluggable widget child-slot dropped when authored as `container ` +-- ============================================================================ +-- +-- Symptom: +-- A TreeNode authored with `container children { ... }` (or Timeline +-- `container customTitle { ... }`) persisted with NO child content — the +-- nested widgets were silently discarded. describe page showed only the +-- scalar props; the slot was empty. +-- +-- Root cause: +-- applyChildSlots matched a child block to a slot by the child's Type +-- (lowercased) against the slot keyword. The natural authoring form +-- `container children { ... }` has Type "container" and Name "children", so it +-- never matched the "children" slot and fell through to the discarded default +-- widgets. Only the bare-keyword form (`children { ... }`) matched. +-- +-- Fix: +-- Route a `container { ... }` to the slot whose keyword matches the +-- container's NAME (mirroring the object-list item-slot matcher). The +-- bare-keyword form still matches by Type. +-- +-- Verify: +-- mxcli exec mdl-examples/bug-tests/pluggable-child-slot-container.mdl -p app.mpr +-- mx check app.mpr # 0 errors; the treeChildText dynamictext persists in the +-- # TreeNode `children` slot (MPR v2 preserved) +-- Requires: TreeNode .mpk installed in the project's widgets/. +-- ============================================================================ + +create module ChildSlot; +/ +create entity ChildSlot.DemoItem ( + Title: String, + Score: Integer +); +/ +create or replace page ChildSlot.Home ( layout: Atlas_Core.Atlas_Default ) { + pluggablewidget 'com.mendix.widget.web.treenode.TreeNode' treeNested ( + datasource: database from ChildSlot.DemoItem, + headerType: 'text', + headerCaption: '{Title}', + hasChildren: true, + startExpanded: true, + showIcon: 'left' + ) { + container children { + dynamictext treeChildText (content: 'Score: {1}', contentparams: [{1} = Score]) + } + } +} diff --git a/mdl-examples/bug-tests/pluggable-texttemplate-content.mdl b/mdl-examples/bug-tests/pluggable-texttemplate-content.mdl new file mode 100644 index 000000000..f96bd0b7f --- /dev/null +++ b/mdl-examples/bug-tests/pluggable-texttemplate-content.mdl @@ -0,0 +1,69 @@ +-- ============================================================================ +-- Bug: textTemplate content silently dropped for generated-def pluggable widgets +-- ============================================================================ +-- +-- Symptom: +-- Authoring a Badge `value`, TreeNode `headerCaption`, or Timeline +-- `title`/`description`/`timeIndication` (all textTemplate properties) via MDL +-- left the caption empty on the persisted widget: +-- - `describe page` showed the instance with no such property +-- - `mxcli check` reported MDL-WIDGET01 "widget X has no property " +-- +-- Root cause: +-- GenerateDefJSON emitted a texttemplate PropertyMapping only when a friendly +-- MDL alias was hand-registered (propertyAliases, PieChart/HeatMap only). Every +-- other top-level textTemplate was skipped, so the writer had no mapping to +-- read the caption from the MDL and dropped it. +-- +-- Fix: +-- Emit a texttemplate mapping for EVERY top-level textTemplate property (keyed +-- by the property's own name, so it's authorable as `value:`/`title:`/…), and +-- have the engine keep the template's default ClientTemplate when the property +-- is left unset (so emitting the mapping never nulls a default caption). +-- +-- Verify: +-- mxcli exec mdl-examples/bug-tests/pluggable-texttemplate-content.mdl -p app.mpr +-- mxcli check ... -p app.mpr # no MDL-WIDGET01 for value/headerCaption/title +-- mx check app.mpr # 0 errors (v2 preserved) +-- Requires: Badge / TreeNode / Timeline .mpk installed in the project's widgets/. +-- ============================================================================ + +create module TTContent; +/ +create entity TTContent.DemoItem ( + Title: String, + Description: String, + Status: String +); +/ +-- Timeline: title / description are textTemplate captions (default visualization). +create or replace page TTContent.Home ( layout: Atlas_Core.Atlas_Default ) { + pluggablewidget 'com.mendix.widget.web.timeline.Timeline' tl ( + datasource: database from TTContent.DemoItem, + title: '{Title}', + description: '{Description}', + groupEvents: false, + customVisualization: false + ) + -- TreeNode: headerCaption is a textTemplate. + pluggablewidget 'com.mendix.widget.web.treenode.TreeNode' tn ( + datasource: database from TTContent.DemoItem, + headerType: 'text', + headerCaption: '{Title}', + hasChildren: false, + showIcon: 'no' + ) +} +/ +-- Badge: value is a textTemplate (rendered inside a data view context). +create or replace page TTContent.Detail ( + params: { $Item: TTContent.DemoItem }, + layout: Atlas_Core.Atlas_Default +) { + dataview dv (DataSource: $Item) { + pluggablewidget 'com.mendix.widget.custom.badge.Badge' bdg ( + type: 'badge', + value: '{Status}' + ) + } +} diff --git a/mdl-examples/bug-tests/timeseries-marker-color-null.mdl b/mdl-examples/bug-tests/timeseries-marker-color-null.mdl new file mode 100644 index 000000000..e67030498 --- /dev/null +++ b/mdl-examples/bug-tests/timeseries-marker-color-null.mdl @@ -0,0 +1,45 @@ +-- ============================================================================ +-- Bug: TimeSeries/LineChart series markerColor emitted as empty ClientTemplate +-- ============================================================================ +-- +-- Symptom: +-- A TimeSeries (or LineChart) with a default `line` series failed mx check: +-- [CE0463] "The definition of this widget has changed…" at Time series +-- The series' `markerColor` textTemplate was serialized as an empty +-- Forms$ClientTemplate, but Studio Pro stores it as null when markers are off. +-- +-- Root cause: +-- A chart-series textTemplate that is VISIBLE serializes as an empty +-- ClientTemplate; one that is HIDDEN must be null. chartSeriesTextTemplateVisible +-- honored only the dataSet (static/dynamic) gate — it missed the item-level +-- editorConfig gates: `markerColor` is hidden unless lineStyle == "lineWithMarkers" +-- (default "line"), and `fillColor` is hidden unless enableFillArea is truthy. +-- +-- Fix: +-- chartSeriesTextTemplateHiddenByItemConfig nulls markerColor / fillColor when the +-- item's lineStyle / enableFillArea hides them, mirroring the widget's +-- editorConfig hideNestedPropertiesIn rules. +-- +-- Verify: +-- mxcli exec mdl-examples/bug-tests/timeseries-marker-color-null.mdl -p app.mpr +-- mx check app.mpr # 0 errors — markerColor null on the default line series +-- Requires: Charts .mpk installed in the project's widgets/. +-- ============================================================================ + +create module TSMarker; +/ +create entity TSMarker.Sale ( SaleDate: DateTime, Total: Decimal ); +/ +create or replace page TSMarker.Home ( layout: Atlas_Core.Atlas_Default ) { + -- default line style: markerColor is hidden -> must be null (not empty ClientTemplate) + pluggablewidget 'com.mendix.widget.web.timeseries.TimeSeries' tsPlain { + line lRev ( DataSet: 'static', DataSource: database from TSMarker.Sale, + StaticXAttribute: SaleDate, StaticYAttribute: Total, StaticName: 'Revenue' ) + } + -- lineWithMarkers: markerColor is visible -> empty ClientTemplate is correct + pluggablewidget 'com.mendix.widget.web.timeseries.TimeSeries' tsMarkers { + line lRev2 ( DataSet: 'static', lineStyle: 'lineWithMarkers', + DataSource: database from TSMarker.Sale, + StaticXAttribute: SaleDate, StaticYAttribute: Total, StaticName: 'Revenue' ) + } +} diff --git a/mdl-examples/bug-tests/xpath-assoc-path-expansion.mdl b/mdl-examples/bug-tests/xpath-assoc-path-expansion.mdl new file mode 100644 index 000000000..a19844a19 --- /dev/null +++ b/mdl-examples/bug-tests/xpath-assoc-path-expansion.mdl @@ -0,0 +1,56 @@ +-- ============================================================================ +-- Bug: multi-hop association XPath in a datasource trips CE1613 +-- ============================================================================ +-- +-- Symptom (from a Travel itinerary project): +-- A Timeline over Activities constrained to a Trip two associations away: +-- datasource: database from Travel.Activity +-- where [Travel.Day_Activity/Travel.Trip_Day = $Trip] +-- failed `mx check` / the deploy build with: +-- [CE1613] "The selected entity 'Travel.Trip_Day' no longer exists." +-- mxbuild read the second association segment as an ENTITY. +-- +-- Root cause: +-- Mendix XPath requires the intermediate entity between two association hops +-- — `[Assoc1/Entity/Assoc2 = $x]`. The MDL shorthand `[Assoc1/Assoc2 = $x]` +-- (natural, and what an LLM emits) omits it, and mxcli stored the constraint +-- verbatim, so mxbuild couldn't resolve the second hop. +-- +-- Fix: +-- buildDataSourceV3 now expands consecutive association segments, inserting the +-- intermediate entity (resolved from each association's target). The full form, +-- single-hop constraints, and attribute paths are left verbatim, so it never +-- corrupts a valid constraint. +-- +-- Verify: +-- mxcli exec mdl-examples/bug-tests/xpath-assoc-path-expansion.mdl -p app.mpr +-- mx check app.mpr # 0 errors — the two-hop path resolves +-- ============================================================================ + +create module Travel; +/ +create entity Travel.Trip ( Title: String ); +create entity Travel.Day ( DayNumber: Integer ); +create entity Travel.Activity ( Name: String, StartTime: DateTime ); +/ +-- Day belongs to Trip; Activity belongs to Day. +create association Travel.Trip_Day from Travel.Day to Travel.Trip; +create association Travel.Day_Activity from Travel.Activity to Travel.Day; +/ +create or replace page Travel.Trip_Detail ( + params: { $Trip: Travel.Trip }, + layout: Atlas_Core.Atlas_Default +) { + -- one-hop reverse association (already worked; must stay working) + pluggablewidget 'com.mendix.widget.web.treenode.TreeNode' tnDays ( + datasource: database from Travel.Day where [Travel.Trip_Day = $Trip] sort by DayNumber asc, + headerType: 'text', headerCaption: 'Day {DayNumber}', hasChildren: false, showIcon: 'left' + ) + -- two-hop reverse association shorthand — expanded to Assoc/Entity/Assoc. + pluggablewidget 'com.mendix.widget.web.timeline.Timeline' tlTrip ( + datasource: database from Travel.Activity where [Travel.Day_Activity/Travel.Trip_Day = $Trip] sort by StartTime asc, + title: '{Name}', + groupEvents: false, + customVisualization: false + ) +} diff --git a/mdl-examples/doctype-tests/32-pluggable-widget-object-lists-v010.mdl b/mdl-examples/doctype-tests/32-pluggable-widget-object-lists-v010.mdl index 21728fec6..506e0e6d4 100644 --- a/mdl-examples/doctype-tests/32-pluggable-widget-object-lists-v010.mdl +++ b/mdl-examples/doctype-tests/32-pluggable-widget-object-lists-v010.mdl @@ -385,6 +385,12 @@ create page ObjListV10.P_OL08_LineChart ( -- HeatMap's `scaleColors` object-list → the SCALECOLOR keyword. Each entry maps -- a value percentage to a colour; `ColorValue:` fills the schema's `colour` -- primitive (British spelling — the alias keeps the friendly MDL name). +-- +-- HeatMap binds its data at the WIDGET level (no `series` object-list): the +-- `DataSource:` + `ValueAttribute:` fill the schema's `Series`/`Value attribute` +-- (both required — CE0642 without them), and `HorizontalAxisAttribute:` / +-- `VerticalAxisAttribute:` pick the two category axes. The SCALECOLOR entries +-- then layer the colour scale over that grid. create page ObjListV10.P_OL09_HeatMap ( Title: 'Sales Heat Map', @@ -394,7 +400,12 @@ create page ObjListV10.P_OL09_HeatMap ( layoutgrid lg1 { row r1 { column c1 (DesktopWidth: 12) { - pluggablewidget 'com.mendix.widget.web.heatmap.HeatMap' heatMap1 { + pluggablewidget 'com.mendix.widget.web.heatmap.HeatMap' heatMap1 ( + DataSource: database from ObjListV10.SalesData, + HorizontalAxisAttribute: Region, + VerticalAxisAttribute: PeriodLabel, + ValueAttribute: RevenueValue + ) { scalecolor scLow (ValuePercentage: 0, ColorValue: '#f7fbff') scalecolor scHigh (ValuePercentage: 100, ColorValue: '#08306b') } diff --git a/mdl-examples/doctype-tests/33-alter-page-examples.mdl b/mdl-examples/doctype-tests/33-alter-page-examples.mdl index 48b895573..1a4c00d0f 100644 --- a/mdl-examples/doctype-tests/33-alter-page-examples.mdl +++ b/mdl-examples/doctype-tests/33-alter-page-examples.mdl @@ -131,6 +131,19 @@ alter page AlterPg.Product_Overview { } }; +-- MARK: AP02b — INSERT INTO a container (append as last child) + +-- ============================================================================ +-- AP02b: INSERT INTO — append widgets as a container's children. This is the +-- only way to fill an EMPTY container (here, the `divider` just created above). +-- ============================================================================ + +alter page AlterPg.Product_Overview { + insert into divider { + dynamictext dividerNote (Content: 'Catalog below', RenderMode: paragraph) + } +}; + -- MARK: AP03 — INSERT a DataGrid2 with initial columns -- ============================================================================ diff --git a/mdl-examples/doctype-tests/alter-page-insert-into.mdl b/mdl-examples/doctype-tests/alter-page-insert-into.mdl new file mode 100644 index 000000000..72bdfc2e3 --- /dev/null +++ b/mdl-examples/doctype-tests/alter-page-insert-into.mdl @@ -0,0 +1,50 @@ +-- ============================================================================ +-- ALTER PAGE — INSERT INTO +-- ============================================================================ +-- INSERT INTO appends widgets as the LAST children of a named container, rather +-- than as a sibling before/after another widget. This is the only way to fill an +-- EMPTY container, and the natural way to add a widget to a container/dataview +-- without needing a sibling to anchor to. +-- +-- insert into { } +-- +-- Simple containers are supported (container/DivContainer, dataview, groupbox, +-- scroll-container region). For a layout grid or tab container, insert relative to +-- a widget inside the target column/tab instead. +-- ============================================================================ + +create module AlterInto; +/ +create entity AlterInto.Item ( Title: String, Status: String ); +/ +create or replace page AlterInto.Home ( layout: Atlas_Core.Atlas_Default ) { + container ctnEmpty (class: 'toolbar') { } + container ctnBody (class: 'body') { + dynamictext dtExisting (content: 'Existing content') + } +} +/ +create or replace page AlterInto.Detail ( params: { $Item: AlterInto.Item }, layout: Atlas_Core.Atlas_Default ) { + dataview dv (datasource: $Item) { } +} +/ +-- Fill an empty container. +alter page AlterInto.Home { + insert into ctnEmpty { + actionbutton btnNew (caption: 'New', action: nothing, buttonstyle: primary) + } +} +/ +-- Append as the last child of a container that already has widgets. +alter page AlterInto.Home { + insert into ctnBody { + pluggablewidget 'com.mendix.widget.custom.badge.Badge' bdgStatus (type: 'badge', value: 'Active') + } +} +/ +-- Insert a data-bound widget into a dataview — the children take the dataview's entity. +alter page AlterInto.Detail { + insert into dv { + dynamictext dtTitle (content: '{1}', contentparams: [{1} = Title], rendermode: H1) + } +} diff --git a/mdl/ast/ast_alter_page.go b/mdl/ast/ast_alter_page.go index 6f5a836d2..c857084b0 100644 --- a/mdl/ast/ast_alter_page.go +++ b/mdl/ast/ast_alter_page.go @@ -50,10 +50,10 @@ type SetPropertyOp struct { func (s *SetPropertyOp) isAlterPageOperation() {} -// InsertWidgetOp represents: INSERT AFTER/BEFORE widgetRef { widgets } +// InsertWidgetOp represents: INSERT AFTER/BEFORE/INTO widgetRef { widgets } type InsertWidgetOp struct { - Position string // "AFTER" or "BEFORE" - Target WidgetRef // widget/column to insert relative to + Position string // "AFTER", "BEFORE", or "INTO" (append as children of the container) + Target WidgetRef // widget to insert relative to, or container to insert into Widgets []*WidgetV3 } diff --git a/mdl/backend/mcp/page_mutator.go b/mdl/backend/mcp/page_mutator.go index acdc4f70b..7186b3814 100644 --- a/mdl/backend/mcp/page_mutator.go +++ b/mdl/backend/mcp/page_mutator.go @@ -260,7 +260,7 @@ func (m *mcpPageMutator) InsertWidget(widgetRef string, columnRef string, positi if columnRef != "" { return fmt.Errorf("inserting into a column (%s.%s) is not yet supported by the MCP backend", widgetRef, columnRef) } - parent, key, idx, _, ok := findWidget(m.content, widgetRef) + parent, key, idx, widget, ok := findWidget(m.content, widgetRef) if !ok { return fmt.Errorf("widget %q not found", widgetRef) } @@ -268,6 +268,16 @@ func (m *mcpPageMutator) InsertWidget(widgetRef string, columnRef string, positi if err != nil { return err } + // INSERT INTO: append as children of the target container itself. + if strings.EqualFold(string(position), "into") { + if _, hasChildren := widget["widgets"]; !hasChildren { + return fmt.Errorf("cannot INSERT INTO %q (%v): it has no direct child list — "+ + "use INSERT BEFORE/AFTER a widget inside the target column or tab instead", widgetRef, widget["$Type"]) + } + children, _ := widget["widgets"].([]any) + widget["widgets"] = append(append([]any{}, children...), mapped...) + return nil + } arr, _ := parent[key].([]any) at := idx // The executor passes the AST token ("AFTER"/"BEFORE"), so compare case-insensitively. diff --git a/mdl/backend/mcp/page_mutator_test.go b/mdl/backend/mcp/page_mutator_test.go index 46140dc2a..d81e2c7a9 100644 --- a/mdl/backend/mcp/page_mutator_test.go +++ b/mdl/backend/mcp/page_mutator_test.go @@ -67,6 +67,22 @@ func TestPageMutator_InsertAfterBefore(t *testing.T) { } } +func TestPageMutator_InsertInto(t *testing.T) { + m := newTestMutator() + // INSERT INTO appends as the last child of the container itself (dv), not a sibling. + if err := m.InsertWidget("dv", "", backend.InsertPosition("INTO"), []pages.Widget{dynText("t2")}); err != nil { + t.Fatal(err) + } + if got := dvChildNames(m); len(got) != 2 || got[0] != "t1" || got[1] != "t2" { + t.Fatalf("after INTO insert: %v (want [t1 t2])", got) + } + // Inserting into a non-container (the DynamicText t1) is a clear error. + err := m.InsertWidget("t1", "", backend.InsertPosition("INTO"), []pages.Widget{dynText("x")}) + if err == nil || !strings.Contains(err.Error(), "INSERT INTO") { + t.Fatalf("expected INSERT INTO error for non-container, got %v", err) + } +} + func TestPageMutator_ReplaceAndDrop(t *testing.T) { m := newTestMutator() _ = m.InsertWidget("t1", "", backend.InsertPosition("AFTER"), []pages.Widget{dynText("t2")}) diff --git a/mdl/backend/modelsdk/widget_pluggable_write.go b/mdl/backend/modelsdk/widget_pluggable_write.go index eb3e8d432..671dc7917 100644 --- a/mdl/backend/modelsdk/widget_pluggable_write.go +++ b/mdl/backend/modelsdk/widget_pluggable_write.go @@ -59,13 +59,29 @@ func (b *Backend) BuildCreateAttributeObject(attributePath string, objectTypeID, // of the CustomWidget envelope (Appearance, editability, …) is added when the // widget is serialized via customWidgetToGen. func (b *Backend) BuildFilterWidget(spec backend.FilterWidgetSpec, projectPath string) (pages.Widget, error) { - typeBSON, objBSON, _, _, _, err := mwidgets.GetTemplateFullBSON(spec.WidgetID, mmpr.GenerateID, projectPath) + typeBSON, objBSON, propertyTypeIDs, _, _, err := mwidgets.GetTemplateFullBSON(spec.WidgetID, mmpr.GenerateID, projectPath) if err != nil { return nil, fmt.Errorf("BuildFilterWidget: load template %s: %w", spec.WidgetID, err) } if typeBSON == nil || objBSON == nil { return nil, fmt.Errorf("BuildFilterWidget: no template for widget %s", spec.WidgetID) } + objV1 := v2ToV1BSON(objBSON) + // Null any TextTemplate the widget's editorConfig hides in the filter's default + // configuration (#574) — e.g. a DropdownFilter's screen-reader captions when + // `adjustable` is off — else the populated default trips CE0463. + if len(spec.VisibilityRules) > 0 { + pageIDs := make(map[string]pages.PropertyTypeIDEntry, len(propertyTypeIDs)) + for k, e := range propertyTypeIDs { + pageIDs[k] = pages.PropertyTypeIDEntry{ + PropertyTypeID: e.PropertyTypeID, + ValueTypeID: e.ValueTypeID, + DefaultValue: e.DefaultValue, + ValueType: e.ValueType, + } + } + objV1 = widgetobj.ApplyVisibilityRules(objV1, pageIDs, spec.VisibilityRules) + } return &pages.CustomWidget{ BaseWidget: pages.BaseWidget{ BaseElement: model.BaseElement{ @@ -75,7 +91,7 @@ func (b *Backend) BuildFilterWidget(spec backend.FilterWidgetSpec, projectPath s Name: spec.FilterName, }, Editable: "Always", - RawObject: v2ToV1BSON(objBSON), + RawObject: objV1, RawType: v2ToV1BSON(typeBSON), }, nil } diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index ce47b6914..d07df6470 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -35,6 +35,19 @@ func init() { codec.RegisterTypeDefaults("Forms$ClientTemplate", codec.TypeDefaults{ MandatoryListMarkers: map[string]int32{"Parameters": 2}, }) + // A pluggable-widget XPath datasource (DataGrid2 / Gallery / …) always serializes + // SourceVariable as null when unbound, and its GridSortBar always emits an (empty) + // SortItems list with marker 2. An older embedded template extracted before these + // existed omits them, so a project whose installed widget expects them (e.g. Data + // Widgets updated past the 11.6 template — Gallery@10.24) reports CE0463. Registering + // the defaults makes the emitted datasource match the installed widget. (Axis 2 of + // the object-list drift — see PROPOSAL_multi_version_pluggable_widgets.md.) + codec.RegisterTypeDefaults("CustomWidgets$CustomWidgetXPathSource", codec.TypeDefaults{ + NullFields: []string{"SourceVariable"}, + }) + codec.RegisterTypeDefaults("Forms$GridSortBar", codec.TypeDefaults{ + MandatoryListMarkers: map[string]int32{"SortItems": 2}, + }) // LayoutGrid and its rows carry a null ConditionalVisibilitySettings; the grid, // rows, and columns all use the typed-array marker 2 in their parent lists. for _, t := range []string{"Forms$LayoutGrid", "Forms$LayoutGridRow"} { diff --git a/mdl/backend/mutation.go b/mdl/backend/mutation.go index 6a4e77a88..91bcffb7f 100644 --- a/mdl/backend/mutation.go +++ b/mdl/backend/mutation.go @@ -402,6 +402,12 @@ type DataGridColumnSpec struct { type FilterWidgetSpec struct { WidgetID string // e.g. pages.WidgetIDDataGridTextFilter FilterName string // widget name + // VisibilityRules are the widget's editorConfig-derived property-visibility + // rules (#574). A filter is built with its default configuration, so any + // TextTemplate the widget hides in that state must be nulled to avoid CE0463; + // the executor resolves these from the installed .mpk and the backend applies + // them via widgetobj.ApplyVisibilityRules. + VisibilityRules []types.WidgetVisibilityRule } // WidgetBuilderBackend provides pluggable widget construction capabilities. diff --git a/mdl/backend/pagemutator/mutator.go b/mdl/backend/pagemutator/mutator.go index 834a7e1bf..e5f533168 100644 --- a/mdl/backend/pagemutator/mutator.go +++ b/mdl/backend/pagemutator/mutator.go @@ -184,6 +184,21 @@ func (m *Mutator) InsertWidget(widgetRef string, columnRef string, position back return fmt.Errorf("serialize widgets: %w", err) } + // INSERT INTO: append the widgets as children of the target container itself + // (its `Widgets` array), rather than as siblings in the target's parent array. + // Enables inserting into an empty container or as a container's last child. + if strings.EqualFold(string(position), "into") { + newContainer, err := appendChildrenToContainer(result.widget, widgetRef, newBsonWidgets) + if err != nil { + return err + } + // Write the (possibly reallocated — an empty container gains a new Widgets + // field) container doc back into its parent slot. + result.parentArr[result.index] = newContainer + bsonnav.DSetArray(result.parentDoc, result.parentKey, result.parentArr) + return nil + } + insertIdx := result.index if strings.EqualFold(string(position), "after") { insertIdx = result.index + 1 @@ -198,6 +213,78 @@ func (m *Mutator) InsertWidget(widgetRef string, columnRef string, position back return nil } +// insertIntoContainer appends widgets as the last children of a container widget +// (its `Widgets` array). Simple containers (Pages$DivContainer, Forms$Container, +// DataView, GroupBox, ScrollContainer, …) keep their children in a single +// `Widgets` list. LayoutGrid (Rows/Columns) and TabContainer (TabPages) have no +// single child list, so INSERT INTO can't target them directly — the caller is +// told to insert relative to a widget inside the target column/tab instead. +// appendChildrenToContainer appends widgets to a container's `Widgets` list and +// returns the (possibly reallocated) container doc. An empty container omits the +// Widgets field entirely, so it is added — which grows the bson.D slice, hence the +// caller must store the returned doc back into its parent slot. +func appendChildrenToContainer(container bson.D, widgetRef string, newBsonWidgets []any) (bson.D, error) { + if !containerAcceptsWidgets(container) { + typeName := bsonnav.DGetString(container, "$Type") + return nil, fmt.Errorf("cannot INSERT INTO %q (%s): it is not a simple container — "+ + "use INSERT BEFORE/AFTER a widget inside the target column or tab instead", widgetRef, typeName) + } + // Append after any existing children, preserving the Mendix list marker (a + // leading int32). An empty container omits the Widgets field, so create it with + // the default marker (2) that Mendix uses for widget lists. + raw := bsonnav.ToBsonA(bsonnav.DGet(container, "Widgets")) + var out bson.A + switch { + case len(raw) > 0 && isListMarker(raw[0]): + out = append(out, raw...) // marker + existing children + out = append(out, newBsonWidgets...) + case len(raw) == 0: + out = append(out, int32(2)) // fresh (empty container): Mendix widget-list marker + out = append(out, newBsonWidgets...) + default: + out = append(out, raw...) // children without a marker (unusual): keep as-is + out = append(out, newBsonWidgets...) + } + // DSet updates an existing field in place; if Widgets is absent (empty + // container), append the field, growing the slice. + if bsonnav.DSet(container, "Widgets", out) { + return container, nil + } + return append(container, bson.E{Key: "Widgets", Value: out}), nil +} + +// isListMarker reports whether a value is a Mendix list-version marker (a leading int). +func isListMarker(v any) bool { + switch v.(type) { + case int32, int: + return true + } + return false +} + +// containerAcceptsWidgets reports whether a widget keeps its children in a single +// `Widgets` list that INSERT INTO can append to. A container that currently holds +// children always has the field; an empty one omits it, so recognise the common +// simple-container types by $Type. LayoutGrid (Rows/Columns) and TabContainer +// (TabPages) are intentionally excluded — they have no single child list. +func containerAcceptsWidgets(container bson.D) bool { + for _, e := range container { + if e.Key == "Widgets" { + return true + } + } + switch bsonnav.DGetString(container, "$Type") { + case "Forms$DivContainer", + "Forms$Container", + "Forms$DataView", + "Forms$GroupBox", + "Forms$ScrollContainerRegion", + "Forms$Section": + return true + } + return false +} + func (m *Mutator) DropWidget(refs []backend.WidgetRef) error { for _, ref := range refs { // Re-find widget each iteration because previous drops mutate the tree. diff --git a/mdl/backend/widgetobj/builder.go b/mdl/backend/widgetobj/builder.go index bfca48302..d2febccd8 100644 --- a/mdl/backend/widgetobj/builder.go +++ b/mdl/backend/widgetobj/builder.go @@ -607,31 +607,44 @@ func (ob *Builder) EnsureRequiredObjectLists() { // TextTemplate. The widget's current primitive values (read from the assembled // object) drive rule evaluation, so a rule keyed on e.g. `type` sees the value // just set from MDL. + func (ob *Builder) ApplyPropertyVisibility(rules []types.WidgetVisibilityRule) { + ob.object = ApplyVisibilityRules(ob.object, ob.propertyTypeIDs, rules) +} + +// ApplyVisibilityRules nulls the TextTemplate of any TextTemplate-typed property +// the rules mark as hidden under the object's current configuration, returning +// the updated object. It is the engine-agnostic form of ApplyPropertyVisibility +// so build paths that construct a widget object without a full Builder (e.g. the +// filter-widget build path) can apply the same #574 nulling. Non-TextTemplate +// properties are left untouched — only the populated-vs-null ClientTemplate +// choice triggers CE0463. +func ApplyVisibilityRules(object bson.D, propertyTypeIDs map[string]pages.PropertyTypeIDEntry, rules []types.WidgetVisibilityRule) bson.D { if len(rules) == 0 { - return + return object } - values := ob.currentPrimitiveValues() + values := primitiveValuesOf(object, propertyTypeIDs) for _, rule := range rules { if !rule.HiddenWhen.Hidden(values) { continue } - entry, ok := ob.propertyTypeIDs[rule.PropertyKey] + entry, ok := propertyTypeIDs[rule.PropertyKey] if !ok || entry.ValueType != "TextTemplate" { continue } - ob.object = updateWidgetPropertyValue(ob.object, ob.propertyTypeIDs, rule.PropertyKey, func(val bson.D) bson.D { + object = updateWidgetPropertyValue(object, propertyTypeIDs, rule.PropertyKey, func(val bson.D) bson.D { return setBSONField(val, "TextTemplate", nil) }) } + return object } -// currentPrimitiveValues maps each known property key to its current -// PrimitiveValue string in the assembled object (e.g. type → "expression", -// customVisualization → "true"). Properties absent from the object map to "". -func (ob *Builder) currentPrimitiveValues() map[string]string { - byID := make(map[string]string) - for _, elem := range ob.object { +// primitiveValuesOf maps each known property key to its current comparable value +// string in the object (e.g. type → "expression", itemSelection → "Single"). +// Properties absent from the object map to "". +func primitiveValuesOf(object bson.D, propertyTypeIDs map[string]pages.PropertyTypeIDEntry) map[string]string { + rawByID := make(map[string]bson.D) + for _, elem := range object { if elem.Key != "Properties" { continue } @@ -648,19 +661,57 @@ func (ob *Builder) currentPrimitiveValues() map[string]string { if id == "" { continue } - byID[id] = primitiveValueOfProperty(prop) + if val := widgetValueOfProperty(prop); val != nil { + rawByID[id] = val + } } } - out := make(map[string]string, len(ob.propertyTypeIDs)) - for key, entry := range ob.propertyTypeIDs { - if v, ok := byID[strings.ReplaceAll(entry.PropertyTypeID, "-", "")]; ok { - out[key] = v + out := make(map[string]string, len(propertyTypeIDs)) + for key, entry := range propertyTypeIDs { + if val, ok := rawByID[strings.ReplaceAll(entry.PropertyTypeID, "-", "")]; ok { + out[key] = effectiveValue(val, entry.ValueType) } } return out } +// effectiveValue reads the field of a WidgetValue that holds a property's +// primitive-comparable value, keyed by its ValueType: a Selection-typed property +// (e.g. DataGrid2 itemSelection = None/Single/Multi) stores its value in the +// `Selection` field, everything else in `PrimitiveValue`. editorConfig +// visibility conditions compare against this value, so reading the wrong field +// (always PrimitiveValue) made selection-conditioned rules evaluate against "" +// and mis-fire (issue #574: singleSelectionColumnLabel nulled under Selection:Single). +func effectiveValue(val bson.D, valueType string) string { + field := "PrimitiveValue" + if valueType == "Selection" { + field = "Selection" + } + for _, ve := range val { + if ve.Key == field { + if s, ok := ve.Value.(string); ok { + return s + } + return "" + } + } + return "" +} + +// widgetValueOfProperty returns a WidgetProperty's Value document, or nil. +func widgetValueOfProperty(prop bson.D) bson.D { + for _, elem := range prop { + if elem.Key == "Value" { + if val, ok := elem.Value.(bson.D); ok { + return val + } + return nil + } + } + return nil +} + // propertyTypePointerID returns a WidgetProperty's TypePointer as a normalized // (dash-stripped) UUID hex string, or "" when absent. func propertyTypePointerID(prop bson.D) string { @@ -1240,9 +1291,14 @@ func createDefaultWidgetValue(entry pages.PropertyTypeIDEntry) bson.D { textTemplate = createDefaultClientTemplateBSON(primitiveVal) } case "String": - if primitiveVal == "" { - primitiveVal = " " - } + // Leave an unset String property empty (its schema default, "") — do NOT + // manufacture a single space. Studio Pro stores "" here, and some widgets + // (chart series `customSeriesOptions`, `customLayout`, `customConfigurations`) + // feed the value straight to a client-side JSON.parse whose empty-guard is + // `value !== "" ? value : "{}"`; a lone space passes that guard and makes + // the widget throw `JSON.parse(" ")` → "Unexpected end of JSON input" at + // render time (object-list-item path only; the top-level path already + // leaves it empty). } return bson.D{ diff --git a/mdl/backend/widgetobj/widget_visibility_test.go b/mdl/backend/widgetobj/widget_visibility_test.go index ef62ca846..975d3aca3 100644 --- a/mdl/backend/widgetobj/widget_visibility_test.go +++ b/mdl/backend/widgetobj/widget_visibility_test.go @@ -102,6 +102,84 @@ func TestApplyPropertyVisibility(t *testing.T) { }) } +// TestApplyPropertyVisibility_SelectionTypedCondition locks in the fix for a +// visibility condition keyed on a Selection-typed property (e.g. DataGrid2 +// itemSelection = None/Single/Multi): its value lives in the WidgetValue's +// `Selection` field, not `PrimitiveValue`. Reading the wrong field made the +// condition see "" and mis-fire (#574: singleSelectionColumnLabel was nulled +// even under Selection:Single). +func TestApplyPropertyVisibility_SelectionTypedCondition(t *testing.T) { + const ( + selID = "44444444-4444-4444-4444-444444444444" // Selection "itemSelection" + lblID = "55555555-5555-5555-5555-555555555555" // TextTemplate "singleSelectionColumnLabel" + tmplTy = "Forms$ClientTemplate" + ) + populated := bson.D{{Key: "$Type", Value: tmplTy}} + + build := func(selectionValue string) *Builder { + return &Builder{ + object: bson.D{{Key: "Properties", Value: bson.A{ + int32(2), + bson.D{ + {Key: "$Type", Value: "CustomWidgets$WidgetProperty"}, + {Key: "TypePointer", Value: types.UUIDToBlob(selID)}, + {Key: "Value", Value: bson.D{ + {Key: "$Type", Value: "CustomWidgets$WidgetValue"}, + {Key: "PrimitiveValue", Value: ""}, // selection value is NOT here + {Key: "Selection", Value: selectionValue}, + }}, + }, + bson.D{ + {Key: "$Type", Value: "CustomWidgets$WidgetProperty"}, + {Key: "TypePointer", Value: types.UUIDToBlob(lblID)}, + {Key: "Value", Value: bson.D{ + {Key: "$Type", Value: "CustomWidgets$WidgetValue"}, + {Key: "TextTemplate", Value: populated}, + }}, + }, + }}}, + propertyTypeIDs: map[string]pages.PropertyTypeIDEntry{ + "itemSelection": {PropertyTypeID: selID, ValueType: "Selection"}, + "singleSelectionColumnLabel": {PropertyTypeID: lblID, ValueType: "TextTemplate"}, + }, + } + } + rules := []types.WidgetVisibilityRule{{ + PropertyKey: "singleSelectionColumnLabel", + HiddenWhen: &types.WidgetVisibilityCondition{PropertyKey: "itemSelection", Operator: "ne", Value: "Single"}, + }} + tt := func(ob *Builder) any { + for _, elem := range ob.object { + if elem.Key != "Properties" { + continue + } + for _, item := range elem.Value.(bson.A) { + prop, ok := item.(bson.D) + if !ok || propertyTypePointerID(prop) != normalizeID(lblID) { + continue + } + return findField(t, findField(t, prop, "Value").(bson.D), "TextTemplate") + } + } + return "not-found" + } + + t.Run("Single → visible → preserved", func(t *testing.T) { + ob := build("Single") + ob.ApplyPropertyVisibility(rules) + if tt(ob) == nil { + t.Error("singleSelectionColumnLabel nulled under Selection:Single (Selection field not read)") + } + }) + t.Run("None → hidden → nulled", func(t *testing.T) { + ob := build("None") + ob.ApplyPropertyVisibility(rules) + if tt(ob) != nil { + t.Error("singleSelectionColumnLabel not nulled under Selection:None") + } + }) +} + func normalizeID(id string) string { out := make([]byte, 0, len(id)) for i := 0; i < len(id); i++ { diff --git a/mdl/executor/cmd_alter_page.go b/mdl/executor/cmd_alter_page.go index dfa4374d0..2dd42870b 100644 --- a/mdl/executor/cmd_alter_page.go +++ b/mdl/executor/cmd_alter_page.go @@ -190,8 +190,14 @@ func applyInsertWidgetMutator(ctx *ExecContext, mutator backend.PageMutator, op return mutator.InsertColumns(op.Target.Widget, op.Target.Column, backend.InsertPosition(op.Position), specs) } - // Find entity context from enclosing DataView/DataGrid/ListView for regular widget insert. + // Find entity context for the new widgets. For INSERT BEFORE/AFTER the target + // is a sibling, so use its enclosing container's context; for INSERT INTO the + // target IS the container, so the children take the target's own context (e.g. + // a dataview's entity). entityCtx := mutator.EnclosingEntity(op.Target.Widget) + if strings.EqualFold(op.Position, "INTO") { + entityCtx = mutator.EnclosingEntityForChildren(op.Target.Widget) + } // Build new widgets from AST widgets, err := buildWidgetsFromAST(ctx, op.Widgets, moduleName, moduleID, entityCtx, mutator) diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index c860a8966..b7f40c341 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -5,6 +5,7 @@ package executor import ( "fmt" "log" + "regexp" "strings" "github.com/mendixlabs/mxcli/mdl/ast" @@ -297,10 +298,12 @@ func (pb *pageBuilder) buildSnippetV3(s *ast.CreateSnippetStmtV3) (*pages.Snippe // // Keyword dispatch (Phase 2 — issue #539): the keywordDispatchTable encodes // our editorial policy for dual-stack keywords (e.g. DATAGRID → pluggable -// Datagrid 2.x). Today the existing switch cases handle this correctly via -// the hand-coded builders (buildDataGridV3 already produces pluggable BSON). -// The dispatch table is consumed by inspection commands and DESCRIBE-side -// keyword resolution rather than overriding write-side routing here. +// Datagrid 2.x). DATAGRID has no switch case below; it falls through to the +// default branch, which resolves it via the widget registry (a datagrid.def.json +// generated on demand from the project .mpk) and builds it through the generic +// pluggable engine (buildPluggable) — columns as an ObjectList, per-column filters +// as item slots. The dispatch table is consumed by inspection commands and +// DESCRIBE-side keyword resolution rather than overriding write-side routing here. func (pb *pageBuilder) buildWidgetV3(w *ast.WidgetV3) (pages.Widget, error) { var widget pages.Widget var err error @@ -661,9 +664,11 @@ func (pb *pageBuilder) buildDataSourceV3(ds *ast.DataSourceV3) (pages.DataSource EntityName: ds.Reference, } - // Handle WHERE clause + // Handle WHERE clause. Expand association-only paths to the Assoc/Entity/Assoc + // form Mendix requires (see expandXPathAssociationPath), so the shorthand + // `[Mod.Assoc1/Mod.Assoc2 = $x]` doesn't trip CE1613 at build time. if ds.Where != "" { - dbSource.XPathConstraint = ds.Where + dbSource.XPathConstraint = pb.expandXPathAssociationPath(ds.Where, ds.Reference) } // Handle ORDER BY @@ -785,6 +790,59 @@ func (pb *pageBuilder) buildDataSourceV3(ds *ast.DataSourceV3) (pages.DataSource // // Convention (per CLAUDE.md): ParentID = FROM entity, ChildID = TO entity. // For `Module.OrderLine_Order` (`FROM OrderLine TO Order`), context=Order → dest=OrderLine (parent side). +// xpathAssocRunRe matches a run of two or more slash-joined qualified names +// (Module.Name/Module.Name[/...]) — an association/entity path. Single qualified +// names (one-hop `[Mod.Assoc = $x]`) and unqualified attribute steps (`Mod.Assoc/Attr`) +// don't match, so they're never rewritten. +var xpathAssocRunRe = regexp.MustCompile(`[A-Za-z_]\w*\.[A-Za-z_]\w*(?:/[A-Za-z_]\w*\.[A-Za-z_]\w*)+`) + +// expandXPathAssociationPath rewrites an XPath constraint so that consecutive +// association segments get the intermediate entity Mendix requires between them: +// `[Mod.Assoc1/Mod.Assoc2 = $x]` → `[Mod.Assoc1/Mod.Entity/Mod.Assoc2 = $x]`. Without +// this, mxbuild reads the second association as an entity and fails with CE1613 +// ("selected entity ... no longer exists"). +// +// The rewrite is safe by construction: it only touches runs of slash-joined qualified +// names anchored at an association from the datasource entity, and inserts an entity +// only between two segments that both resolve as associations. The already-correct +// full form (where the middle segment is an entity, not an association), single-hop +// constraints, attribute paths, and anything unresolvable are returned verbatim. +func (pb *pageBuilder) expandXPathAssociationPath(constraint, contextEntity string) string { + if contextEntity == "" || !strings.Contains(constraint, "/") { + return constraint + } + return xpathAssocRunRe.ReplaceAllStringFunc(constraint, func(run string) string { + segs := strings.Split(run, "/") + // Only expand a run that starts with an association reachable from the + // datasource entity; otherwise leave it untouched (it may be an unrelated + // qualified path we don't understand). + if pb.resolveAssociationDestination(segs[0], contextEntity) == "" { + return run + } + out := make([]string, 0, len(segs)*2) + ctx := contextEntity + for i, seg := range segs { + dest := pb.resolveAssociationDestination(seg, ctx) + out = append(out, seg) + if dest == "" { + // Not an association from here — treat as an entity segment. + ctx = seg + continue + } + // Association: if the next segment is also an association (i.e. the + // intermediate entity was omitted), insert the destination entity. + if i+1 < len(segs) { + next := segs[i+1] + if next != dest && pb.resolveAssociationDestination(next, dest) != "" { + out = append(out, dest) + } + } + ctx = dest + } + return strings.Join(out, "/") + }) +} + func (pb *pageBuilder) resolveAssociationDestination(assocQN, contextEntity string) string { if assocQN == "" { return "" diff --git a/mdl/executor/cmd_pages_builder_v3_widgets.go b/mdl/executor/cmd_pages_builder_v3_widgets.go index 9b5c15681..d6fecd936 100644 --- a/mdl/executor/cmd_pages_builder_v3_widgets.go +++ b/mdl/executor/cmd_pages_builder_v3_widgets.go @@ -179,8 +179,9 @@ func (pb *pageBuilder) buildColumnSpecFromAST(child *ast.WidgetV3) (*backend.Dat for _, grandchild := range child.Children { if filterWidgetID := dataGridFilterWidgetID(grandchild.Type); filterWidgetID != "" { fw, err := pb.widgetBackend.BuildFilterWidget(backend.FilterWidgetSpec{ - WidgetID: filterWidgetID, - FilterName: grandchild.Name, + WidgetID: filterWidgetID, + FilterName: grandchild.Name, + VisibilityRules: resolveWidgetVisibilityRules(pb.backend.Path(), filterWidgetID), }, pb.backend.Path()) if err != nil { return nil, mdlerrors.NewBackend("build column filter widget", err) diff --git a/mdl/executor/editorconfig_extract.go b/mdl/executor/editorconfig_extract.go new file mode 100644 index 000000000..80838ff1c --- /dev/null +++ b/mdl/executor/editorconfig_extract.go @@ -0,0 +1,495 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "path/filepath" + "regexp" + "strings" + "sync" + + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/modelsdk/widgets/mpk" +) + +// editorConfigExtractStats reports how much of a widget's editorConfig.js the +// extractor could lift into declarative WidgetVisibilityRules — the coverage +// number that tells `check`/serialization how far to trust the rules and when +// to fall back to mxbuild's update-widgets. +type editorConfigExtractStats struct { + TotalHideCalls int // hidePropertyIn + hidePropertiesIn call sites seen + Recognized int // lifted into a top-level WidgetVisibilityRule + SkippedNested int // object-list-nested (e.g. hidePropertyIn(...,"columns",n,"key")) + SkippedComplex int // ternary/compound guard, or an alias we couldn't resolve +} + +// hideCallRE locates a hidePropertyIn / hidePropertiesIn call and captures its +// (balanced-enough) argument list. hideNestedPropertiesIn is deliberately not +// matched — it only ever targets object-list items (Phase 2). +var hideCallRE = regexp.MustCompile(`hidePropert(?:y|ies)In\(`) + +// aliasAssignRE finds `IDENT=OBJ.PROP` (a `var x=e.selection`-style alias). +// Resolution is scoped to the enclosing function body (see enclosingAliases), +// because minified editorConfig reuses single-letter identifiers across scopes. +var aliasAssignRE = regexp.MustCompile(`([A-Za-z_$][\w$]*)=([A-Za-z_$][\w$]*)\.([A-Za-z_$][\w$]*)`) + +// stringLitRE matches a double-quoted JS string literal (no escapes in the +// property/enum keys we care about). +var stringLitRE = regexp.MustCompile(`"([^"\\]*)"`) + +// visibilityCache memoizes extracted rules per (projectPath, widgetID) so the +// editorConfig.js is parsed once per build session, not per widget instance. +var ( + visibilityCache = map[string][]types.WidgetVisibilityRule{} + visibilityCacheMu sync.Mutex +) + +// WidgetVisibilityRules returns the editorConfig-derived property-visibility rules +// for a widget installed in the given project (nil when the .mpk or its editor +// config can't be found). Exported for the `mxcli widget describe` command, which +// surfaces the dynamic property rules mxcli discovered for a project's widget. +func WidgetVisibilityRules(projectPath, widgetID string) []types.WidgetVisibilityRule { + return resolveWidgetVisibilityRules(projectPath, widgetID) +} + +// ExtractWidgetVisibilityStats lifts a widget's editorConfig visibility rules from +// a specific .mpk path and also returns the extractor's coverage stats (how many +// hide-calls were recognized vs skipped). Exported for `mxcli widget describe` to +// report extraction coverage. +func ExtractWidgetVisibilityStats(mpkPath, widgetID string) ([]types.WidgetVisibilityRule, int, int) { + js, err := mpk.ReadEditorConfig(mpkPath, widgetID) + if err != nil || js == "" { + return nil, 0, 0 + } + rules, stats := extractVisibilityRulesFromJS(js) + return rules, stats.Recognized, stats.TotalHideCalls +} + +// resolveWidgetVisibilityRules returns the property-visibility rules for a +// widget, lifted from its installed .mpk's editorConfig.js. Used to enrich +// built-in widget definitions (DataGrid2, Gallery, …) — which the .def.json +// generator skips — with the version-specific applicability logic of the Data +// Widgets package actually installed in the project. Returns nil when the .mpk +// or its editor config can't be found (degrades to "no rules" → template +// default, exactly today's behaviour). Best-effort: see extractVisibilityRules- +// FromJS for coverage limits. +func resolveWidgetVisibilityRules(projectPath, widgetID string) []types.WidgetVisibilityRule { + if projectPath == "" || widgetID == "" { + return nil + } + key := projectPath + "\x00" + widgetID + visibilityCacheMu.Lock() + if r, ok := visibilityCache[key]; ok { + visibilityCacheMu.Unlock() + return r + } + visibilityCacheMu.Unlock() + + // getProjectPath() yields the .mpr file path; FindMPK wants the directory + // that contains widgets/. + projectDir := projectPath + if strings.EqualFold(filepath.Ext(projectDir), ".mpr") { + projectDir = filepath.Dir(projectDir) + } + + var rules []types.WidgetVisibilityRule + if mpkPath, err := mpk.FindMPK(projectDir, widgetID); err == nil && mpkPath != "" { + rules = extractVisibilityRulesFromMPK(mpkPath, widgetID) + } + + visibilityCacheMu.Lock() + visibilityCache[key] = rules + visibilityCacheMu.Unlock() + return rules +} + +// extractVisibilityRulesFromMPK reads a widget's editorConfig.js from its .mpk +// and lifts its property-visibility rules. Returns nil when the widget ships no +// editor config. Used both at build time (via resolveWidgetVisibilityRules) and +// at .def.json generation time, so generated definitions carry the rules and the +// hand-transcribed table can retire. +func extractVisibilityRulesFromMPK(mpkPath, widgetID string) []types.WidgetVisibilityRule { + js, err := mpk.ReadEditorConfig(mpkPath, widgetID) + if err != nil || js == "" { + return nil + } + rules, _ := extractVisibilityRulesFromJS(js) + return rules +} + +// extractVisibilityRulesFromJS lifts top-level property-hide rules from a +// widget's compiled editorConfig.js into declarative WidgetVisibilityRules. +// +// It recognizes the dominant `getProperties` idioms — `"V"===ref && hide(...)`, +// `"V"!==ref && hide(...)`, `ref && hide(...)`, `ref || hide(...)`, and +// `ref ? hide(...) : …` — where `ref` is `obj.prop` or a locally-aliased +// identifier resolved within the enclosing function scope. Everything it cannot +// lift (object-list-nested hides, compound/computed guards, unresolved aliases) +// is counted in the returned stats so callers can gauge coverage; unrecognized +// hides simply produce no rule, which degrades safely to "not hidden". +func extractVisibilityRulesFromJS(js string) ([]types.WidgetVisibilityRule, editorConfigExtractStats) { + var rules []types.WidgetVisibilityRule + var stats editorConfigExtractStats + seen := map[string]bool{} // dedupe propertyKey+condition + + for _, loc := range hideCallRE.FindAllStringIndex(js, -1) { + stats.TotalHideCalls++ + callStart, argsOpen := loc[0], loc[1] // argsOpen points just past '(' + args, ok := balancedArgs(js, argsOpen) + if !ok { + stats.SkippedComplex++ + continue + } + keys, nested := hideTargetKeys(args) + if nested { + stats.SkippedNested++ + continue + } + if len(keys) == 0 { + stats.SkippedComplex++ + continue + } + cond, ok := parseGuard(js, callStart) + if !ok { + stats.SkippedComplex++ + continue + } + stats.Recognized++ + for _, key := range keys { + sig := key + "\x00" + cond.PropertyKey + cond.Operator + cond.Value + if seen[sig] { + continue + } + seen[sig] = true + c := cond // copy per rule + rules = append(rules, types.WidgetVisibilityRule{PropertyKey: key, HiddenWhen: &c}) + } + } + return rules, stats +} + +// hideTargetKeys returns the property key(s) a hide call targets and whether the +// call is object-list-nested (which we skip). A top-level hidePropertyIn has a +// single string arg (the key); a top-level hidePropertiesIn has one array of +// string keys. A `"columns"`/`"..."`-prefixed string arg followed by more args +// marks the nested form. +func hideTargetKeys(args string) (keys []string, nested bool) { + parts := splitTopLevelCommas(args) + // Collect string-literal positional args and any array literal. + var stringArgs []string + var arrayKeys []string + for _, p := range parts { + p = strings.TrimSpace(p) + if strings.HasPrefix(p, "[") { + for _, m := range stringLitRE.FindAllStringSubmatch(p, -1) { + arrayKeys = append(arrayKeys, m[1]) + } + continue + } + if m := stringLitRE.FindStringSubmatch(p); m != nil && strings.HasPrefix(p, `"`) { + stringArgs = append(stringArgs, m[1]) + } + } + if len(arrayKeys) > 0 { + // hidePropertiesIn(obj, obj, [keys]) — nested if a leading string arg + // (e.g. "columns") also appears. + if len(stringArgs) > 0 { + return nil, true + } + return arrayKeys, false + } + switch len(stringArgs) { + case 1: + return stringArgs, false // hidePropertyIn(obj, obj, "key") + case 0: + return nil, false + default: + return nil, true // "columns","key" etc. — nested + } +} + +// nsPrefixRE matches the widget-editor namespace prefix before a hide call — the +// minifier names it per widget (`_.`, `D.`, `M.`, `j.`, `A.`, …), so it must be +// matched generically rather than hard-coded to `_.`. +var nsPrefixRE = regexp.MustCompile(`[A-Za-z_$][\w$]*\.$`) + +// parseGuard reads the guard expression immediately preceding a hide call and +// converts it to a WidgetVisibilityCondition. callStart points at the hide +// function name; the connector just before it is `&&`, `||`, or `?`. +func parseGuard(js string, callStart int) (types.WidgetVisibilityCondition, bool) { + pre := strings.TrimRight(js[:callStart], " ") + // Strip the widget-editor namespace prefix (any `.`, not just `_.`). + if loc := nsPrefixRE.FindStringIndex(pre); loc != nil { + pre = pre[:loc[0]] + } + pre = strings.TrimRight(pre, " ") + // Strip an optional grouping paren: `cond && ( hide(...), … )` groups several + // hides under one condition; the first hide sits right after the `(`. + if strings.HasSuffix(pre, "(") { + pre = strings.TrimRight(pre[:len(pre)-1], " ") + } + // Identify the connector. + var falsy bool // || connector or !prefix ⇒ hide when guard is falsy + switch { + case strings.HasSuffix(pre, "&&"): + pre = pre[:len(pre)-2] + case strings.HasSuffix(pre, "?"): + pre = pre[:len(pre)-1] + case strings.HasSuffix(pre, "||"): + pre = pre[:len(pre)-2] + falsy = true + default: + return types.WidgetVisibilityCondition{}, false + } + guard, boundary := lastGuardExpr(pre) + guard = stripReturnPrefix(guard) // getProperties' first statement is `return && hide…` + if guard == "" { + return types.WidgetVisibilityCondition{}, false + } + // Skip guards nested inside a larger expression. A clean statement-level guard + // is bounded by a statement separator (`,`, `;`, `{`, or start-of-input); a + // boundary of `(`/`?`/`:`/`&`/`|` means the guard is one operand of a compound + // or ternary condition (e.g. `"web"===r ? (e.advanced || hide(...))`), of which + // we'd capture only a fragment — producing a WRONG rule that over-fires. Better + // to emit no rule (→ "not hidden" → template default), which is safe. + switch boundary { + case 0, ',', ';', '{': + // clean + default: + return types.WidgetVisibilityCondition{}, false + } + aliases := enclosingAliases(js, callStart) + return guardToCondition(guard, falsy, aliases) +} + +// stripReturnPrefix removes a leading `return` keyword from a guard expression. +// A widget's getProperties body often starts `return && hide(…), …`, so +// the first guard is prefixed with `return` (minified: `return"none"===…`). +func stripReturnPrefix(g string) string { + g = strings.TrimSpace(g) + const kw = "return" + if strings.HasPrefix(g, kw) { + rest := g[len(kw):] + // Only strip when `return` is a standalone keyword, not the start of an + // identifier like `returnValue`. + if rest == "" || !isIdentByte(rest[0]) { + return strings.TrimSpace(rest) + } + } + return g +} + +// isIdentByte reports whether b can appear in a JS identifier. +func isIdentByte(b byte) bool { + return b == '_' || b == '$' || + (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || (b >= '0' && b <= '9') +} + +// lastGuardExpr returns the balanced expression ending at the end of `pre`, +// bounded by the previous top-level separator, and the boundary byte that +// terminated it (0 for start-of-input). The boundary lets the caller reject +// guards nested inside a compound/ternary expression. +func lastGuardExpr(pre string) (string, byte) { + depth := 0 + var boundary byte + i := len(pre) - 1 + for ; i >= 0; i-- { + c := pre[i] + switch c { + case ')', ']', '}': + depth++ + case '(', '[', '{': + if depth == 0 { + boundary = c + goto done + } + depth-- + case ',', ';', ':', '?', '&', '|': + if depth == 0 { + boundary = c + goto done + } + } + } +done: + return strings.TrimSpace(pre[i+1:]), boundary +} + +var ( + eqCmpRE = regexp.MustCompile(`^"([^"]*)"===([A-Za-z_$][\w$.]*)$`) + neCmpRE = regexp.MustCompile(`^"([^"]*)"!==([A-Za-z_$][\w$.]*)$`) + eqCmpRE2 = regexp.MustCompile(`^([A-Za-z_$][\w$.]*)==="([^"]*)"$`) + neCmpRE2 = regexp.MustCompile(`^([A-Za-z_$][\w$.]*)!=="([^"]*)"$`) + refRE = regexp.MustCompile(`^(!?)([A-Za-z_$][\w$.]*)$`) +) + +// guardToCondition parses a single guard expression into a visibility +// condition, resolving a bare identifier through the scope's alias map. +func guardToCondition(guard string, falsy bool, aliases map[string]string) (types.WidgetVisibilityCondition, bool) { + // "V" === ref / ref === "V" + if m := eqCmpRE.FindStringSubmatch(guard); m != nil { + if key, ok := resolveRef(m[2], aliases); ok { + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: "eq", Value: m[1]}, true + } + return types.WidgetVisibilityCondition{}, false + } + if m := eqCmpRE2.FindStringSubmatch(guard); m != nil { + if key, ok := resolveRef(m[1], aliases); ok { + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: "eq", Value: m[2]}, true + } + return types.WidgetVisibilityCondition{}, false + } + if m := neCmpRE.FindStringSubmatch(guard); m != nil { + if key, ok := resolveRef(m[2], aliases); ok { + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: "ne", Value: m[1]}, true + } + return types.WidgetVisibilityCondition{}, false + } + if m := neCmpRE2.FindStringSubmatch(guard); m != nil { + if key, ok := resolveRef(m[1], aliases); ok { + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: "ne", Value: m[2]}, true + } + return types.WidgetVisibilityCondition{}, false + } + // bare ref (truthy) or !ref (falsy), combined with the connector polarity: + // ref && hide → hide when ref truthy + // ref || hide → hide when ref falsy (falsy==true here) + // !ref && hide → hide when ref falsy + // ref ? hide:… → hide when ref truthy + if m := refRE.FindStringSubmatch(guard); m != nil { + key, ok := resolveRef(m[2], aliases) + if !ok { + return types.WidgetVisibilityCondition{}, false + } + neg := (m[1] == "!") + wantFalsy := falsy != neg // XOR: || or ! flips polarity (both flips cancel) + op := "truthy" + if wantFalsy { + op = "falsy" + } + return types.WidgetVisibilityCondition{PropertyKey: key, Operator: op}, true + } + return types.WidgetVisibilityCondition{}, false +} + +// resolveRef turns a guard reference into a widget property key: `obj.prop` +// yields `prop`; a bare identifier is looked up in the scope alias map. A bare +// identifier with no alias (e.g. a computed local) is unresolvable. +func resolveRef(ref string, aliases map[string]string) (string, bool) { + if i := strings.LastIndexByte(ref, '.'); i >= 0 { + return ref[i+1:], true + } + if key, ok := aliases[ref]; ok { + return key, true + } + return "", false +} + +// enclosingAliases returns the `ident → property` aliases declared in the +// function body that encloses the hide call at pos, resolved by scanning back +// to the nearest unbalanced `{`. Scoping matters: minified editorConfig reuses +// identifiers like `r`/`n` across functions, so only same-scope `var r=e.prop` +// assignments are trustworthy. +func enclosingAliases(js string, pos int) map[string]string { + // Walk back to the enclosing block's opening brace. + depth := 0 + open := 0 + for i := pos - 1; i >= 0; i-- { + switch js[i] { + case '}', ')', ']': + depth++ + case '{', '(', '[': + if depth == 0 { + open = i + goto found + } + depth-- + } + } +found: + body := js[open:pos] + aliases := map[string]string{} + ambiguous := map[string]bool{} + for _, m := range aliasAssignRE.FindAllStringSubmatch(body, -1) { + ident, prop := m[1], m[3] + if ambiguous[ident] { + continue + } + if existing, ok := aliases[ident]; ok && existing != prop { + delete(aliases, ident) + ambiguous[ident] = true + continue + } + aliases[ident] = prop + } + return aliases +} + +// balancedArgs returns the argument-list text between the '(' just before +// `open` and its matching ')'. Handles nested (), [], {} and string literals. +func balancedArgs(js string, open int) (string, bool) { + depth := 1 + inStr := byte(0) + for i := open; i < len(js); i++ { + c := js[i] + if inStr != 0 { + if c == '\\' { + i++ + continue + } + if c == inStr { + inStr = 0 + } + continue + } + switch c { + case '"', '\'', '`': + inStr = c + case '(', '[', '{': + depth++ + case ')', ']', '}': + depth-- + if depth == 0 { + return js[open:i], true + } + } + } + return "", false +} + +// splitTopLevelCommas splits an argument list on commas that are not nested +// inside (), [], {}, or string literals. +func splitTopLevelCommas(args string) []string { + var parts []string + depth := 0 + inStr := byte(0) + start := 0 + for i := 0; i < len(args); i++ { + c := args[i] + if inStr != 0 { + if c == '\\' { + i++ + continue + } + if c == inStr { + inStr = 0 + } + continue + } + switch c { + case '"', '\'', '`': + inStr = c + case '(', '[', '{': + depth++ + case ')', ']', '}': + depth-- + case ',': + if depth == 0 { + parts = append(parts, args[start:i]) + start = i + 1 + } + } + } + parts = append(parts, args[start:]) + return parts +} diff --git a/mdl/executor/editorconfig_extract_test.go b/mdl/executor/editorconfig_extract_test.go new file mode 100644 index 000000000..3e8a062ab --- /dev/null +++ b/mdl/executor/editorconfig_extract_test.go @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "os" + "testing" + + "github.com/mendixlabs/mxcli/mdl/types" +) + +// find returns the first rule for the given property key, or nil. +func findRule(rules []types.WidgetVisibilityRule, key string) *types.WidgetVisibilityRule { + for i := range rules { + if rules[i].PropertyKey == key { + return &rules[i] + } + } + return nil +} + +// TestExtractVisibility_DataGrid runs the extractor against the real +// (minified) Data Widgets 3.10.0 DataGrid2 editorConfig.js and asserts it lifts +// the three top-level textTemplate hides that drive the #600 CE0463 — the ones +// the hand-transcribed table never covered — with the correct conditions. +func TestExtractVisibility_DataGrid(t *testing.T) { + js, err := os.ReadFile("testdata/datagrid.editorConfig.js") + if err != nil { + t.Fatalf("read fixture: %v", err) + } + rules, stats := extractVisibilityRulesFromJS(string(js)) + t.Logf("coverage: %d/%d hide calls recognized (nested=%d complex=%d), %d rules", + stats.Recognized, stats.TotalHideCalls, stats.SkippedNested, stats.SkippedComplex, len(rules)) + + want := []struct { + key, condKey, op, val string + }{ + {"clearSelectionButtonLabel", "itemSelection", "ne", "Multi"}, + {"singleSelectionColumnLabel", "itemSelection", "ne", "Single"}, + {"loadMoreButtonCaption", "pagination", "ne", "loadMore"}, + } + for _, w := range want { + r := findRule(rules, w.key) + if r == nil || r.HiddenWhen == nil { + t.Errorf("%s: no rule extracted", w.key) + continue + } + c := r.HiddenWhen + if c.PropertyKey != w.condKey || c.Operator != w.op || c.Value != w.val { + t.Errorf("%s: got {%s %s %q}, want {%s %s %q}", + w.key, c.PropertyKey, c.Operator, c.Value, w.condKey, w.op, w.val) + } + } + + // A couple more expected top-level lifts (non-textTemplate, but proves the + // idioms generalize): emptyPlaceholder hidden when showEmptyPlaceholder=="none". + if r := findRule(rules, "emptyPlaceholder"); r == nil || r.HiddenWhen == nil || + r.HiddenWhen.PropertyKey != "showEmptyPlaceholder" || r.HiddenWhen.Operator != "eq" || r.HiddenWhen.Value != "none" { + t.Errorf("emptyPlaceholder rule missing/wrong: %+v", r) + } +} + +// TestExtractVisibility_VideoPlayerPattern reproduces the hand-transcribed +// VideoPlayer rule: `"expression"===e.type && hidePropertiesIn(t,e,["videoUrl","posterUrl"])`. +func TestExtractVisibility_VideoPlayerPattern(t *testing.T) { + js := `exports.getProperties=function(e,t){"expression"===e.type&&_.hidePropertiesIn(t,e,["videoUrl","posterUrl"]);return t}` + rules, _ := extractVisibilityRulesFromJS(js) + for _, key := range []string{"videoUrl", "posterUrl"} { + r := findRule(rules, key) + if r == nil || r.HiddenWhen == nil || r.HiddenWhen.PropertyKey != "type" || + r.HiddenWhen.Operator != "eq" || r.HiddenWhen.Value != "expression" { + t.Errorf("%s: got %+v, want hidden when type==expression", key, r) + } + } +} + +// TestExtractVisibility_TimelinePattern reproduces the hand-transcribed Timeline +// rule, which uses a ternary truthy guard: +// `e.customVisualization ? hidePropertiesIn(t,e,["title","description"]) : x`. +func TestExtractVisibility_TimelinePattern(t *testing.T) { + js := `exports.getProperties=function(e,t){e.customVisualization?_.hidePropertiesIn(t,e,["title","description","timeIndication"]):null;return t}` + rules, _ := extractVisibilityRulesFromJS(js) + for _, key := range []string{"title", "description", "timeIndication"} { + r := findRule(rules, key) + if r == nil || r.HiddenWhen == nil || r.HiddenWhen.PropertyKey != "customVisualization" || + r.HiddenWhen.Operator != "truthy" { + t.Errorf("%s: got %+v, want hidden when customVisualization truthy", key, r) + } + } +} + +// TestExtractVisibility_ScopedAlias proves alias resolution is scoped: the same +// identifier `r` aliased to different properties in two functions must not leak. +func TestExtractVisibility_ScopedAlias(t *testing.T) { + js := `f1=function(e,t){var r=Object.keys(e);return r};` + + `f2=function(e,t){var r=t.itemSelection;"Multi"!==r&&_.hidePropertyIn(e,t,"clearSelectionButtonLabel")}` + rules, _ := extractVisibilityRulesFromJS(js) + r := findRule(rules, "clearSelectionButtonLabel") + if r == nil || r.HiddenWhen == nil || r.HiddenWhen.PropertyKey != "itemSelection" || + r.HiddenWhen.Operator != "ne" || r.HiddenWhen.Value != "Multi" { + t.Fatalf("scoped alias not resolved: %+v", r) + } +} + +// TestExtractVisibility_SkipsNested confirms object-list-nested hides +// (hidePropertyIn(...,"columns",n,"key")) are not lifted as top-level rules. +func TestExtractVisibility_SkipsNested(t *testing.T) { + js := `g=function(e,t){e.columns.forEach(function(r,n){e.columnsSortable||_.hidePropertyIn(t,e,"columns",n,"sortable")})}` + rules, stats := extractVisibilityRulesFromJS(js) + if findRule(rules, "sortable") != nil { + t.Error("nested column hide must not produce a top-level rule") + } + if stats.SkippedNested == 0 { + t.Error("expected the nested hide to be counted as skipped") + } +} + +// TestExtractVisibility_NamespaceAndReturn covers the guard forms the filter / +// Gallery / TreeNode editorConfigs use that a `_.`-only, no-`return` parser missed: +// arbitrary namespace prefixes (D./M./j.), a leading `return`, grouping parens +// `cond&&(hide,…)`, and `||` falsy guards. +func TestExtractVisibility_NamespaceAndReturn(t *testing.T) { + cases := []struct { + name, js, key, condKey, op, val string + }{ + {"return + || falsy, namespace j.", `f=function(e,t){return e.adjustable||j.hidePropertyIn(t,e,"screenReaderButtonCaption")}`, + "screenReaderButtonCaption", "adjustable", "falsy", ""}, + {"eq && namespace j.", `f=function(e,t){x,"auto"===e.attrChoice&&j.hidePropertyIn(t,e,"attributes")}`, + "attributes", "attrChoice", "eq", "auto"}, + {"return + eq, minified no space, namespace D.", `f=function(e,t){return"none"===e.showEmptyPlaceholder&&D.hidePropertyIn(t,e,"emptyPlaceholder")}`, + "emptyPlaceholder", "showEmptyPlaceholder", "eq", "none"}, + {"grouping paren cond&&(hide,…), namespace D.", `f=function(e,t){y,"None"===e.itemSelection&&(D.hidePropertyIn(t,e,"autoSelect"),D.hidePropertyIn(t,e,"x"))}`, + "autoSelect", "itemSelection", "eq", "None"}, + {"ternary truthy, namespace M.", `f=function(e,t){z,e.customVisualization?M.hidePropertiesIn(t,e,["title"]):null}`, + "title", "customVisualization", "truthy", ""}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + rules, _ := extractVisibilityRulesFromJS(c.js) + r := findRule(rules, c.key) + if r == nil || r.HiddenWhen == nil { + t.Fatalf("no rule for %s (rules=%+v)", c.key, rules) + } + if r.HiddenWhen.PropertyKey != c.condKey || r.HiddenWhen.Operator != c.op || r.HiddenWhen.Value != c.val { + t.Errorf("%s: got {%s %s %q}, want {%s %s %q}", c.key, + r.HiddenWhen.PropertyKey, r.HiddenWhen.Operator, r.HiddenWhen.Value, c.condKey, c.op, c.val) + } + }) + } +} diff --git a/mdl/executor/roundtrip_doctype_test.go b/mdl/executor/roundtrip_doctype_test.go index 2942fd804..e613d8832 100644 --- a/mdl/executor/roundtrip_doctype_test.go +++ b/mdl/executor/roundtrip_doctype_test.go @@ -117,9 +117,11 @@ var scriptModuleDeps = map[string][]moduleDep{ // not actionable on this branch. var scriptSkipList = map[string]string{} -// scriptKnownCEErrors lists CE error codes that are expected for specific scripts. -// These are syntax showcase scripts that intentionally omit entities, constants, -// headers etc. that full validation requires. +// scriptKnownCEErrors lists CE error codes (or, where an error carries no CE +// code, a distinctive substring of its message) that are expected for specific +// scripts. These are syntax showcase scripts that intentionally omit entities, +// constants, headers etc. that full validation requires. Matched by substring +// against each `[error]` line (see allErrorsKnown). var scriptKnownCEErrors = map[string][]string{ "03-page-examples.mdl": { "CE3637", // Data view listen to gallery in sibling layout-grid column — Mendix scoping limitation @@ -127,6 +129,18 @@ var scriptKnownCEErrors = map[string][]string{ "06b-soap-examples.mdl": { "CE1613", // Dangling service/mapping refs — no web service defined in the test project }, + "17-custom-widget-examples.mdl": { + // The IMAGE showcases author bare/default (static-image-mode) Image + // widgets to demonstrate the IMAGE keyword's basic/dimensions/onclick + // syntax. MDL's image widget exposes no property to bind a static image + // resource (only imageUrl mode), so a default Image is legitimately + // "No image selected." — exactly what a freshly-dropped Image shows in + // Studio Pro until you pick one. Not an mxcli defect: the widget BSON is + // valid. It previously passed only because a stale + // `Atlas_Core.Content.Mendix` template default masked it — that default + // caused CE0463 and was removed in the Image CE0463 fix (549c44f). + "No image selected.", + }, } // TestMxCheck_DoctypeScripts executes each doctype-tests/*.mdl example script diff --git a/mdl/executor/testdata/datagrid.editorConfig.js b/mdl/executor/testdata/datagrid.editorConfig.js new file mode 100644 index 000000000..a62245e7e --- /dev/null +++ b/mdl/executor/testdata/datagrid.editorConfig.js @@ -0,0 +1 @@ +"use strict";function e(t){return e="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e},e(t)}function t(e,t){var r=Object.keys(e);if(Object.getOwnPropertySymbols){var n=Object.getOwnPropertySymbols(e);t&&(n=n.filter(function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable})),r.push.apply(r,n)}return r}function r(e){for(var r=1;re.length)&&(t=e.length);for(var r=0,n=Array(t);r0&&void 0!==arguments[0]?arguments[0]:"";try{return e.split(";").reduce(function(e,t){var r=t.split(":");2===r.length&&(e[r[0].trim().replace(/(-.)/g,function(e){return e[1].toUpperCase()})]=r[1].trim());return e},{})}catch(e){return{}}},m}var g,v,P={},w={};function C(){return g||(g=1,Object.defineProperty(w,"__esModule",{value:!0})),w}function S(){return v||(v=1,function(e){var t=P&&P.__createBinding||(Object.create?function(e,t,r,n){void 0===n&&(n=r);var o=Object.getOwnPropertyDescriptor(t,r);o&&!("get"in o?!t.__esModule:o.writable||o.configurable)||(o={enumerable:!0,get:function(){return t[r]}}),Object.defineProperty(e,n,o)}:function(e,t,r,n){void 0===n&&(n=r),e[n]=t[r]}),r=P&&P.__exportStar||function(e,r){for(var n in e)"default"===n||Object.prototype.hasOwnProperty.call(r,n)||t(r,e,n)};Object.defineProperty(e,"__esModule",{value:!0}),r(C(),e)}(P)),P}var O,F,A,j={},D={};function I(){if(O)return D;function e(e,r,n,o,i){t(function(e,t,r){return r.splice(t,1)},e,n,o,i)}function t(e,r,n,o,i){r.forEach(function(r){var a;r.propertyGroups&&t(e,r.propertyGroups,n,o,i),null===(a=r.properties)||void 0===a||a.forEach(function(r,a,u){r.key===n&&(void 0===o||void 0===i?e(r,a,u):r.objects?t(e,r.objects[o].properties,i):r.properties&&t(e,r.properties[o],i))})})}return O=1,Object.defineProperty(D,"__esModule",{value:!0}),D.hidePropertyIn=e,D.hidePropertiesIn=function(e,r,n){n.forEach(function(r){return t(function(e,t,r){return r.splice(t,1)},e,r,void 0,void 0)})},D.hideNestedPropertiesIn=function(t,r,n,o,i){i.forEach(function(i){return e(t,r,n,o,i)})},D.changePropertyIn=function(e,r,n,o,i,a){t(n,e,o,i,a)},D.transformGroupsIntoTabs=function(e){var t=[];e.forEach(function(e){e.propertyGroups&&(t.push.apply(t,i(e.propertyGroups)),e.propertyGroups=[])}),e.push.apply(e,t)},D.moveProperty=function(e,t,r){e>=0&&t>=0&&e