Skip to content

Add Playground e2e tests and a "try in Playground" button on PRs - #56

Draft
roborourke wants to merge 3 commits into
claude/query-string-validation-0rgckrfrom
claude/e2e-playground-0rgckr
Draft

Add Playground e2e tests and a "try in Playground" button on PRs#56
roborourke wants to merge 3 commits into
claude/query-string-validation-0rgckrfrom
claude/e2e-playground-0rgckr

Conversation

@roborourke

@roborourke roborourke commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #55. Base is claude/query-string-validation-0rgckr, not main — six of the tests here are regression coverage for that fix and would fail against main. GitHub will retarget this to main automatically once #55 merges. Review #55 first; the diff below is this PR's own changes only.

The repo had no tests. The filter blocks are almost entirely server-side behaviour driven by the query string, which is cheap to break and expensive to notice.

The harness

Playwright driving WordPress Playground via @wp-playground/cli, following hm-query-loop's migration off wp-env. Playground boots in process, so:

  • No Docker, and no environment to start or stop — npm run test:e2e is the whole thing.
  • PHP and WordPress versions become parameters, so CI runs a PHP 8.2/8.3 × WP 6.8/latest matrix off the same blueprint used locally.
  • The port is derived from a hash of the working directory, so worktrees don't collide.

blueprint.json describes the environment once and is shared between the test run and npm run playground:start (manual poking, no tests). Fixtures are seeded by tests/seed.php, and tests/mu-plugins/register-test-content.php registers the post types and taxonomies the tests filter against — including deliberately private ones, so the suite can prove they stay invisible.

Page Query ID Contains
/taxonomy-filter/ 1 Taxonomy filter (select)
/taxonomy-checkboxes/ 2 Taxonomy filter (checkboxes)
/post-type-filter/ 3 Post type filter + core search block

Unfiled Post belongs to no category on purpose, so "filtered" is always distinguishable from "unfiltered". Secret One is published in the private qf_secret post type and must never reach the front end.

The tests

16 tests in three specs:

  • Taxonomy filter — options list non-empty terms; selecting a term filters the loop and updates the URL; a term in the URL filters on first render and the control reflects it; returning to All clears it; an unknown slug returns nothing rather than everything; checkbox mode accumulates terms into alpha,beta.
  • Post type filter — the control lists the loop's post types; a public post type in the URL switches the loop.
  • Search — the core search block is rewritten to the loop's own query var (query-3-s) and narrows the loop; a term in the URL is reflected in the input.
  • Regression coverage for Only accept public, registered taxonomies and post types from the query string #55 — a private taxonomy, a private post type, an unregistered post type, a mixed public/private list, and an array-shaped value (?query-3-post_type[]=, previously an uncaught TypeError) are all ignored, and Secret One never appears.

Filtering navigates through the Interactivity API router, which swaps the loop region in without a document navigation, so the loop.expectTitles() helper polls the rendered titles rather than waiting on a load event.

Playground preview button on PRs

pr-playground-preview.yml adds a Preview in WordPress Playground button to every PR description, booting that PR's build with demo content already seeded — a reviewer can try a change without a local checkout.

build/ is gitignored, so the preview needs somewhere to get compiled assets: the workflow builds the PR and force-pushes the result to a throwaway pr-<n>-built branch, which Playground installs from as a zip. pr-cleanup.yml deletes that branch when the PR closes.

Two deliberate differences from the hm-query-loop version:

  • Fork PRs are skipped (if: head.repo.full_name == github.repository). A fork PR's token is read-only and cannot push the preview branch, so the step would hard-fail rather than being unavailable.
  • The PR title never reaches the shell. hm-query-loop interpolates github.event.pull_request.title into a sed command; a PR title is attacker-controlled, so that is a command injection vector. This stamps 0.0.0-pr<number> instead, with the number passed via env.

Testing

Everything that can be checked without booting WordPress has been:

  • npm ci clean from the regenerated lockfile; npm run build succeeds.
  • npx playwright test --list collects all 16 tests across 3 files.
  • wp-scripts lint-js clean on the harness and specs.
  • composer phpcs clean; php -l clean on the seed and mu-plugin, and on the PHP embedded in the preview workflow's blueprint.
  • All three workflows parse as YAML; both blueprints parse as JSON, including the one embedded in the workflow after expression substitution.
  • Action SHAs pinned, resolved from upstream: actions/checkout v7.0.1, actions/setup-node v7.0.0, actions/upload-artifact v7.0.1, daun/playwright-report-comment v4.1.0, WordPress/action-wp-playground-pr-preview v3.

CI: 16/16 passing on all four matrix legs (PHP 8.2/8.3 × WP 6.8/latest), plus PHPCS and the preview job. Playground boots in ~20s per leg.

The suite could not be run locally — this environment's network policy blocks wordpress.org, so Playground cannot download WordPress — so the first run was on CI, and it took three rounds to get green. Both fixes were to the fixtures, not the plugin:

  1. WordPress ships a "Hello world!" post, so every assertion on the unfiltered result set saw five titles instead of four. tests/seed.php now clears pre-existing posts and pages before seeding.
  2. wp_insert_post() assigns the default category when a post is created without one, so "Unfiled Post" — whose entire purpose is to belong to no category — was silently filed under Uncategorized, making it non-empty and adding a third option to the term list. Now cleared explicitly. Separately, the checkbox labels wrap their text across lines and toHaveText compares textContent verbatim, so that assertion compares innerText.

The preview button path is also verified end to end: pr-56-built carries the compiled assets and is stamped 0.0.0-pr56.

One thing to fold in on merge

#54 adds .gitattributes, which controls what ships in the release ZIP. It doesn't exist on this branch, so adding the entries here would conflict. Whichever of #54 / this PR merges second needs these four lines added:

/blueprint.json export-ignore
/global-setup.js export-ignore
/global-teardown.js export-ignore
/playwright.config.js export-ignore

(/tests is already covered by #54's list.) Happy to push that once the merge order is settled.


Generated by Claude Code

Open WordPress Playground Preview

Generated by Claude Code

There were no tests. The filter blocks are almost entirely server-side
behaviour driven by the query string, which is exactly the kind of thing
that is cheap to break and expensive to notice.

Follow the hm-query-loop harness: Playwright driving WordPress Playground
via @wp-playground/cli, rather than wp-env and Docker. Playground boots in
process, so there is no environment to start or stop, and the PHP and
WordPress versions are just parameters — CI runs a PHP 8.2/8.3 x WP
6.8/latest matrix off the same blueprint developers use locally.

blueprint.json describes the environment once and is shared between the
test run and `npm run playground:start`. Fixture content is seeded by
tests/seed.php, and tests/mu-plugins/register-test-content.php registers
the post types and taxonomies the tests filter against, including
deliberately private ones so the suite can prove they stay invisible.

16 tests covering the taxonomy filter (select, checkbox, multi-term, URL
round-tripping), the post type filter, the search block, and the query
string validation added in the previous commit.

Also add a Playground preview button to every PR. build/ is gitignored,
so pr-playground-preview.yml builds the PR and pushes the result to a
throwaway pr-<n>-built branch that Playground installs from; pr-cleanup
deletes the branch when the PR closes. Unlike the hm-query-loop version
this skips fork PRs, whose read-only token cannot push that branch, and
keeps the PR title out of the shell — it is attacker controlled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015J9z6XLV1hT1FB1BSXPczo
@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown

Playwright — PHP 8.3 / WP latest

passed  16 passed

Details

stats  16 tests across 3 suites
duration  22.1 seconds
commit  1caf181

Every unfiltered assertion failed on the same diff: WordPress ships with
a "Hello world!" post, so the loop returned five titles where the
fixtures define four. It also sat in Uncategorized, which put a third
option in the taxonomy filter's derived term list.

Delete all pre-existing posts and pages before seeding so the fixtures
are the only content the loops can return.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015J9z6XLV1hT1FB1BSXPczo
github-actions Bot added a commit that referenced this pull request Aug 11, 2026
Two things kept the taxonomy term list assertions red.

wp_insert_post() assigns the default category when a post is created
without one, so "Unfiled Post" — whose whole purpose is to belong to no
category — landed in Uncategorized. That made Uncategorized non-empty and
put a third option in the filter's derived term list. Clear it explicitly.

The checkbox labels wrap their text across lines in the template, and
toHaveText compares textContent verbatim, so the tabs and newlines came
through. Compare innerText instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015J9z6XLV1hT1FB1BSXPczo
github-actions Bot added a commit that referenced this pull request Aug 11, 2026
github-actions Bot added a commit that referenced this pull request Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants