docs: Add Snapshots EA product documentation#17327
Conversation
Add three new pages for the Snapshots visual regression testing feature (Early Adopter). Covers the overview and prerequisites, upload setup with CI workflow, and the review/approval workflow with project settings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
|
|
||
| ## Status Checks | ||
|
|
||
| After you upload snapshots from a pull request branch, a **Snapshot Testing** GitHub Check appears on the pull request. If no snapshots changed, the check passes. If any snapshots were added, removed, or modified, the check fails and requires approval. |
There was a problem hiding this comment.
Should we explain that the default is that added snapshots don't fail here?
There was a problem hiding this comment.
will have that in the settings section and maybe just a link to "if you want to change your failure settings..."
| - **Wipe** — Drag a slider across the image to compare base and current branch. | ||
| - **Onion** — Overlay both images with an adjustable opacity slider. | ||
|
|
||
| IMAGE: Sentry Snapshot comparison viewer showing a modified image with split diff |
There was a problem hiding this comment.
Can we make this a gif of using the onion mode on an ogre ? that would make my day.
There was a problem hiding this comment.
lol we'll save that for the EA blog
| ```json | ||
| { | ||
| "display_name": "Billing Page", | ||
| "group": "Settings" |
There was a problem hiding this comment.
we can also add other fields as extras here, they will become tags.
For example:
"device": "id:pixel_5",
"nightMode": true
There was a problem hiding this comment.
didn't want to put that until they're actually used downstream (my understanding is that it's not in any way atm)
| branches: [main] | ||
| pull_request: | ||
|
|
||
| concurrency: |
There was a problem hiding this comment.
Let's leave the concurrency out of this snippet.
There was a problem hiding this comment.
whoops, slipped in, 👍
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| # Use PR head SHA, not the merge commit — avoids phantom diffs |
There was a problem hiding this comment.
took from our sentry yml, not sure if we want in example workflow or not: https://github.com/getsentry/sentry/blob/master/.github/workflows/frontend-snapshots.yml#L42
There was a problem hiding this comment.
@rbro112 I read through the commit history and looks like we'll probably want to recommend this to customers? WDYT?
| build snapshots "${{ env.SNAPSHOT_OUTPUT_DIR }}" | ||
| --app-id <your-app-id> | ||
| --project <your-sentry-project> | ||
| --head-sha "${{ github.event.pull_request.head.sha || github.sha }}" |
There was a problem hiding this comment.
head-sha, vcs-provider and head-repo-name will be auto-detected in github actions so we should rely on that.
There was a problem hiding this comment.
going off our workflow: https://github.com/getsentry/sentry/blob/master/.github/workflows/frontend-snapshots.yml#L86
also how much do we want to write for github actions vs. general? not sure what we do for size/distro
There was a problem hiding this comment.
Well the syntax here only applies to github only i.e github.event.pull_request.head.sha. There's no way we can provide all these for all CI systems and it isn't snapshot specific. We should documet these somewhere else.
Anyways, I made a PR to update the workflow you linked to remove all the slop.
| --head-repo-name "${{ github.repository }}" | ||
| ) | ||
|
|
||
| if [[ "${{ github.event_name }}" == "pull_request" ]]; then |
There was a problem hiding this comment.
these will also all be auto detected so we should just rely on that.
| sentry-cli "${ARGS[@]}" | ||
| ``` | ||
|
|
||
| A few things to note about this workflow: |
There was a problem hiding this comment.
let's remove these 3 points. they are slop.
|
|
||
| ## How It Works | ||
|
|
||
| 1. **Generate snapshots** — Your CI job produces PNG screenshots however you like: Playwright, Storybook, simulator captures, or any tool that outputs PNGs. |
There was a problem hiding this comment.
Should we mention Paparazzi?
|
|
||
| <Include name="feature-available-for-user-group-early-adopter" /> | ||
|
|
||
| ## Status Checks |
There was a problem hiding this comment.
Should we add PR comments on this page as well?
There was a problem hiding this comment.
I'll put mention in the settings but since status check control a core workflow i think worth the callout. tbh this page is all slop rn and will get a rework
runningcode
left a comment
runningcode
left a comment
There was a problem hiding this comment.
Gave it another round!
| - PNG | ||
| - JPEG | ||
|
|
||
| ## Upload With CI |
There was a problem hiding this comment.
I think we should re-use the uploading with CI like we did for size analysis, or just bring it to a separate section since we are repeating ourselves everywhere https://docs.sentry.io/product/size-analysis/integrating-into-ci/
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| # Use PR head SHA, not the merge commit — avoids phantom diffs |
There was a problem hiding this comment.
@rbro112 I read through the commit history and looks like we'll probably want to recommend this to customers? WDYT?
| ## How It Works | ||
|
|
||
| 1. **Generate snapshots** — Your CI job produces screenshots however you like: Playwright, Paparazzi, or any tool that outputs PNGs | ||
| 2. **Upload to Sentry** — `sentry-cli` uploads the snapshot directory to Sentry |
There was a problem hiding this comment.
Might not necessarily be sentry-cli. Could be fastlane/gradle
| 1. **Generate snapshots** — Your CI job produces screenshots however you like: Playwright, Paparazzi, or any tool that outputs PNGs | ||
| 2. **Upload to Sentry** — `sentry-cli` uploads the snapshot directory to Sentry | ||
| 3. **Sentry diffs against the baseline** — Sentry compares images against corresponding base build (TODO: link to section explaining automated comparisons). With Sentry, you do not have to manage "Golden" images and everything is determined via git metadata. | ||
| 4. **Results post to your PR** — Snapshot results post to your PR\*. You can configure settings(TODO: link to settings) to control status check and comment behavior. |
There was a problem hiding this comment.
post might mean "comment" to some people. i think we should be clear that by default we only have status checks here but comments can also be added
Co-authored-by: Nelson Osacky <nelson.osacky@sentry.io>
Co-authored-by: Nelson Osacky <nelson@osacky.com>
## DESCRIBE YOUR PR Restructure the snapshots docs so CI setup lives on its own page, mirroring the pattern used by the Size Analysis docs. The current \`uploading-snapshots\` page embeds a full GitHub Actions workflow with a \`<your-snapshot-command>\` placeholder. Snapshot generation varies wildly by platform (Playwright, Paparazzi, XCUITests, Laravel Dusk…), so the single baked-in template is misleading. This PR extracts the CI guidance into a dedicated page and keeps the upload page focused on the universal bits (directory structure, JSON metadata, \`sentry-cli\` reference). - New \`docs/product/snapshots/integrating-into-ci.mdx\` with a skeleton GitHub Actions workflow that handles \`push\` to main and \`pull_request\` in one file, plus troubleshooting - Trimmed \`docs/product/snapshots/uploading-snapshots/index.mdx\` — removed the opinionated CI block, added the CI-alert include - New includes \`includes/snapshots/ci-alert.mdx\` and \`includes/snapshots/github-only.mdx\`, modeled on the size-analysis includes - Cross-links from \`index.mdx\` and \`reviewing-snapshots/index.mdx\` back to the new CI page - Left \`TODO\` links next to Android/iOS bullets for future platform-specific snapshot pages ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## DESCRIBE YOUR PR Add Android setup documentation for the Snapshots EA product. Covers enabling snapshots via the Sentry Android Gradle Plugin and wiring it up to popular snapshot tools. - Adds `docs/product/snapshots/android/index.mdx` - Documents SAGP 6.4.0+ setup with `sentry.snapshots` DSL - Documents Paparazzi (recommended), Roborazzi, and generic-tool integration - Links back to the existing uploading-snapshots CI guide Linked to EME-1031. Stacked on top of #[snapshots-ea-product-docs](https://github.com/getsentry/sentry-docs/pull/new/mtopo27/snapshots-ea-product-docs). Merge that PR first. ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) --------- Co-authored-by: Claude <noreply@anthropic.com>
## DESCRIBE YOUR PR Small follow-ups on top of `mtopo27/snapshots-ea-product-docs`: - Bump the Sentry Android Gradle Plugin minimum version from 6.4.0 to 6.5.0 in the Android snapshots setup guide. - Reword "snapshot tests" / "snapshot testing" to emphasize snapshot generation, since Sentry generates snapshots rather than runs tests. Note: references to the **Snapshot Testing** GitHub Check were left as-is — those appear to name a literal status check produced by Sentry. Let me know if you'd like those renamed too. ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 participants
Add three new pages for the Snapshots visual regression testing feature (Early Adopter) at
/product/snapshots/.Snapshots lets users generate PNG screenshots in CI, upload them to Sentry via
sentry-cli, and get pixel-level diffs reported as GitHub Checks on pull requests. This is the initial EA documentation covering the core workflow.Pages added:
index.mdx) — Overview, how it works (5-step flow), and prerequisitessetup/index.mdx) — Directory format, JSON sidecar metadata, and a complete GitHub Actions workflowreview-workflow/index.mdx) — Status checks, comparison viewer, approval flow, and project settingsAll pages include the EA banner. Screenshot placeholders (
IMAGE: ...) are used where real screenshots will be added later.REFS EME-1029