Repository navigation
Add skill: ship-ready-quick-scan (offline launch check for Supabase apps) 🤖🤖🤖 - #6539
getshipready wants to merge 2 commits into
Conversation
|
Update: the first commit was missing the helper modules that |
🚦 Submission status: ⏳ Awaiting automationRisk tier: Why this tier
Automated checks
Action needed
Review
Commands
Updated for |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical defects can suppress real security findings, generate false CI blockers, and expose credentials in diagnostic output.
29 open issues
Pin scanner checkout to a reviewed commit SHA · New Bound SQL and dollar-quote parsing · New Recognize wrapped auth.jwt() metadata access · New Redact SQL literals in policy evidence · New Recommend new revoking migrations and recognize them · New Reject symlinks and bound file reads · New Require actual signature verification evidence · New Require secret-key prefixes at token boundaries · New Apply ignore rules per env file · New Support quoted schema identifiers in SQL patterns · New Do not suppress secrets in tracked env files · New Let explicit secret markers override safe-name exemptions · New Gate Supabase checks on Supabase evidence · New Exclude SQL comments from RLS checks · New Recognize unquoted policies and default ALL commands · New Evaluate the final active policy state · New Report only effective RLS settings · New Evaluate authentication separately for each route handler · New Require a Stripe signal in each webhook handler · New Match the complete identifier after UPDATE · New
And 9 more that still need to be addressed.
What changed in this PR
Adds a Copilot skill for offline, static launch-readiness checks of Supabase apps, extending the repository’s collection of bundled developer tools.
Changes:
- Adds 12 checks covering secrets, database access, API authentication, and Stripe webhooks.
- Documents a scan-and-confirm workflow, remediation guidance, and optional CI failure thresholds.
- Includes a GitHub Actions example, MIT license, and skill catalog entry.
| File | Description |
|---|---|
skills/ship-ready-quick-scan/SKILL.md |
Defines usage, confirmation steps, and limitations. |
skills/ship-ready-quick-scan/scripts/sql001.py |
Detects functions executing caller-supplied SQL. |
skills/ship-ready-quick-scan/scripts/rls014.py |
Checks default and bulk table grants. |
skills/ship-ready-quick-scan/scripts/rls011.py |
Checks anonymous access to privileged functions. |
skills/ship-ready-quick-scan/scripts/quickscan.py |
Coordinates scanning, reporting, and CI exit behavior. |
skills/ship-ready-quick-scan/scripts/auth013.py |
Detects authorization based on user-editable metadata. |
skills/ship-ready-quick-scan/LICENSE |
Supplies the skill’s MIT license. |
skills/ship-ready-quick-scan/examples/github-actions/ship-ready-quick-scan.yml |
Provides a pull-request scanning workflow. |
docs/README.skills.md |
Registers the skill in the generated catalog. |
Review effort: Max (adaptive)
| - uses: actions/checkout@v4 | ||
| with: | ||
| repository: getshipready/ship-ready-skills | ||
| ref: v0.2.7 |
| raw_user_meta_data. Stdlib, static, on migrations.""" | ||
| import re, os, sys | ||
| POLICY_RX = re.compile(r"\bcreate\s+policy\b[^;]*;", re.I | re.S) | ||
| FUNC_RX = re.compile(r"\bcreate\s+(?:or\s+replace\s+)?function\b.*?(?:\$\w*\$).*?(?:\$\w*\$)[^;]*;", re.I | re.S) |
| import re, os, sys | ||
| POLICY_RX = re.compile(r"\bcreate\s+policy\b[^;]*;", re.I | re.S) | ||
| FUNC_RX = re.compile(r"\bcreate\s+(?:or\s+replace\s+)?function\b.*?(?:\$\w*\$).*?(?:\$\w*\$)[^;]*;", re.I | re.S) | ||
| META_RX = re.compile(r"(auth\.jwt\(\)\s*->+\s*'user_metadata'|request\.jwt\.claims[^;]*user_metadata|raw_user_meta_data|user_metadata\s*->>?\s*'(role|is_admin|admin|plan|tier|org_id|tenant_id|team_id))", re.I) |
| name = re.search(r"create\s+policy\s+(\"[^\"]+\"|\w+)", stmt, re.I) | ||
| out.append({"id": "AUTH-013", "severity": "critical", "file": rel, "line": _line(text, m.start() + hit.start()), | ||
| "title": f"Policy {name.group(1) if name else ''} authorizes from user_metadata, which any signed-in user can edit (updateUser) to grant themselves access", | ||
| "evidence": " ".join(stmt.split())[:110], |
| "rlsview": "Recreate the view `with (security_invoker = true)` so the caller's Row Level Security applies.", | ||
| "defnfn": "For each function listed, run `revoke execute on function public.<name> from public, anon;` (Postgres lets everyone call new functions by default, so revoking from anon alone is not enough), then grant it back only to `authenticated` if signed-in users need it, or make it `security invoker`. Supabase lint 0028 describes this.", | ||
| "usermeta": "Stop authorizing from `user_metadata` / `raw_user_meta_data` (users can change it with `supabase.auth.updateUser`). Read roles from `app_metadata` (`auth.jwt() -> 'app_metadata'`, writable only by the server) or from a roles table users cannot write, and make any sign-up trigger set a safe default role instead of copying one from the metadata.", | ||
| "defgrant": "Remove the `alter default privileges ... grant ... on tables to anon/authenticated` statement (and any `grant ... on all tables in schema public to anon`). Instead grant per table, in the migration that creates it (`grant select, insert on public.<table> to authenticated;`), together with `enable row level security` and a policy.", |
| """ | ||
| import base64, json, os, re, sys | ||
| sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) | ||
| import sql001, rls011, rls014, auth013 |
| for rel, text in repo.files.items(): | ||
| if rel.split("/")[-1] != "config.toml" or "supabase" not in rel: | ||
| continue |
| if hits: | ||
| skills = [skill] | ||
| if cid == "apiauth" and any(is_edge_finding(f) for f in hits): | ||
| skills = ["edge-function-auth"] + ([skill] if any(not is_edge_finding(f) for f in hits) else []) | ||
| body.append(" Fix it free: Lite skill " + " + ".join(f"`{x}`" for x in skills)) | ||
| if na: | ||
| body.append(" Skipped: " + ("no Stripe usage found." if cid == "webhook" else "no SQL migrations in the repo; run the Lite skill `rls-quick-check` in the Supabase SQL editor.")) |
| 2. Run the bundled script from this skill's folder: `python3 <this skill's folder>/scripts/quickscan.py <app root> --no-promo`. It needs only Python 3 (standard library), reads files, writes nothing and makes no network calls. | ||
| 3. Paste the output to the user unchanged. Secrets are already redacted; never print the raw value from the file. | ||
| 4. For each FAIL, open the file:line and confirm it is real (not a test fixture or a placeholder). Say which ones you confirmed. |
| out += [f"That was the quick scan: {len(CHECKS)} day-one checks for leaked keys, open data and unpaid upgrades.", | ||
| f"Ship-Ready Crew: {FULL_RULES - nrules} more checks (self-promotion to admin/Pro, client-only admin checks,", | ||
| "public storage buckets, Stripe price tampering and entitlements, deploy, performance, launch basics),", | ||
| "a ship score /100, a fix plan with patches, and Proof: Playwright launch tests for your money paths.", | ||
| "One-time $39, 10 agents.", | ||
| f" See a full sample report (49 findings on our demo app): {SAMPLE_URL}", | ||
| f" Get the Crew: {CREW_URL}"] |
|
Thanks for submitting this skill. The scanner includes useful offline Supabase checks, but |

Adds
skills/ship-ready-quick-scan: an offline, read-only check of an app repo before launch. Standard-library Python 3, no network calls, writes nothing. The skill uses the Agent Skills format Copilot also reads; I have not yet run it inside Copilot, so I'm making no claim about that.It runs 12 checks on AI-built Supabase/Next.js/Vite apps, including: committed secrets,
.envnot git-ignored, secrets behindNEXT_PUBLIC_/VITE_, tables with RLS off,using (true)read policies, views that skip RLS, API routes with no caller check, unsigned Stripe webhooks,security definerfunctions callable by anon, default grants exposing new tables, and roles read fromuser_metadata.SKILL.md calls the script with
--no-promo, so the output has no sales text. Without that flag the script prints a closing note about the author's paid product; the flag (and envSHIPREADY_NO_PROMO=1) was added in v0.2.7 for this reason.Limitations: static analysis of the files in the repo only. It can't see policies changed in the Supabase dashboard; SKILL.md points to checking those in the SQL editor. Findings are leads to confirm at the file:line shown.
Checks run:
npm run skill:validatepasses (465 skills),npm startregenerateddocs/README.skills.md. Upstream scanner has 86 passing tests.Disclosure: this PR was prepared by an AI agent working for the Ship-Ready project. I'm affiliated with it: Ship-Ready is the maintainer of the skill (MIT, https://github.com/getshipready/ship-ready-skills). The project also sells a separate paid product; this skill is free and complete on its own, and the PR contains no sales links.
🤖🤖🤖