Skip to content

Add skill: ship-ready-quick-scan (offline launch check for Supabase apps) 🤖🤖🤖 - #6539

Closed
getshipready wants to merge 2 commits into
github:mainfrom
getshipready:add-ship-ready-quick-scan
Closed

getshipready wants to merge 2 commits into
github:mainfrom
getshipready:add-ship-ready-quick-scan

Conversation

@getshipready

Copy link
Copy Markdown

Adds skills/ship-ready-quick-scan: an offline, read-only check of an app repo before launch. Standard-library Python 3, no network calls, writes nothing. The skill uses the Agent Skills format Copilot also reads; I have not yet run it inside Copilot, so I'm making no claim about that.

It runs 12 checks on AI-built Supabase/Next.js/Vite apps, including: committed secrets, .env not git-ignored, secrets behind NEXT_PUBLIC_/VITE_, tables with RLS off, using (true) read policies, views that skip RLS, API routes with no caller check, unsigned Stripe webhooks, security definer functions callable by anon, default grants exposing new tables, and roles read from user_metadata.

SKILL.md calls the script with --no-promo, so the output has no sales text. Without that flag the script prints a closing note about the author's paid product; the flag (and env SHIPREADY_NO_PROMO=1) was added in v0.2.7 for this reason.

Limitations: static analysis of the files in the repo only. It can't see policies changed in the Supabase dashboard; SKILL.md points to checking those in the SQL editor. Findings are leads to confirm at the file:line shown.

Checks run: npm run skill:validate passes (465 skills), npm start regenerated docs/README.skills.md. Upstream scanner has 86 passing tests.

Disclosure: this PR was prepared by an AI agent working for the Ship-Ready project. I'm affiliated with it: Ship-Ready is the maintainer of the skill (MIT, https://github.com/getshipready/ship-ready-skills). The project also sells a separate paid product; this skill is free and complete on its own, and the PR contains no sales links.

🤖🤖🤖

@getshipready
getshipready requested review from a team as code owners November 19, 2026 23:02
Copilot AI adaptive review requested due to automatic review settings November 19, 2026 23:02
@getshipready

Copy link
Copy Markdown
Author

Update: the first commit was missing the helper modules that quickscan.py imports (sql001.py, rls011.py, rls014.py, auth013.py). Added in the latest commit; I ran the script from the skill folder with --no-promo and it completes. 🤖 (prepared by an AI agent for the Ship-Ready project)

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

🚦 Submission status: ⏳ Awaiting automation

Risk tier: merge-risk:high — Privileged execution, automation, or review-policy change
Required to merge: passing submission-gate checks plus 1 approval from reviewers with write access, including a maintainer with admin or maintain permission.

Why this tier
  • skills/ship-ready-quick-scan/scripts/auth013.py is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • skills/ship-ready-quick-scan/scripts/quickscan.py is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • skills/ship-ready-quick-scan/scripts/rls011.py is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • skills/ship-ready-quick-scan/scripts/rls014.py is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • skills/ship-ready-quick-scan/scripts/sql001.py is a high-risk path (automation, scripts, MCP config, hooks, or review policy)

Automated checks

Check Status Details
Line endings 🔧 Infrastructure failure Waiting for a maintainer to approve workflow runs for this PR · logs
Spelling 🔧 Infrastructure failure Waiting for a maintainer to approve workflow runs for this PR · logs
Generated README consistency 🔧 Infrastructure failure Waiting for a maintainer to approve workflow runs for this PR · logs
Skill validation 🔧 Infrastructure failure Waiting for a maintainer to approve workflow runs for this PR · logs
Skill lint (vally) 🔧 Infrastructure failure Waiting for a maintainer to approve workflow runs for this PR · logs
Risk scan 🔧 Infrastructure failure Waiting for a maintainer to approve workflow runs for this PR · logs
Contributor reputation 🔧 Infrastructure failure Waiting for a maintainer to approve workflow runs for this PR · logs
Duplicate resource scan ⚠️ Failed (advisory, non-blocking) Waiting for a maintainer to approve workflow runs for this PR · logs
PR quality signal ⚠️ Failed (advisory, non-blocking) Waiting for a maintainer to approve workflow runs for this PR · logs

Action needed

  • 🔧 Line endings, Spelling, Generated README consistency, Skill validation, Skill lint (vally), Risk scan, Contributor reputation hit an automation problem that is not caused by your contribution. Comment /rerun-checks to retry; maintainers are notified if it keeps failing.
  • ⚠️ Advisory checks did not complete (Duplicate resource scan, PR quality signal). This does not block the PR.

Review

  • Approvals: 0/1
  • Assigned reviewer: not assigned yet — comment /request-review to ask for one
  • Review target date: not set
  • Still needed: 1 more approval(s); an approval from a maintainer with admin or maintain permission
  • The core-maintainers pool is not staffed yet; an approver with admin or maintain permission is required instead.

Commands

Command Who What it does
/rerun-checks PR author, maintainers Re-runs failed or incomplete checks and re-evaluates this gate
/request-review PR author, maintainers Asks the review rotation to assign a reviewer (adds needs-reviewer)

Updated for 8d5372e · gate run · This comment is maintained automatically — see submission gate docs.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical defects can suppress real security findings, generate false CI blockers, and expose credentials in diagnostic output.

29 open issues

And 9 more that still need to be addressed.

What changed in this PR

Adds a Copilot skill for offline, static launch-readiness checks of Supabase apps, extending the repository’s collection of bundled developer tools.

Changes:

  • Adds 12 checks covering secrets, database access, API authentication, and Stripe webhooks.
  • Documents a scan-and-confirm workflow, remediation guidance, and optional CI failure thresholds.
  • Includes a GitHub Actions example, MIT license, and skill catalog entry.
File Description
skills/​ship-ready-quick-scan/​SKILL.md Defines usage, confirmation steps, and limitations.
skills/​ship-ready-quick-scan/​scripts/​sql001.py Detects functions executing caller-supplied SQL.
skills/​ship-ready-quick-scan/​scripts/​rls014.py Checks default and bulk table grants.
skills/​ship-ready-quick-scan/​scripts/​rls011.py Checks anonymous access to privileged functions.
skills/​ship-ready-quick-scan/​scripts/​quickscan.py Coordinates scanning, reporting, and CI exit behavior.
skills/​ship-ready-quick-scan/​scripts/​auth013.py Detects authorization based on user-editable metadata.
skills/​ship-ready-quick-scan/​LICENSE Supplies the skill’s MIT license.
skills/​ship-ready-quick-scan/​examples/​github-actions/​ship-ready-quick-scan.yml Provides a pull-request scanning workflow.
docs/​README.skills.md Registers the skill in the generated catalog.

Review effort: Max (adaptive)

- uses: actions/checkout@v4
with:
repository: getshipready/ship-ready-skills
ref: v0.2.7
raw_user_meta_data. Stdlib, static, on migrations."""
import re, os, sys
POLICY_RX = re.compile(r"\bcreate\s+policy\b[^;]*;", re.I | re.S)
FUNC_RX = re.compile(r"\bcreate\s+(?:or\s+replace\s+)?function\b.*?(?:\$\w*\$).*?(?:\$\w*\$)[^;]*;", re.I | re.S)
import re, os, sys
POLICY_RX = re.compile(r"\bcreate\s+policy\b[^;]*;", re.I | re.S)
FUNC_RX = re.compile(r"\bcreate\s+(?:or\s+replace\s+)?function\b.*?(?:\$\w*\$).*?(?:\$\w*\$)[^;]*;", re.I | re.S)
META_RX = re.compile(r"(auth\.jwt\(\)\s*->+\s*'user_metadata'|request\.jwt\.claims[^;]*user_metadata|raw_user_meta_data|user_metadata\s*->>?\s*'(role|is_admin|admin|plan|tier|org_id|tenant_id|team_id))", re.I)
name = re.search(r"create\s+policy\s+(\"[^\"]+\"|\w+)", stmt, re.I)
out.append({"id": "AUTH-013", "severity": "critical", "file": rel, "line": _line(text, m.start() + hit.start()),
"title": f"Policy {name.group(1) if name else ''} authorizes from user_metadata, which any signed-in user can edit (updateUser) to grant themselves access",
"evidence": " ".join(stmt.split())[:110],
"rlsview": "Recreate the view `with (security_invoker = true)` so the caller's Row Level Security applies.",
"defnfn": "For each function listed, run `revoke execute on function public.<name> from public, anon;` (Postgres lets everyone call new functions by default, so revoking from anon alone is not enough), then grant it back only to `authenticated` if signed-in users need it, or make it `security invoker`. Supabase lint 0028 describes this.",
"usermeta": "Stop authorizing from `user_metadata` / `raw_user_meta_data` (users can change it with `supabase.auth.updateUser`). Read roles from `app_metadata` (`auth.jwt() -> 'app_metadata'`, writable only by the server) or from a roles table users cannot write, and make any sign-up trigger set a safe default role instead of copying one from the metadata.",
"defgrant": "Remove the `alter default privileges ... grant ... on tables to anon/authenticated` statement (and any `grant ... on all tables in schema public to anon`). Instead grant per table, in the migration that creates it (`grant select, insert on public.<table> to authenticated;`), together with `enable row level security` and a policy.",
"""
import base64, json, os, re, sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import sql001, rls011, rls014, auth013
Comment on lines +386 to +388
for rel, text in repo.files.items():
if rel.split("/")[-1] != "config.toml" or "supabase" not in rel:
continue
Comment on lines +464 to +470
if hits:
skills = [skill]
if cid == "apiauth" and any(is_edge_finding(f) for f in hits):
skills = ["edge-function-auth"] + ([skill] if any(not is_edge_finding(f) for f in hits) else [])
body.append(" Fix it free: Lite skill " + " + ".join(f"`{x}`" for x in skills))
if na:
body.append(" Skipped: " + ("no Stripe usage found." if cid == "webhook" else "no SQL migrations in the repo; run the Lite skill `rls-quick-check` in the Supabase SQL editor."))
Comment on lines +18 to +20
2. Run the bundled script from this skill's folder: `python3 <this skill's folder>/scripts/quickscan.py <app root> --no-promo`. It needs only Python 3 (standard library), reads files, writes nothing and makes no network calls.
3. Paste the output to the user unchanged. Secrets are already redacted; never print the raw value from the file.
4. For each FAIL, open the file:line and confirm it is real (not a test fixture or a placeholder). Say which ones you confirmed.
Comment on lines +509 to +515
out += [f"That was the quick scan: {len(CHECKS)} day-one checks for leaked keys, open data and unpaid upgrades.",
f"Ship-Ready Crew: {FULL_RULES - nrules} more checks (self-promotion to admin/Pro, client-only admin checks,",
"public storage buckets, Stripe price tampering and entitlements, deploy, performance, launch basics),",
"a ship score /100, a fix plan with patches, and Proof: Playwright launch tests for your money paths.",
"One-time $39, 10 agents.",
f" See a full sample report (49 findings on our demo app): {SAMPLE_URL}",
f" Get the Crew: {CREW_URL}"]
@aaronpowell

Copy link
Copy Markdown
Contributor

Thanks for submitting this skill. The scanner includes useful offline Supabase checks, but quickscan.py promotes the paid Ship-Ready Crew by default, and the upstream README presents the free skill pack alongside a $39 Crew and $129 audit. Although --no-promo suppresses the scanner’s upsell, the contribution is still framed as an entry point to those paid offerings rather than as a standalone resource for this collection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants