fix: harden diff baseline resolution

Author: lelia

CHANGELOG.md

@@ -20,11 +20,12 @@
   commit instead of failing the run, and logs which commit was used and how far
   back it is. Squash merges, rebases, and multi-commit pushes all leave a merge
   base unscanned even when default-branch scanning is configured correctly. The
-  run still fails when no scanned ancestor is reachable.
+  lookup follows paginated scan history and the run still fails when no scanned
+  ancestor is reachable or the exact-commit lookup itself fails.
 - Implicit diff baselines are selected from the same workspace, scan type,
-  repository, and default branch. A baseline lookup that fails is reported as an
-  API error instead of resolving to an empty baseline, and temporary scans are
-  skipped when selecting one.
+  repository, and default branch, including when no workspace is supplied. A
+  baseline lookup that fails is reported as an API error instead of resolving to
+  an empty baseline, and temporary scans are skipped when selecting one.
 ## 2.8.1
 
 ### Changed: bump pinned @coana-tech/cli to 15.10.40

README.md

@@ -44,21 +44,22 @@ socketcli --enable-gitlab-security --gitlab-security-file gl-dependency-scanning
 
 ### PR scan diffed against the merge base
 
-By default, PR scans are diffed against the repository's latest head scan. To diff against
-the exact commit your PR branched from instead, pass the merge base as the baseline:
+By default, PR scans are diffed against the repository's latest matching head scan. To
+prefer the commit your PR branched from as the baseline, pass the merge base:
 
 ```bash
 BASE_SHA=$(git merge-base origin/main HEAD)
 socketcli --pr-number 123 --base-commit-sha "$BASE_SHA"
 ```
 
-> **Requirement:** `--base-commit-sha` only works if Socket already has a full scan for that
-> exact commit. In practice this means your CI must run `socketcli` on **every commit that
-> lands on your default branch** — not just some of them. If merges can land without a scan
-> (skipped/canceled builds, `[skip ci]`, path-filtered pipelines), the PR scan will fail with
-> exit code 3 rather than silently diff against the wrong baseline. See
+> The CLI uses the exact commit's newest matching full scan when one exists. Otherwise, it
+> searches up to 100 first-parent commits in the local checkout and uses the nearest scanned
+> ancestor, with a warning that the diff is wider than the merge base. Run `socketcli`
+> regularly on your default branch and ensure PR checkouts contain enough history for that
+> walk. The run fails with the configured API-error exit code only when no scanned ancestor
+> is reachable (or when the scan lookup itself fails). See
 > [`docs/cli-reference.md`](https://github.com/SocketDev/socket-python-cli/blob/main/docs/cli-reference.md)
-> for the full requirements and a backfill pattern that makes PR jobs self-sufficient.
+> for the full behavior and an optional exact-baseline backfill pattern.
 
 A specific full scan ID also works: `--base-scan-id <id>`.
 

docs/ci-cd.md

@@ -347,11 +347,14 @@ generator rather than a static YAML file:
   BASE_SHA=$(git merge-base "origin/${TARGET}" HEAD)
   ```
 
-- **Emit the backfill step conditionally from the generator.** The generator is the
-  natural place for the "does a baseline scan exist?" check
+- **If an exact baseline is required, emit a backfill step conditionally from the
+  generator.** The generator is the natural place for the "does an exact baseline
+  scan exist?" check
   (`GET /orgs/{org}/full-scans?repo=<repo>&commit_hash=$BASE_SHA&per_page=1`): only
   emit the baseline-scan step when it returns nothing. The emitted pipeline then shows
-  in the UI whether a backfill will run.
+  in the UI whether a backfill will run. Without a backfill, the CLI automatically
+  uses the nearest scanned first-parent ancestor within 100 commits and warns that
+  the diff is wider.
 
 - **Keep the backfill inside one command step.** The checkout-base → scan →
   checkout-PR sequence must not be split across steps — steps can land on different
@@ -360,17 +363,18 @@ generator rather than a static YAML file:
   checkout: `git worktree add /tmp/socket-base "$BASE_SHA"` then
   `socketcli --target-path /tmp/socket-base --branch "$TARGET" --disable-blocking`.
 
-- **Soft-fail infra errors, not findings.** A missing baseline (or any API error)
-  exits with code 3 (`--exit-code-on-api-error` to change it); real findings exit 1.
+- **Soft-fail infra errors, not findings.** No reachable scanned ancestor (or any API
+  error) exits with code 3 (`--exit-code-on-api-error` to change it); real findings exit 1.
   [`soft_fail: [{exit_status: 3}]`](https://buildkite.com/docs/pipelines/configure/step-types/command-step)
   on the PR scan step keeps infra errors from blocking merges while security findings
   still do.
 
 - **["Cancel intermediate builds"](https://buildkite.com/docs/pipelines/configure/canceling-builds#cancel-running-intermediate-builds)
-  on the default branch is the main source of baseline gaps.** Canceled builds never
-  scan their commit, so merge-base lookups for PRs based on those commits fail. The
-  conditional backfill step above is the remedy; there is no per-step exemption from
-  build cancellation in Buildkite. If you need strict scan-once semantics for
+  on the default branch is a common source of exact-baseline gaps.** Canceled builds
+  never scan their commit, so these PRs fall back to an older scanned ancestor. Use
+  the conditional backfill step above when an exact merge-base comparison is required;
+  there is no per-step exemption from build cancellation in Buildkite. If you need
+  strict scan-once semantics for
   concurrent backfills of the same merge base, serialize the backfill step with a
   [concurrency group](https://buildkite.com/docs/pipelines/configure/workflows/controlling-concurrency)
   keyed on the merge-base SHA.

docs/cli-reference.md

@@ -256,24 +256,22 @@ If you don't want to provide the Socket API Token every time then you can use th
 | `--commit-message` | False    | *auto*  | Commit message (auto-detected from git)       |
 | `--commit-sha`     | False    | *auto*  | Commit SHA (auto-detected from git)           |
 | `--base-scan-id`   | False    |         | Full scan ID to diff against, overriding the repository's head scan as the baseline. Mutually exclusive with `--base-commit-sha` |
-| `--base-commit-sha`| False    |         | Commit SHA to diff against, overriding the repository's head scan as the baseline. The most recent full scan for that commit is used; the CLI errors (exit code 3, or `--exit-code-on-api-error`) if no scan exists for it. Mutually exclusive with `--base-scan-id` |
+| `--base-commit-sha`| False    |         | Commit SHA to prefer as the diff baseline, overriding the repository's head scan. The CLI uses its most recent matching full scan or the nearest scanned first-parent ancestor within 100 local commits. It errors (exit code 3, or `--exit-code-on-api-error`) if no scanned ancestor is reachable. Mutually exclusive with `--base-scan-id` |
 
-> **Diffing against the merge base** — by default, PR scans are diffed against the repository's *latest* head scan, which may include newer default-branch commits than your PR branched from. To diff against the exact commit your PR is based on, compute the merge base and pass it as the baseline:
+> **Diffing against the merge base** — by default, PR scans are diffed against the repository's latest matching head scan, which may include newer default-branch commits than your PR branched from. To prefer the commit your PR is based on, compute the merge base and pass it as the baseline:
 >
 > ```shell
 > BASE_SHA=$(git merge-base origin/main HEAD)
 > socketcli --pr-number 123 --base-commit-sha "$BASE_SHA"
 > ```
 >
-> **Requirement: a full scan must already exist for the merge-base commit.** `--base-commit-sha` does not create a scan of that commit; it looks up an existing one. That lookup only succeeds if your CI runs `socketcli` on **every commit that lands on your default branch** — every merge and direct push, not just periodic or latest-only scans. Common ways commits slip through without a scan:
+> `--base-commit-sha` does not create a scan of that commit. The CLI first looks for the newest non-temporary scan matching the repository, workspace, scan type, and exact commit. If the exact commit was not scanned, it walks up to 100 first-parent commits from that SHA in the local checkout and uses the nearest matching scanned ancestor. It logs a warning with the selected commit and distance because this produces a wider diff than the merge base.
 >
-> - CI settings that cancel or skip intermediate builds when newer commits land (e.g. Buildkite's ["cancel intermediate builds"](https://buildkite.com/docs/pipelines/configure/canceling-builds#cancel-running-intermediate-builds))
-> - `[skip ci]` commits, path-filtered pipelines, or failed/canceled scan steps
-> - merge-base commits that predate your Socket rollout
+> Run `socketcli` regularly on the default branch so recent ancestors have scans. PR checkouts must also retain the merge base and enough first-parent history; shallow clones can shorten the search. Gaps are expected when CI cancels intermediate builds, commits use `[skip ci]`, pipelines are path-filtered, or the merge base predates your Socket rollout.
 >
-> If no scan exists for the commit, the CLI **fails** (exit code 3, or your `--exit-code-on-api-error` value; exit 0 with `--disable-blocking`) instead of silently falling back to the head scan — a wrong baseline would misreport which alerts the PR introduces. Don't adopt this flag without default-branch scan coverage in place; you'll fail PR builds on lookup misses.
+> If no scanned ancestor is reachable within the local 100-commit walk, the CLI **fails** (exit code 3, or your `--exit-code-on-api-error` value; exit 0 with `--disable-blocking`) instead of silently falling back to the repository head. API or permission failures also fail rather than being treated as a missing exact scan.
 >
-> **Backfill pattern** — if your default-branch coverage has gaps, the PR job can create the missing baseline itself before scanning:
+> **Optional exact-baseline backfill** — if the wider ancestor fallback is not acceptable, the PR job can create the missing exact baseline before scanning:
 >
 > ```shell
 > BASE_SHA=$(git merge-base origin/main HEAD)
@@ -285,7 +283,7 @@ If you don't want to provide the Socket API Token every time then you can use th
 > socketcli --pr-number 123 --base-commit-sha "$BASE_SHA"
 > ```
 >
-> Run the baseline step with `--disable-blocking` (findings on the default branch must not fail the PR job) and an explicit `--branch`, since branch auto-detection is unreliable at a detached HEAD.
+> Run the baseline step with `--disable-blocking` (findings on the default branch must not fail the PR job) and an explicit `--branch`, since branch auto-detection is unreliable at a detached HEAD. Without this step, the CLI automatically uses the nearest scanned ancestor.
 >
 > Buildkite users with dynamically generated pipelines: see [Merge-base baselines in Buildkite](ci-cd.md#merge-base-baselines-in-buildkite-dynamic-pipelines) for generation-time vs. step-time guidance.
 

socketsecurity/config.py

@@ -587,9 +587,9 @@ def create_argument_parser() -> argparse.ArgumentParser:
         metavar="<sha>",
         default=None,
         help="Commit SHA to diff the new scan against, overriding the repository's head "
-             "scan as the baseline. The most recent full scan matching this commit (e.g. "
-             "the merge base from 'git merge-base origin/main HEAD') is used; the CLI "
-             "errors if no scan exists for it. Mutually exclusive with --base-scan-id."
+             "scan as the baseline. The CLI uses the most recent matching full scan, or "
+             "the nearest scanned first-parent ancestor within 100 local commits when "
+             "the commit itself was not scanned. Mutually exclusive with --base-scan-id."
     )
 
     # Path and File options