AI-First Starting UX: Build from scratch by lmac-1 · Pull Request #4918 · OpenFn/lightning

lmac-1 · 2026-07-02T18:18:07Z

Adds the "Build from scratch" path from the new-workflow landing screen (part of the #4848 AI-first starting UX epic).

Clicking the "Build from scratch" card:

Seeds a blank workflow (a webhook trigger, one "Untitled step" job on @openfn/language-common@latest, and an always-on edge) directly into the already-open collaborative session's Y.Doc, via the same importWorkflow → saveWorkflow client path that "Browse templates" and "Import YAML" already use on this screen. There's no server-side creation endpoint.
saveWorkflow's own success handler rewrites the URL from /w/new to the real /w/<workflowId> in place (replace: true, no full-page navigation).
Because the workflow is already persisted by the time the landing screen dismisses, the canvas opens in the normal saved state: no pink unsaved toolbar, Save button visible.
The trigger inspector opens on the normal read-only show panel, same as any other trigger.

Workflows are named "Untitled workflow", "Untitled workflow 1", etc. via Workflows.unique_workflow_name/2, invoked through the client's existing validate_workflow_name channel event (the same one templates/YAML import already trigger inside importWorkflow). This also fixes a bug where clicking the card a second time in the same project failed on the name unique constraint.

Double-clicks are prevented client-side: useActionLock disables the card while a save is pending.

Closes #4895

Additional notes for the reviewer

This PR changed approach since Frank's review: the original version created the workflow server-side (Workflows.create_webhook_workflow/2) and navigated to the canvas with a trigger_view=picker param to auto-open the trigger inspector. Frank pointed out this didn't need its own server endpoint when the client-side workflowStore path already covers creation for templates and YAML import — this description reflects that rewrite, not the original implementation.
Creation goes through the same collaborative-session save_workflow pipeline templates/YAML import use, so snapshots, lock_version, and audit behave like any other workflow save.
The workflow-activation limiter needs no special handling here: Session.handle_call({:save_workflow, ...}) already runs maybe_disable_triggers_on_limit/1 on every collaborative save, so a project at quota gets its new trigger auto-disabled after save, the same as any other save on this path.
Failure to save shows a "Failed to create workflow" flash and keeps the user on the landing screen.
Known gap, not introduced by this PR: a viewer's channel join for a new workflow is rejected server-side (WorkflowChannel.join/3 requiring :create_workflow), so a viewer can never persist a workflow — but the rejection currently produces no visible error. The vendored y-phoenix-channel provider only registers an "ok" receiver on channel.join(), not an "error" one, so the join rejection never reaches the client. The landing screen isn't gated on join success, so a viewer sees a normal, clickable card that silently does nothing on click. This is shared with templates/YAML import (same join, same missing receiver) — flagging for a product/UX call rather than fixing here.

Validation steps

Open the new workflow screen in a project — the landing screen appears.
Click Build from scratch.
Land on the canvas of a saved "Untitled workflow" with a webhook trigger connected to a job, toolbar in the normal saved state. The trigger inspector should not be open — clicking the trigger node opens the normal read-only show panel.
Repeat in the same project: the second workflow should be created as "Untitled workflow 1" instead of failing.
Delete one and build from scratch again: the freed name is reused.
Double-click the card rapidly: only one workflow should be created, and the card should grey out while pending.
As a project viewer: the card is still visible and clickable, but clicking it should never actually create a workflow. There is currently no visible error message for this case (see note 4 above).
Press Back from the canvas: you should return to wherever you were before the landing screen, not back to it.

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

I have used Claude Code
I have used another model
I have not used AI

You can read more details in our
Responsible AI Policy

Pre-submission checklist

I have performed an AI review of my code (we recommend using /review
with Claude Code)
I have implemented and tested all related authorization policies.
(e.g., :owner, :admin, :editor, :viewer)
I have updated the changelog.
I have ticked a box in "AI usage" in this PR

Adds the "Build from scratch" entry point on the new-workflow landing screen: Workflows.create_webhook_workflow/2 builds a workflow with a webhook trigger, one job, and an always-on edge; the build_from_scratch LiveView event authorizes via the role-scoped ProjectUsers :create_workflow check so viewers can't trigger it.

…flow_name/2 Clicking "Build from scratch" twice in one project failed on the workflow name unique constraint, since create_webhook_workflow/2 hardcoded its name. Extract the collision-avoidance logic duplicated in workflow_channel.ex and new_workflow_component.ex into a public Workflows.unique_workflow_name/2 (names-only query, includes soft-deleted rows since the unique index is not partial) and use it in create_webhook_workflow/2, which now defaults to "Untitled workflow" with " 1", " 2"... suffixes on collision. The channel delegates to the shared function. The legacy component keeps its private copy because that path is being deleted in this epic.

Rapid double-clicks on the "Build from scratch" card sent multiple build_from_scratch events, each creating a workflow. Wrap the push in useActionLock so only one event goes out, disable the card while pending, and guard the LiveView handler with a creating_workflow? assign as a server-side backstop for events that arrive anyway. Also normalize the new-workflow placeholder to "Untitled workflow" to match the rest of the codebase, and add regression tests: repeated clicks create suffixed workflows, creation after deletion reuses the freed name, and a failed save shows an error flash.

Build-from-scratch now redirects with ?trigger_view=picker so the newly created webhook trigger opens on the "What triggers this workflow?" picker instead of the read-only show panel, inviting the user to actively choose a trigger type. TriggerInspector captures the one-shot signal via a lazy useState initializer, then clears it from both React state (so a later Edit click lands on Choose, not the picker) and the URL via updateSearchParams (so a page refresh doesn't reopen the picker).

updateSearchParams always pushed a history entry, so stripping the one-shot ?trigger_view=picker signal after build-from-scratch left a phantom Back-button stop pointing right back at the picker. Add an optional { replace: true } to patch the current entry in place instead, and use it in TriggerInspector.

…nvas The build_from_scratch redirect pushed a new history entry on top of /new, so Back from the canvas landed on the now-pointless landing screen instead of wherever the user was before it. Pass replace: true to push_navigate so the canvas URL takes over /new's slot.

…limit create_webhook_workflow/2 was calling save_workflow/2 with enabled: true unconditionally, bypassing the WorkflowUsageLimiter. A project at its active-workflow quota could use the Build from scratch entry point to create a live webhook trigger regardless of quota. Now calls limit_workflow_creation/1 before building attrs and sets trigger enabled: false if the limit is exceeded, mirroring the auto-disable behaviour the collab-editor session path already applies to new workflows at the limit.

…toast Wraps the build_from_scratch pushEventTo call in a 10s timeout so the card doesn't stay permanently disabled if the server never acknowledges the push (socket disconnect or server-side error). Shows an error toast on failure so the user knows to retry. Also fixes disabled card hover styles bleeding through, renames the default job to "Untitled job", and removes a stale comment.

save_workflow's @SPEC only listed changeset and :workflow_deleted as error shapes, but handle_save_result can also return {:error, false} (snapshot failure) or other Multi step reasons. create_webhook_workflow inherited the same too-narrow spec. Widen both specs to match what the code actually returns, and rename the bound error variables from `changeset` to `reason` since they aren't always changesets.

The build_from_scratch handler is fully synchronous, so a second event on the same socket can never observe creating_workflow?: true — the flag is always reset or the socket has already navigated away by the time the process could dequeue another event. The client-side useActionLock is the guard that actually matters. Drop the dead assign, guard clause, and the test that only exercised it via a hand-built socket.

useAIWorkflowApplications.globalStep.test.ts was missing the streamingApply/streamingApplyActions props that its sibling test files (autoApply, jobCode, workflow) already pass. The hook calls streamingApplyActions.clear() unconditionally in handleApplyWorkflow, so leaving it undefined threw a TypeError as soon as that path ran.

github-actions · 2026-07-07T08:59:50Z

Security Review ✅

S0 (project scoping): New unique_workflow_name/2 at lib/lightning/workflows.ex:239 filters by project_id, and create_webhook_workflow/2 receives project_id from socket.assigns.project (set by the :project_scope mount hook), not user params.
S1 (authorization): The new build_from_scratch event at lib/lightning_web/live/workflow_live/collaborate.ex:182 gates on Permissions.can(ProjectUsers, :create_workflow, ...) before creation, with a proper {:error, :unauthorized} branch; the :create_workflow action already exists in ProjectUsers (:owner/:admin/:editor).
S2 (audit trail): Workflow creation flows through the existing save_workflow/3, which already inserts audit_snapshot_creation and maybe_audit_workflow_state_changes in the same Ecto.Multi, so audit coverage is unchanged.

The union `Ecto.Changeset.t(Workflow.t()) | false | term()` is redundant since term() already covers every possible value, including a changeset and false. Dialyzer treats the whole thing as term() anyway, so spelling out the specific shapes added nothing. Checked all four callers (edit.ex, helpers.ex, workflows_controller.ex, session.ex) - none pattern-match on the specific error shape, they all fall through to a generic error branch. {:error, term()} is also the pattern used elsewhere in the codebase (lightning.ex, credentials.ex, projects.ex, etc.), so this brings save_workflow/create_webhook_workflow in line with that convention rather than carrying a one-off wide union.

lmac-1 · 2026-07-07T09:13:19Z

+        await new Promise<void>((resolve, reject) => {
+          const timeout = setTimeout(
+            () => reject(new Error('build_from_scratch timed out')),
+            10_000


useActionLock requires a promise, so we create a timeout in order to only build from scratch if successful.

lmac-1 · 2026-07-07T09:15:54Z

+            () => reject(new Error('build_from_scratch timed out')),
+            10_000
+          );
+          pushEventTo('build_from_scratch', {}, () => {


Claude tells me it's okay to use pushEventTo here in collaborative editor but would like confirmation that it's the right pattern to do stuff like that here.

elias-ba

Amazing work @lmac-1, you are a rockstar 🎸

midigofrank

Hey @lmac-1 , this looks great. Two things from the elixirland:

Instead of blanking all the errors paths as term() maybe we could just add the missing ones? So something like {:error, Ecto.Changeset.t() | :workflow_deleted | false}. We should change the false into something readable like :snapshot_failed
Do we really need to define the create_webhook_workflow function on the server side? This is way too specific. Given we already have an api for creating workflows from the workflowStore, I would very much prefer to just have this go through that path.

Snapshot failures during save_workflow were returned as {:error, false}, giving callers no way to pattern-match on the failure. Introduce a :snapshot_failed atom, thread it through the save_workflow and create_webhook_workflow specs, and handle it explicitly in the collaboration session and workflow channel error replies.

…f a server endpoint Route the landing screen's build-from-scratch card through the same importWorkflow -> saveWorkflow client path templates and YAML import already use, instead of a bespoke create_webhook_workflow LiveView event. Drop the picker-auto-focus mechanism (trigger_view URL param / startOnPicker prop) that only existed to support the old server-navigation flow — the new trigger now opens on the normal read-only show panel like any other trigger. Also drop the landing screen's hand-tuned dot-grid background, which duplicated the canvas's real Background component from memory and could silently drift; use a flat background instead.

…dler Build-from-scratch no longer needs a server-side creation endpoint now that it goes through the client-side workflowStore. Delete the now-unreachable Workflows.create_webhook_workflow/2, collaborate.ex's build_from_scratch handler, and their tests.

The { replace: true } option on updateSearchParams/replaceSearchParams/ updateHash was added to support stripping the one-shot trigger_view=picker param without leaving a phantom Back-button entry. That flow was dropped when build-from-scratch moved off the trigger_view/startOnPicker mechanism, leaving the option with no callers anywhere in the codebase. Revert to the plain pushState-only API and drop its dedicated test block.

lmac-1 · 2026-07-09T17:52:28Z

Thanks @midigofrank

On 1 - noted and done. While checking every place that reads the result of save_workflow, I found one more spot: workflows_controller.ex (the REST API for creating/updating workflows). It has a big list of "if the error is X, send back this response" checks, but there's no check for "workflow was deleted" or "snapshot failed" — so if either of those ever happens through the API, the request would crash instead of returning a normal error message. This bug already existed before this PR for the "deleted" case - I didn't introduce it, and this PR doesn't touch that file. I'd rather not fix it here since it's unrelated to what this PR is doing. I've filed #4960 to keep track.

On 2 - this had me going around in circles all day. Yes I agree it makes it simpler but it made the implementation a bit more complicated. I decided to deprioritise having the "Trigger picker" component open on "What triggers this workflow" when you build from scratch. I will try to build this in later but it was getting really messy and I need to move forward. I've applied the changes so it goes through importWorkflow -> saveWorkflow and uses a new base template in baseTemplates file. I've updated PR description to reflect the new approach so it can be understood what happened. I also cleaned up any code related to having trigger inspector open.

midigofrank · 2026-07-10T04:48:33Z

Hey @lmac-1 , great catch. Opening an issue is the right call here.

Regarding 2, I think that's fine

lmac-1 · 2026-07-10T08:20:51Z

Thanks @midigofrank! Appreciate you reviewing this one quickly. Merging 🚀

github-project-automation Bot added this to Core Jul 2, 2026

github-project-automation Bot moved this to New Issues in Core Jul 2, 2026

Base automatically changed from 4868-build-with-ai to 4848-ai-first-starting-ux-parent July 6, 2026 08:17

lmac-1 force-pushed the 4895-ai-first-build-from-scratch branch from 0110e36 to 290612b Compare July 6, 2026 09:05

lmac-1 added 10 commits July 6, 2026 12:54

lmac-1 marked this pull request as ready for review July 7, 2026 08:57

lmac-1 commented Jul 7, 2026

View reviewed changes

Comment thread lib/lightning/workflows.ex Outdated

lmac-1 commented Jul 7, 2026

View reviewed changes

fix(build-from-scratch): capitalize placeholder workflow name

53ace55

lmac-1 requested review from elias-ba and midigofrank July 7, 2026 09:23

elias-ba approved these changes Jul 8, 2026

View reviewed changes

midigofrank requested changes Jul 9, 2026

View reviewed changes

github-project-automation Bot moved this from New Issues to In review in Core Jul 9, 2026

lmac-1 mentioned this pull request Jul 9, 2026

WorkflowsController can crash on :workflow_deleted / :snapshot_failed #4960

Open

lmac-1 added 2 commits July 9, 2026 12:02

lmac-1 added 2 commits July 9, 2026 18:28

lmac-1 requested a review from midigofrank July 9, 2026 17:52

midigofrank approved these changes Jul 10, 2026

View reviewed changes

undo comment change

24c1628

lmac-1 merged commit bc07171 into 4848-ai-first-starting-ux-parent Jul 10, 2026
4 of 5 checks passed

github-project-automation Bot moved this from In review to Done in Core Jul 10, 2026

lmac-1 deleted the 4895-ai-first-build-from-scratch branch July 10, 2026 08:21

lmac-1 mentioned this pull request Jul 10, 2026

AI-First Starting UX: Build from scratch #4895

Closed

jmtdev0 mentioned this pull request Jul 11, 2026

Handle deleted workflows and snapshot failures in API #4965

Draft

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

AI-First Starting UX: Build from scratch#4918

AI-First Starting UX: Build from scratch#4918
lmac-1 merged 18 commits into
4848-ai-first-starting-ux-parentfrom
4895-ai-first-build-from-scratch

lmac-1 commented Jul 2, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

Uh oh!

lmac-1 Jul 7, 2026

Uh oh!

lmac-1 Jul 7, 2026

Uh oh!

elias-ba left a comment

Uh oh!

midigofrank left a comment

Uh oh!

lmac-1 commented Jul 9, 2026

Uh oh!

midigofrank commented Jul 10, 2026

Uh oh!

lmac-1 commented Jul 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

lmac-1 commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Additional notes for the reviewer

Validation steps

AI Usage

Pre-submission checklist

Uh oh!

github-actions Bot commented Jul 7, 2026

Security Review ✅

Uh oh!

Uh oh!

lmac-1 Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

lmac-1 Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

elias-ba left a comment

Choose a reason for hiding this comment

Uh oh!

midigofrank left a comment

Choose a reason for hiding this comment

Uh oh!

lmac-1 commented Jul 9, 2026

Uh oh!

midigofrank commented Jul 10, 2026

Uh oh!

lmac-1 commented Jul 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lmac-1 commented Jul 2, 2026 •

edited

Loading