Repository navigation
inspect and inspect_stamped do not observe frontiers - #834
Merged
frankmcsherry merged 2 commits intoNov 8, 2026
Merged
frankmcsherry merged 2 commits into
frankmcsherry merged 2 commits into
Conversation
`inspect` was built on `inspect_core`, which is scheduled whenever its input frontier changes. A region takes the strongest frontier interest of its children, so one `inspect` kept its whole region awake, and the region did progress work for every child each time any frontier moved. `inspect` now builds on `unary`, which declares no frontier interest. It is scheduled only when records arrive. `inspect_core` still observes frontiers, and its documentation says what that costs. A new test counts how often a region holding only an `inspect` is scheduled as an input advances without data. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`inspect_stamped` runs a closure on each container and the stamp under which it travels. Like `inspect`, it does not observe frontiers, so it is scheduled only when containers arrive. Before this, the only way to see stamps was `inspect_core`, which also observes frontiers and is scheduled whenever they change. `inspect` is now built on `inspect_stamped`, so there is one data-only implementation. The test checks that a region holding either one sleeps while frontiers move. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
inspectno longer observes frontiers, so it no longer keeps an enclosing region awake, and a newinspect_stampeddoes the same for stamped containers.inspectwas built oninspect_core, which is scheduled whenever its input frontier changes.A region takes the strongest frontier interest among its children, so one
inspectinside a region means the region is scheduled every time time moves, and then does progress work for all of its children.This PR adds
inspect_stamped(|stamp, container| ..), built onunary, which declares no frontier interest, so it is scheduled only when containers arrive.inspectis built oninspect_stamped.inspect_corestill observes frontiers, and its documentation now says what that costs; it was previously the only way to see stamps.Data.
N dataflows of
input -> region { map^L -> inspect } -> probe, one record into one input per round, all inputs advanced, one worker, release build.On master
inspectisinspect_coreunderneath, so the left column is master's behavior.inspect_core(master'sinspect)inspect(this PR)This came up in a spreadsheet engine built on differential dataflow (see #833), which observes each formula's output with
inspect.With its 2,000 formulas grouped into regions, an edit that reaches one formula took 1.75 ms per edit with frontier-observing inspects, and 0.07 ms with this change.
The new test counts how often a region holding only an inspect operator is scheduled as an input advances 100 rounds without data: 104 times with
inspect_core, and 5 times with eitherinspect_stampedorinspect.🤖 Generated with Claude Code