feat: bundle local (non-pip) modules for serverless endpoints#352
feat: bundle local (non-pip) modules for serverless endpoints#352deanq wants to merge 15 commits into
Conversation
|
Capy auto-review is paused for this organization because the usage-cycle auto-review limit has been reached. Increase the limit or turn it off in billing settings to resume automatic reviews. |
|
Promptless prepared a documentation update related to this change. Triggered by PR #352: feat: bundle local (non-pip) modules for serverless endpoints Documents Flash's new first-class support for bundling local (non-pip) Python modules that endpoints import: a new "Import local modules" section in the Flash endpoints guide (supported import forms, transitive resolution, the 8 MiB live-execution cap, and the parent-directory absolute-import constraint), a force-include note in the build CLI reference, and two troubleshooting entries for the new resolution and payload-size errors. Review: Document Flash local (non-pip) module bundling for endpoints |
Add a shared materialized_modules context manager in runpod_flash.runtime
that writes a {relative_path: source} map to a temp dir, prepends it to
sys.path, and cleans up on exit (including on exception). Wire it into
the /execute LB handler so function_code exec has access to local
modules shipped in the request body's "modules" field.
The LB /execute endpoint parses the request as a plain JSON dict (not a
FunctionRequest model), so modules are read via body.get("modules", {})
rather than an attribute on the FastAPI Request object.
deanq
force-pushed
the
deanquinanola/sls-360-local-module-bundling
branch
from
724bc98 to
0c8c90c
Compare
There was a problem hiding this comment.
Pull request overview
Adds first-class support for bundling/shipping local (non-pip) Python modules that serverless endpoints import, across both live execution (inline module shipping) and build/deploy (force-include local imports in artifacts), with new protocol support via FunctionRequest.modules.
Changes:
- Added a shared local-import resolver (
resolve_local_modules) to compute a transitive closure of local dependencies for endpoint code. - Live execution paths now populate
modules(subject to an 8 MiB cap) and worker runtime materializes shipped modules ontosys.pathfor the duration of the function call. - Build path now force-includes locally imported modules that would otherwise be ignored, and fails builds for unresolved endpoint-local imports.
Reviewed changes
Copilot reviewed 20 out of 22 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_stub_live_serverless.py | Verifies live stub request includes sibling local modules. |
| tests/unit/test_load_balancer_sls_stub.py | Verifies LB stub request includes sibling local modules. |
| tests/unit/stubs/test_local_modules_resolve.py | Unit tests for transitive local module resolution behavior and error cases. |
| tests/unit/stubs/test_local_modules_classify.py | Unit tests for stdlib detection, local path resolution, and size cap constant. |
| tests/unit/stubs/test_live_serverless_modules.py | Tests build_modules_map behavior and size cap enforcement. |
| tests/unit/runtime/test_module_loader.py | Tests temp materialization behavior and sys.path restoration. |
| tests/unit/runtime/test_lb_handler_extended.py | Regression test ensuring shipped modules remain importable during function execution. |
| tests/unit/protos/test_remote_execution.py | Verifies FunctionRequest.modules defaults and round-trips. |
| tests/unit/core/test_exceptions.py | Tests new exception types’ message behavior. |
| tests/unit/cli/commands/test_build_local_modules.py | Tests build-time force-include and endpoint-only strictness behavior. |
| tests/unit/_live_serverless_prepare_request_sibling.py | Sibling module fixture for live-serverless stub tests. |
| tests/unit/_lb_prepare_request_sibling.py | Sibling module fixture for LB stub tests. |
| src/runpod_flash/stubs/local_modules.py | New AST-based resolver for local module import closure and warnings/errors. |
| src/runpod_flash/stubs/load_balancer_sls.py | Populates modules in LB request using build_modules_map. |
| src/runpod_flash/stubs/live_serverless.py | Adds build_modules_map and populates modules in live request with cap enforcement. |
| src/runpod_flash/runtime/module_loader.py | New temp-dir module materializer that prepends to sys.path during execution. |
| src/runpod_flash/runtime/lb_handler.py | Keeps module materialization active through exec + function call/await. |
| src/runpod_flash/protos/remote_execution.py | Adds FunctionRequest.modules protocol field with default empty dict. |
| src/runpod_flash/core/exceptions.py | Adds LocalModuleResolutionError and LocalModulePayloadTooLargeError. |
| src/runpod_flash/cli/commands/build.py | Adds endpoint detection and build-time augmentation to force-include local imports. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Worker uses runpod_flash.runtime.module_loader, which is not in a published runpod-flash yet. Temporary git pin so CI installs a flash with the module. Revert to a released constraint once runpod/flash#352 ships.
- runtime/module_loader: reject module paths that resolve outside the temp dir (modules is untrusted request-body input); restructure cleanup so the temp dir is not leaked when the guard raises mid-loop - cli/build: normalize a raw UnicodeDecodeError from an endpoint's dependency file into LocalModuleResolutionError so run_build() emits a clean error instead of a traceback - add tests for path-escape rejection (incl. mid-loop + disk cleanup) and the wrapped endpoint dependency-decode error
![capy-ai[bot]](https://avatars.githubusercontent.com/in/1915919?s=60&v=4){kind=small}
KAJdev
left a comment
KAJdev
left a comment
There was a problem hiding this comment.
for the most part I think this is okay, though I dont really like that we are re-introducing the AST-walking pattern. I think it's okay just because its far more difficult than trying to do it via inspect and its probably more efficient than sending the entire tree every request. Logistically though, _defines_endpoint should probably live as a step within the scanner somewhere instead of duplicating work that we already do via inspect.
I think what I disagree with is force including files that are ignored by .flashignore, which is a very deliberate user signal that we shouldn't just overlook.
…#100) * feat: import shipped local modules before executing function code * fix(review): address Copilot feedback for #100 - Use getattr(request, "modules", {}) for older-request compatibility - Rename test's shipped module to a unique name to avoid collision with installed/cached modules * build(deps): pin runpod-flash to flash SLS-360 branch for CI Worker uses runpod_flash.runtime.module_loader, which is not in a published runpod-flash yet. Temporary git pin so CI installs a flash with the module. Revert to a released constraint once runpod/flash#352 ships.
Address PR #352 review feedback. - build: replace force-inclusion of ignore-dropped modules with a loud build failure. validate_local_module_imports names each excluded file and its importer and tells the user to remove the ignore pattern or stop importing it, rather than silently overriding a deliberate .gitignore / built-in exclusion. _defines_endpoint stays AST-based (pre-copy) and now only scopes the unresolvable-import strictness. - local_modules: resolve `from pkg import name` submodule candidates (pkg/name.py), not just pkg/__init__.py, so package submodules ship and the worker no longer hits ModuleNotFoundError. A name that is only a re-exported symbol does not back a file and is skipped, never errored.
|
Addressed in 7272691. Force-including ignored files — removed entirely. One correction on the framing:
|
Summary
Flash could not reliably include local (non-pip) Python modules an endpoint imports — a sibling
utils.py, or ahelpers/package next to the endpoint, imported viaimport utils/from helpers import x. This adds first-class support across both code paths that ship user code to a worker.Resolves SLS-360.
@Endpoint(...).run(...),flash devLB dispatch): the endpoint's transitive local-import closure is now shipped inline alongside the function source.flash build/flash deploy): local modules an endpoint imports are force-included in the artifact even when the ignore filter would drop them, and the build fails loudly (clean error, not a traceback) if an endpoint's local import can't be resolved.What changed
FunctionRequest.modules: dict[str, str]— additive protocol field (POSIX relative path → source text), default empty. Backward compatible: old workers ignore it, old SDKs send nothing. Shared contract with the worker repo — see the companion worker PR: feat: SLS-360 import shipped local modules before executing user code runpod-workers/flash#100.stubs/local_modules.py— a shared, pureresolve_local_modules(...)that walks the transitive local-import closure (absolute, relative, andimportlib.import_module("literal")imports, at module level and inside function bodies), classifies local vs stdlib/installed, pulls in package__init__.py, terminates on cycles, warns on non-literal dynamic imports, and raisesLocalModuleResolutionErroron unresolved relative imports / out-of-root files.stubs/live_serverless.pyandstubs/load_balancer_sls.pypopulatemodulesfrom the resolved closure, with an 8 MiB inline cap raisingLocalModulePayloadTooLargeError(points users toflash deployfor larger local dependencies).runtime/module_loader.py::materialized_moduleswrites shipped modules to a temp dir onsys.pathand cleans up; wired intoruntime/lb_handler.py's/executeso the temp dir stays live through the function call (not justexec).augment_with_local_modulesincli/commands/build.pyforce-includes resolved local files past the ignore filter. Strictness is scoped to endpoint files (@Endpoint/@remote): an endpoint with an unresolved local import fails the build loudly; an incidental non-endpoint file that fails resolution is warned and skipped (no regression on unrelated files).Test plan
sys.pathon exception, no-op when empty), both live senders populatemodules, LB/executedef-now/call-later import regression, build force-include + endpoint-only-strict A/B.make quality-checkgreen (86%+ coverage).flash build: an ignoredtest_*.pysibling imported by an endpoint is force-included intoartifact.tar.gz; an endpoint with an unresolvable relative import fails the build with a clean error (exit 1, no traceback).Coordinated release
The worker-side materializer requires the companion worker PR (runpod-workers/flash#100) to be released for the live inline-shipping path to work end-to-end. The deploy path is independent (bundled files sit on the worker filesystem regardless of image version). Live-path E2E against real endpoints is a gating step once the worker image is rebuilt/published.
Follow-ups (non-blocking)
flash deploy.sys.pathmaterialization assumes one invocation at a time per worker; isolate per-invocation if concurrent execution is enabled.ep = Endpoint(...)assignment form is not yet detected (fails safe toward warn).