WIP: Base branch for epic 1917-java-embed-rust - #2049
Conversation
There was a problem hiding this comment.
Pull request overview
Documents research and an implementation plan for embedding the Rust-based Copilot runtime in the Java SDK.
Changes:
- Records research discussions about native packaging, JNA, and Panama.
- Proposes phased implementation, packaging, testing, and CI work.
- Captures a workflow-skill creation transcript.
Show a summary per file
| File | Description |
|---|---|
20260721-prompts.md |
Records skill-development prompts. |
20260710-prompts.md |
Records JNA and packaging research. |
20260706-prompts.md |
Records native-runtime architecture research. |
1917-embed-cli-runtime-ignorance-reduction-plan.md |
Defines the proposed Java implementation plan. |
Review details
- Files reviewed: 3/4 changed files
- Comments generated: 7
- Review effort level: Medium
| ### C ABI entry points to bind (from .NET PR #1901 and Rust PR #1915) | ||
|
|
||
| | Entry point | Signature (C) | Purpose | | ||
| |-------------|---------------|---------| | ||
| | `copilot_runtime_host_start` | `(const uint8_t* entrypoint, size_t len, const uint8_t* args, size_t args_len) → uint32_t` | Start the runtime host; returns server handle | | ||
| | `copilot_runtime_host_shutdown` | `(uint32_t server) → bool` | Shut down the host | | ||
| | `copilot_runtime_connection_open` | `(uint32_t server, callback, user_data) → uint32_t` | Open a connection; registers outbound callback | | ||
| | `copilot_runtime_connection_write` | `(uint32_t conn, const uint8_t* data, size_t len) → bool` | Write JSON-RPC frame to the runtime | | ||
| | `copilot_runtime_connection_close` | `(uint32_t conn) → bool` | Close a connection | |
|
|
||
| **Spike needed:** Write a minimal JNA program that loads a test `.so`, registers a callback, and verifies callback invocation from a native thread. Confirm JNA's thread attachment behavior. | ||
|
|
||
| **Recommendation:** Use `PipedInputStream`/`PipedOutputStream` to bridge the callback into the existing `JsonRpcClient` input stream model. Hold the `Callback` instance as a field in the transport class (prevents GC). Track active callbacks with `AtomicInteger` and drain on close, mirroring the Rust pattern. |
| 3. **Scope:** Should JNA be a required dependency or optional? If the SDK works without native binaries (subprocess transport), JNA is only needed for InProcess transport. Making it `<optional>true</optional>` means consumers using only CLI transport don't pull it in. | ||
| 4. **GraalVM native-image:** JNA has established `native-image.properties` in its JAR. Verify this works for the callback pattern we need. | ||
|
|
||
| **Recommendation:** Add JNA as an `<optional>true</optional>` dependency. Only required when using InProcess transport. Use `jna` (not `jna-platform`). Version 5.16.0 or later. |
|
|
||
| **Spike needed:** Determine whether the replay proxy can be adapted to work with InProcess transport, or if InProcess tests must use the real runtime binary. | ||
|
|
||
| **Recommendation:** InProcess E2E tests use the real `runtime.node` binary (not the replay proxy). They run only in CI environments where the binary is available, gated by a Maven profile or system property. Existing YAML snapshots are orthogonal (they're for the replay proxy). Unit tests for the binding layer use a test `.so`/`.dylib` with a minimal C ABI surface. |
|
|
||
| Options for resolution order: | ||
|
|
||
| 1. `COPILOT_CLI_PATH` environment variable → explicit path to the runtime binary |
|
|
||
| ### 4.6 — Maven module for per-platform classifier JARs | ||
|
|
||
| **What:** New `copilot-sdk-java-runtime` Maven module that packages `runtime.node` binaries into classifier JARs. |
| **Files to modify:** | ||
| - `.github/workflows/java-sdk-tests.yml` | ||
|
|
||
| **Gating criteria:** CI runs InProcess E2E tests on linux-x64 and darwin-arm64. Tests are skipped gracefully when `runtime.node` is not available. |
7b65681 to
1fe2d25
Compare
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (5)
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:29
- These signatures do not match the merged FFI implementations.
host_startreceives argv JSON and environment JSON, whileconnection_openalso has extension-source, extension-name, and connection-token pointer/length pairs (rust/src/ffi.rs:27-42;dotnet/src/FfiRuntimeHost.cs:369-377). Treating the abbreviated table as the C contract would generate incompatible JNA bindings.
| `copilot_runtime_host_start` | `(const uint8_t* entrypoint, size_t len, const uint8_t* args, size_t args_len) → uint32_t` | Start the runtime host; returns server handle |
| `copilot_runtime_host_shutdown` | `(uint32_t server) → bool` | Shut down the host |
| `copilot_runtime_connection_open` | `(uint32_t server, callback, user_data) → uint32_t` | Open a connection; registers outbound callback |
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:162
- A
PipedOutputStreamwrite can block the native callback thread once its small bounded pipe fills (or before the JSON-RPC reader starts). Both reference transports intentionally copy each callback buffer into a nonblocking unbounded channel (rust/src/ffi.rs:47-65;dotnet/src/FfiRuntimeHost.cs:576-586). The Java design should preserve that property rather than introducing callback-thread backpressure.
**Recommendation:** Use `PipedInputStream`/`PipedOutputStream` to bridge the callback into the existing `JsonRpcClient` input stream model. Hold the `Callback` instance as a field in the transport class (prevents GC). Track active callbacks with `AtomicInteger` and drain on close, mirroring the Rust pattern.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:249
- An optional Maven dependency is not propagated to consumers. With this recommendation, applications can select
IN_PROCESSand include the native classifier yet still fail with missing JNA classes unless they independently discover and declare JNA. Either make JNA a normal implementation dependency or move the JNA transport into an explicitly selected artifact that declares JNA normally.
**Recommendation:** Add JNA as an `<optional>true</optional>` dependency. Only required when using InProcess transport. Use `jna` (not `jna-platform`). Version 5.16.0 or later.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:298
- In-process only replaces the SDK-to-runtime transport; the runtime still makes CAPI requests, so discarding the replay proxy would make this test hit live services or fail authentication. The Rust and .NET in-process harnesses retain the proxy and mirror its proxy/auth environment into the host process (
rust/tests/e2e/support.rs:603-650,793-826;dotnet/test/Harness/E2ETestContext.cs:273-319).
**Recommendation:** InProcess E2E tests use the real `runtime.node` binary (not the replay proxy). They run only in CI environments where the binary is available, gated by a Maven profile or system property. Existing YAML snapshots are orthogonal (they're for the replay proxy). Unit tests for the binding layer use a test `.so`/`.dylib` with a minimal C ABI surface.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:447
- A dedicated in-process CI job must not skip when
runtime.nodeis absent; that would let artifact provisioning or classifier-packaging regressions pass with zero in-process coverage. Match the .NET smoke test's fail-hard behavior and reserve graceful skipping for ordinary local/non-in-process test runs.
**Gating criteria:** CI runs InProcess E2E tests on linux-x64 and darwin-arm64. Tests are skipped gracefully when `runtime.node` is not available.
- Files reviewed: 3/4 changed files
- Comments generated: 3
- Review effort level: Medium
|
|
||
| @FunctionalInterface | ||
| interface OutboundCallback extends Callback { | ||
| void invoke(Pointer userData, Pointer data, int length); |
|
|
||
| The Java SDK currently has no JNA dependency. Adding it introduces: | ||
|
|
||
| 1. **Version selection:** JNA 5.x is current. The latest is 5.16.0 (as of 2025). It supports Java 8+. The SDK targets Java 17. |
| |---------|-------| | ||
| | **Java 17 baseline** | JNA works on Java 17. No Panama FFM. No `--enable-native-access` needed. | | ||
| | **GraalVM native-image** | Verify JNA callback pattern works under native-image. Add reachability metadata if needed. | | ||
| | **Windows path handling** | `runtime.node` on Windows is `copilot_runtime.dll`. Path separators, temp directory behavior differ. | |
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (9)
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:27
- The
host_startcontract is misidentified. The reference implementations serialize the complete argv array and environment map as JSON byte buffers; passing an entrypoint string plus an args string as this table suggests would give the native host malformed input (dotnet/src/FfiRuntimeHost.cs:127-130,358-362).
| `copilot_runtime_host_start` | `(const uint8_t* entrypoint, size_t len, const uint8_t* args, size_t args_len) → uint32_t` | Start the runtime host; returns server handle |
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:29
- This omits six required parameters from the current C ABI.
connection_openalso accepts nullable source, name, and connection-token byte buffers with asize_tlength for each; a JNA binding generated from this table would use the wrong calling convention/stack layout (rust/src/ffi.rs:31-41).
| `copilot_runtime_connection_open` | `(uint32_t server, callback, user_data) → uint32_t` | Open a connection; registers outbound callback |
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:162
- Writing callback bytes to a
PipedOutputStreamcan block the native callback thread once its small pipe buffer fills, potentially stalling the runtime while the JSON-RPC reader is delayed. The .NET and Rust implementations instead copy callback data into unbounded channels; use the analogous queue-backedInputStreamso the callback returns promptly.
**Recommendation:** Use `PipedInputStream`/`PipedOutputStream` to bridge the callback into the existing `JsonRpcClient` input stream model. Hold the `Callback` instance as a field in the transport class (prevents GC). Track active callbacks with `AtomicInteger` and drain on close, mirroring the Rust pattern.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:291
- In-process only replaces the SDK↔runtime JSON-RPC transport; the loaded runtime still makes HTTP requests to CAPI/GitHub. The Rust in-process harness explicitly sets
COPILOT_API_URLandCOPILOT_DEBUG_GITHUB_API_URLto its replay proxy (rust/tests/e2e/support.rs:301-308), so treating the proxy as incompatible would send tests toward live services.
1. Can E2E tests use the InProcess transport against the replay proxy? The replay proxy is a network endpoint — InProcess transport bypasses network entirely. These are different transport paths.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:298
- A real
runtime.nodeand the replay proxy are complementary, not alternatives. The integration test needs the real cdylib to exercise FFI while retaining the proxy and snapshots to keep CAPI traffic deterministic and hermetic; otherwise this plan requires live credentials/network and cannot safely replay existing scenarios.
**Recommendation:** InProcess E2E tests use the real `runtime.node` binary (not the replay proxy). They run only in CI environments where the binary is available, gated by a Maven profile or system property. Existing YAML snapshots are orthogonal (they're for the replay proxy). Unit tests for the binding layer use a test `.so`/`.dylib` with a minimal C ABI surface.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:325
COPILOT_CLI_PATHnames the CLI entrypoint passed throughhost_startargv, not the cdylib. The .NET reference resolves that entrypoint first and separately locates the sibling/prebuilds native library (dotnet/src/Client.cs:2270-2284); interpreting this variable as a library path would attempt to load or invoke the wrong artifact.
1. `COPILOT_CLI_PATH` environment variable → explicit path to the runtime binary
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:331
- This resolution order conflates two independent artifacts. Resolve the CLI entrypoint used by
host_startseparately from the native library loaded by JNA; otherwise the explicit override path cannot work with the current host ABI.
**Recommendation:** Resolution order: `COPILOT_CLI_PATH` (explicit) → classpath resource (classifier JAR) → alongside bundled CLI. This matches the .NET pattern and gives operators an override.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:447
- The dedicated InProcess CI jobs must fail when
runtime.nodeis missing; skipping would make a broken download/package/classifier pipeline appear green. The Rust reference test explicitly “fails hard if the in-process runtime library cannot be loaded” (rust/tests/e2e/inprocess.rs:3-9).
**Gating criteria:** CI runs InProcess E2E tests on linux-x64 and darwin-arm64. Tests are skipped gracefully when `runtime.node` is not available.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:123
- The native callback length is
size_t, so mapping it to Javainttruncates the ABI on every supported 64-bit target. Define a pointer-widthsize_tJNA type (including Windows LLP64) and use that consistently for callback and downcall lengths before treating this sketch as the binding contract.
void invoke(Pointer userData, Pointer data, int length);
- Files reviewed: 3/4 changed files
- Comments generated: 1
- Review effort level: Medium
|
|
||
| **Question:** Should the InProcess transport API be annotated with `@CopilotExperimental`? | ||
|
|
||
| The existing SDK marks experimental features with `@CopilotExperimental` (compile-time check via `CopilotExperimentalProcessor`). The .NET PR's InProcess transport appears to be non-experimental (it's opt-in via connection type). The Rust PR's `Transport::InProcess` is additive. |
1fe2d25 to
69d5490
Compare
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (6)
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:30
- These two declarations do not match the ABI used by the referenced implementations.
host_startreceives JSON-encoded argv and environment buffers, whileconnection_openalso requires the extension-source, extension-name, and connection-token buffer/length pairs (go/internal/ffihost/ffihost.go:19-28,dotnet/src/FfiRuntimeHost.cs:358-377). Implementing this table would omit required native arguments and can cause failed startup or ABI-level memory corruption; update the laterNativeBindingsketch consistently.
| `copilot_runtime_host_start` | `(const uint8_t* entrypoint, size_t len, const uint8_t* args, size_t args_len) → uint32_t` | Start the runtime host; returns server handle |
| `copilot_runtime_host_shutdown` | `(uint32_t server) → bool` | Shut down the host |
| `copilot_runtime_connection_open` | `(uint32_t server, callback, user_data) → uint32_t` | Open a connection; registers outbound callback |
| `copilot_runtime_connection_write` | `(uint32_t conn, const uint8_t* data, size_t len) → bool` | Write JSON-RPC frame to the runtime |
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:123
- The native callback's length parameter is
size_t, but this sketch narrows it to Javaint. All supported runtime targets are 64-bit, so the JNA callback declaration must use a pointer-width-compatible mapping; otherwise the Java declaration does not faithfully match the C ABI and large lengths are truncated.
void invoke(Pointer userData, Pointer data, int length);
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:162
- A
PipedOutputStreamwrite blocks when its small pipe buffer fills. Because this write would run inside the native callback, slow JSON-RPC consumption or shutdown can block a runtime-owned thread and deadlock while close waits for active callbacks. Keep the callback limited to copying and enqueueing bytes, as the referenced Rust implementation does with an unbounded channel (rust/src/ffi.rs:45-65,143-146), and expose a queue-backedInputStreamtoJsonRpcClient.
**Recommendation:** Use `PipedInputStream`/`PipedOutputStream` to bridge the callback into the existing `JsonRpcClient` input stream model. Hold the `Callback` instance as a field in the transport class (prevents GC). Track active callbacks with `AtomicInteger` and drain on close, mirroring the Rust pattern.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:291
- In-process only replaces the SDK-to-runtime stdio/TCP hop; the runtime still makes HTTP requests to CAPI. The Rust in-process suite explicitly passes
COPILOT_API_URLandCOPILOT_DEBUG_GITHUB_API_URLto the same replay proxy (rust/tests/e2e/support.rs:298-308,603-644), and Java's harness already configures those values. Avoiding the proxy would make this test non-hermetic and risk live API access instead of replaying snapshots.
1. Can E2E tests use the InProcess transport against the replay proxy? The replay proxy is a network endpoint — InProcess transport bypasses network entirely. These are different transport paths.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:325
- This redefines the established meaning of
COPILOT_CLI_PATH. In the current Rust and .NET in-process implementations it points to the CLI entrypoint, and the host resolves the sibling cdylib (rust/src/resolve.rs:39-43,dotnet/src/Client.cs:2270-2289). Treating it as aruntime.nodepath would makehost_startreceive a shared library as argv[0] and break the residual-worker startup path. Resolve the CLI entrypoint and native library separately; use a new option only if a direct native-library override is needed.
1. `COPILOT_CLI_PATH` environment variable → explicit path to the runtime binary
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:420
- Packaging only
runtime.nodeis not sufficient for the currently merged C ABI host.host_startstill launches the residual worker using a packaged CLI entrypoint (dotnet/src/FfiRuntimeHost.cs:20-23,73-101;rust/tests/e2e/support.rs:190-200). Add an explicit implementation gate: either provision a version-matched CLI entrypoint alongside each native artifact during the transition, or defer this phase until the residual-worker requirement is removed. Otherwise the classifier JARs can pass packaging checks but cannot start.
**What:** New `copilot-sdk-java-runtime` Maven module that packages `runtime.node` binaries into classifier JARs.
- Files reviewed: 3/4 changed files
- Comments generated: 0 new
- Review effort level: Medium
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (9)
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:124
- The callback maps C
size_tto Javaint, but all supported x64/arm64 targets use a 64-bitsize_t. This gives JNA the wrong native callback signature and can truncate the payload length. Use a 64-bit mapping (or a customsize_tIntegerType) and bounds-check before copying into a Java array.
void invoke(Pointer userData, Pointer data, int length);
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:503
- This contradicts the plan's packaging convention: the Windows classifier JAR contains
native/win32-*/runtime.node.copilot_runtime.dllis a natural name used only if the extracted file is deliberately renamed (as the .NET build does), so the plan must distinguish the resource name from the extracted filename.
### 4.8 — CI workflow updates
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:11
- The blanket “at build time” statement is inaccurate: Python downloads the native runtime lazily when InProcess is selected (
python/copilot/_cli_download.py:376-432), while Node resolves packages during installation. Rephrase this as a set of provisioning paths so the supply-chain analysis does not misstate when network access occurs.
Every non-Java SDK in the monorepo downloads `@github/copilot-{platform}` tarballs directly from `registry.npmjs.org` at build time:
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:16
- Python's registry is not hardcoded-only: both its packument and tarball URL builders honor
COPILOT_NPM_REGISTRY_URL(python/copilot/_cli_version.py:145-162). Omitting that override makes the comparison and mitigation assessment incorrect.
- **Python** — `python/copilot/_cli_version.py` (line 44): Hardcoded `_NPM_REGISTRY_BASE_URL = "https://registry.npmjs.org"`
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:23
- This mitigation summary is outdated for Python. It supports a registry override and verifies npm Subresource Integrity, failing closed when integrity is unavailable (
python/copilot/_cli_version.py:145-162,_cli_download.py:325-360,421-432).
- .NET is the only SDK that makes the registry URL configurable (`CopilotNpmRegistryUrl` property), so enterprise consumers can point at a private/mirrored registry.
- Rust verifies SHA-512 integrity after download.
- Go and Python have no registry override and no integrity verification visible in the code I examined.
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:7
- Use the contraction “it's” (“it is”), not the possessive “its.”
I realize we're at GitHub, and its different here, but might we want to be SFI proactive here?
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:297
- In-process hosting only replaces the SDK-to-runtime transport; the runtime still makes HTTP requests to CAPI/GitHub. The Rust in-process suite proves this by propagating
COPILOT_API_URLandCOPILOT_DEBUG_GITHUB_API_URLfrom its replay context (rust/tests/e2e/support.rs:39-44,301-307). Following this recommendation would bypass the repository's deterministic snapshots and risk hitting real services.
1. Can E2E tests use the InProcess transport against the replay proxy? The replay proxy is a network endpoint — InProcess transport bypasses network entirely. These are different transport paths.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:332
COPILOT_CLI_PATHalready means the CLI executable/entrypoint, not the native library (seedotnet/src/Client.cs:2270-2289and the Java test harness). Reinterpreting it asruntime.nodewould break the established cross-SDK contract; the .NET implementation uses it to find the entrypoint and then resolves the sibling cdylib.
1. `COPILOT_CLI_PATH` environment variable → explicit path to the runtime binary
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:119
- This proposed binding abstraction drops required C ABI inputs:
host_startneeds both argv JSON and environment JSON, andconnection_openaccepts extension source/name and connection-token buffers. In particular, omittingenv_jsonprevents the host from receiving auth/proxy configuration. Keep the binding interface aligned with the five signatures documented above; higher-level entrypoint/argument conversion belongs inFfiRuntimeHost.
int hostStart(String entrypoint, String args);
boolean hostShutdown(int serverHandle);
int connectionOpen(int serverHandle, OutboundCallback callback, Pointer userData);
boolean connectionWrite(int connectionHandle, byte[] data);
boolean connectionClose(int connectionHandle);
- Files reviewed: 6/7 changed files
- Comments generated: 3
- Review effort level: Medium
|
|
||
| **Question:** What is the exact implementation of platform detection, particularly the ELF PT_INTERP parsing for musl vs. glibc on Linux? | ||
|
|
||
| ADR-007 specifies reading the first 2 KB of `/proc/self/exe` and parsing the ELF PT_INTERP segment. This is the same approach as the `detect-libc` npm package. |
|
|
||
| Hi Stephen, | ||
|
|
||
| As I was working on #1917, I was reminded of an SFI concern for which I forced the mitigation on my previous team (Copilot modernization, under Dmirty (now retired)). Specifically: the team was pulling artifacts directly from npmjs.org, rather from an internal AzDO pipeline that lists npmjs.org as an upstream. |
|
|
||
| - `.github/workflows/java-sdk-tests.yml` | ||
|
|
||
| **Gating criteria:** CI runs InProcess E2E tests on linux-x64 and darwin-arm64. Tests are skipped gracefully when `runtime.node` is not available. |
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (10)
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:22
- The current runtime is not usable from
runtime.nodealone: both reference hosts put a packaged CLI entrypoint inargv[0], andhost_startuses it to launch the residual worker. The plan must either package that entrypoint alongside each runtime (and update the size/distribution analysis) or explicitly gate delivery on a runtime version that no longer needs it; otherwise the planned classifier JARs cannot start today's runtime.
1. Ship per-platform classifier JARs containing the `runtime.node` binary for each of the 8 platform targets (Option 2).
2. Support uber-jar assembly via `maven-assembly-plugin` that merges all (or a subset of) platform JARs into a single distributable artifact (Option 1 compatibility).
3. Detect the current platform at runtime, extract the matching native binary, and load it via JNA to call the 5 `extern "C"` entry points of the runtime's C ABI front door.
4. Bridge bidirectional JSON-RPC transport over the FFI boundary (Java → native downcalls, native → Java upcall callbacks).
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:119
- This sketch still uses the superseded ABI shape.
host_startreceives argv/environment JSON buffers, andconnection_openalso requires the nullable extension source, extension name, and connection-token buffers. Leaving the old shape here will guide the binding implementation to omit required arguments.
int hostStart(String entrypoint, String args);
boolean hostShutdown(int serverHandle);
int connectionOpen(int serverHandle, OutboundCallback callback, Pointer userData);
boolean connectionWrite(int connectionHandle, byte[] data);
boolean connectionClose(int connectionHandle);
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:124
- The native callback’s
lenissize_t, but Javaintis only 32 bits. All targeted platforms are 64-bit, so the callback mapping must use a 64-bit value (or a dedicated JNAsize_tmapping) to match the ABI.
void invoke(Pointer userData, Pointer data, int length);
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:166
PipedOutputStream.writecan block the native callback thread once its small buffer fills, which can stall runtime progress and deadlock shutdown while waiting for active callbacks to drain. Copy callback bytes into a nonblocking queue and expose a queue-backedInputStream, as the Rust implementation does with an unbounded channel.
**Recommendation:** Use `PipedInputStream`/`PipedOutputStream` to bridge the callback into the existing `JsonRpcClient` input stream model. Hold the `Callback` instance as a field in the transport class (prevents GC). Track active callbacks with `AtomicInteger` and drain on close, mirroring the Rust pattern.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:214
- A fixed 2 KB prefix is not sufficient to parse ELF reliably: ELF permits the program-header table and the
PT_INTERPpayload at offsets beyond that prefix. The implementation should read the ELF header, then the declared program-header table, then the interpreter bytes at the segment’s declared offset.
ADR-007 specifies reading the first 2 KB of `/proc/self/exe` and parsing the ELF PT_INTERP segment. This is the same approach as the `detect-libc` npm package.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:338
COPILOT_CLI_PATHpoints to a CLI entrypoint, not directly toruntime.node. The reference hosts use that entrypoint in argv and locate the native library beside it; treating the variable as a library path would make Java attempt to load an executable through JNA.
1. `COPILOT_CLI_PATH` environment variable → explicit path to the runtime binary
2. Classpath resource (`native/<classifier>/runtime.node`) → from classifier JAR
3. Bundled CLI location (existing `CliServerManager` path) → the current subprocess path, but load the `.so`/`.dylib`/`.dll` sibling
The .NET PR resolves the entrypoint from `COPILOT_CLI_PATH` and falls back to the bundled CLI location. The Rust PR discovers or extracts the platform library alongside the embedded CLI.
**Recommendation:** Resolution order: `COPILOT_CLI_PATH` (explicit) → classpath resource (classifier JAR) → alongside bundled CLI. This matches the .NET pattern and gives operators an override.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:267
- This section still describes the old “entrypoint + args” ABI even though the table above was corrected. The ABI accepts serialized argv and optional environment JSON; in the reference implementations argv contains the CLI entrypoint and
--embedded-hostarguments. Keeping the obsolete questions leaves the core packaging requirement unresolved.
The .NET PR passes an `entrypoint` path and `args`. The Rust PR similarly passes entrypoint and args as byte buffers. We need to understand:
1. What is the `entrypoint` parameter? Is it the path to the `runtime.node` binary itself, or a path to a Node.js entry script?
2. What are the `args`? JSON-formatted startup options? CLI-style flags?
3. Does the host need the `runtime.node` file path passed as entrypoint, or does it use the loaded library's own location?
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:301
- This section incorrectly treats the native runtime and replay proxy as alternatives. In-process changes only the SDK↔runtime transport; the runtime still makes CAPI requests. The Rust in-process suite launches the same replay proxy, sets
COPILOT_API_URL/proxy environment, and reuses snapshots, so Java should do likewise to avoid live CAPI calls.
1. Can E2E tests use the InProcess transport against the replay proxy? The replay proxy is a network endpoint — InProcess transport bypasses network entirely. These are different transport paths.
2. Should InProcess E2E tests use a **real** `runtime.node` binary? This would require the binary to be available in CI.
3. How do we mock/stub the native library for unit testing the JNA binding layer without a real `runtime.node`?
4. Should InProcess E2E tests reuse existing YAML snapshots, or do they need separate snapshots?
5. **Should the entire existing E2E test suite be run with each valid transport (subprocess and InProcess)?** The Rust PR does this — the same E2E tests run in a separate CI job with `COPILOT_SDK_DEFAULT_CONNECTION=inprocess`, providing confidence that both transport paths produce identical behavior. The researcher should determine whether the Java E2E suite can be structured the same way (e.g., a separate Maven profile or CI matrix entry that sets the transport to InProcess and re-runs the full suite).
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:5
- The comparison requires “rather than,” not “rather.”
As I was working on #1917, I was reminded of an SFI concern for which I forced the mitigation on my previous team (Copilot modernization, under Dmirty (now retired)). Specifically: the team was pulling artifacts directly from npmjs.org, rather from an internal AzDO pipeline that lists npmjs.org as an upstream.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:365
- Correct the typo in this retained plan text.
Where is this `runtimes` direcory? Is it committed to `git`? I doubt that. Is it in `~/.copilot`?
- Files reviewed: 6/7 changed files
- Comments generated: 7
- Review effort level: Medium
|
|
||
| - **[napi](#references) front door** — loaded by a Node.js process as a native addon (current CLI path). | ||
| - **[C ABI](#references) front door** — a fixed set of approximately 12 `extern "C"` lifecycle and transport entry points (`copilot_runtime_server_create`, `copilot_runtime_connection_open`, etc.) that any language can call in-process via [FFI](#references) ([JNA](#references) for Java, Python/cffi, C#/`DllImport`, Go/purego) **without a Node.js process**. All API methods travel as JSON-RPC data through this fixed transport; the export list never changes as the method set grows. | ||
| - **[C ABI](#references) front door** — a fixed set of 5 `extern "C"` lifecycle and transport entry points that any language can call in-process via [FFI](#references) ([JNA](#references) for Java, Python/cffi, C#/`DllImport`, Go/purego) **without a Node.js process**. All API methods travel as JSON-RPC data through this fixed transport; the export list never changes as the method set grows. The 5 entry points are: |
| /** Use IN_PROCESS if native binary available, else fall back to CLI. */ | ||
| DEFAULT |
|
|
||
| ## What I found | ||
|
|
||
| Every non-Java SDK in the monorepo downloads `@github/copilot-{platform}` tarballs directly from `registry.npmjs.org` at build time: |
| - **.NET** — `dotnet/src/build/GitHub.Copilot.SDK.targets` (line 58): `DownloadFile` from `registry.npmjs.org` (configurable via `CopilotNpmRegistryUrl` property) | ||
| - **Rust** — `rust/build/in_process.rs` (line 86): Hardcoded `registry.npmjs.org` download in `build.rs` | ||
| - **Go** — `go/cmd/bundler/main.go` (line 38): Hardcoded `registry.npmjs.org` download URL | ||
| - **Python** — `python/copilot/_cli_version.py` (line 44): Hardcoded `_NPM_REGISTRY_BASE_URL = "https://registry.npmjs.org"` |
| - .NET is the only SDK that makes the registry URL configurable (`CopilotNpmRegistryUrl` property), so enterprise consumers can point at a private/mirrored registry. | ||
| - Rust verifies SHA-512 integrity after download. | ||
| - Go and Python have no registry override and no integrity verification visible in the code I examined. |
|
|
||
| As I was working on #1917, I was reminded of an SFI concern for which I forced the mitigation on my previous team (Copilot modernization, under Dmirty (now retired)). Specifically: the team was pulling artifacts directly from npmjs.org, rather from an internal AzDO pipeline that lists npmjs.org as an upstream. | ||
|
|
||
| I realize we're at GitHub, and its different here, but might we want to be SFI proactive here? |
|
|
||
| ### 3.15 Additional human generated questions while reviewing the first draft of this plan, committed in 292a9036aa | ||
|
|
||
| 1. Is the set of C ABI entry points listed in the table at "C ABI entry points to bind" sufficient? I thought ypou said there were "12 `extern "C"` entry points? That table only has 5. |
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (12)
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:11
- This summary is inaccurate for Python: its tarball is fetched when the in-process runtime is provisioned, not during the package build. Rephrase the claim so it does not characterize every SDK as a build-time downloader.
Every non-Java SDK in the monorepo downloads `@github/copilot-{platform}` tarballs directly from `registry.npmjs.org` at build time:
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:16
- This bullet treats the fallback constant as a hardcoded-only endpoint, but the current Python implementation uses
COPILOT_NPM_REGISTRY_URLin both URL builders (python/copilot/_cli_version.py:149,161). The note should record that override rather than reporting Python as fixed to npmjs.org.
- **Python** — `python/copilot/_cli_version.py` (line 44): Hardcoded `_NPM_REGISTRY_BASE_URL = "https://registry.npmjs.org"`
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:23
- The Python mitigation inventory is outdated. Python now supports a registry override and verifies npm Subresource Integrity, failing closed when integrity is absent or mismatched (
python/copilot/_cli_version.py:149-161,python/copilot/_cli_download.py:421-432).
- .NET is the only SDK that makes the registry URL configurable (`CopilotNpmRegistryUrl` property), so enterprise consumers can point at a private/mirrored registry.
- Rust verifies SHA-512 integrity after download.
- Go and Python have no registry override and no integrity verification visible in the code I examined.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:153
- This proposed interface still models the obsolete
entrypoint/argsform and omits the three nullable metadata buffers fromconnection_open, contradicting the corrected ABI table above. If implementation follows this sample, it cannot forward external source/name or connection token and will bind the wrong host-start semantics.
int hostStart(String entrypoint, String args);
boolean hostShutdown(int serverHandle);
int connectionOpen(int serverHandle, OutboundCallback callback, Pointer userData);
boolean connectionWrite(int connectionHandle, byte[] data);
boolean connectionClose(int connectionHandle);
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:158
- The callback's C
size_t lenis 64-bit on every supported target, but this JNA sketch maps it to a 32-bit Javaint. That does not faithfully describe the ABI and can truncate the callback length; use a 64-bit mapping (or a dedicated JNAsize_ttype).
void invoke(Pointer userData, Pointer data, int length);
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:200
- Writing callback payloads directly into a
PipedOutputStreamcan block the native callback thread whenever the pipe fills (the default pipe is small), potentially stalling the runtime during startup or while JSON-RPC consumption is delayed. The callback should synchronously copy the native buffer into a nonblocking queue/channel and return; anInputStreamadapter can consume that queue separately.
**Recommendation:** Use `PipedInputStream`/`PipedOutputStream` to bridge the callback into the existing `JsonRpcClient` input stream model. Hold the `Callback` instance as a field in the transport class (prevents GC). Track active callbacks with `AtomicInteger` and drain on close, mirroring the Rust pattern.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:248
- A fixed 2 KB read cannot parse every valid ELF: the ELF header gives the program-header table offset/count, and the selected
PT_INTERPsegment's bytes may lie beyond that prefix. The implementation should read the header and program headers first, then seek to the segment's declared offset and length.
ADR-007 specifies reading the first 2 KB of `/proc/self/exe` and parsing the ELF PT_INTERP segment. This is the same approach as the `detect-libc` npm package.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:301
- This section was not updated with the corrected ABI.
host_startreceives oneargv_jsonbuffer and one optionalenv_jsonbuffer; the CLI entrypoint is an element of the argv array, while the loadedruntime.nodepath is not a host-start argument. Leaving these obsolete questions in the implementation plan reintroduces the ambiguity this PR intended to resolve.
The .NET PR passes an `entrypoint` path and `args`. The Rust PR similarly passes entrypoint and args as byte buffers. We need to understand:
1. What is the `entrypoint` parameter? Is it the path to the `runtime.node` binary itself, or a path to a Node.js entry script?
2. What are the `args`? JSON-formatted startup options? CLI-style flags?
3. Does the host need the `runtime.node` file path passed as entrypoint, or does it use the loaded library's own location?
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:331
- In-process hosting bypasses only the SDK-to-runtime JSON-RPC process transport; the runtime still performs HTTP requests to CAPI/GitHub endpoints. Existing Rust in-process E2Es explicitly set
COPILOT_API_URLandCOPILOT_DEBUG_GITHUB_API_URLto the replay proxy (rust/tests/e2e/support.rs:301-307), so these are not mutually exclusive paths.
1. Can E2E tests use the InProcess transport against the replay proxy? The replay proxy is a network endpoint — InProcess transport bypasses network entirely. These are different transport paths.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:339
- Running the real runtime without the replay proxy would either contact live services or fail, and it would discard the deterministic snapshot coverage that enables the full dual-transport suite. Keep the real
runtime.nodefor the FFI boundary while routing its outbound model/API traffic through the existing replay proxy and snapshots, as the Rust suite does.
**Spike needed:** Determine whether the replay proxy can be adapted to work with InProcess transport, or if InProcess tests must use the real runtime binary. Determine whether the full E2E suite can run under both transports, or if certain tests are inherently transport-specific.
**Recommendation:** InProcess E2E tests use the real `runtime.node` binary (not the replay proxy). They run only in CI environments where the binary is available, gated by a Maven profile or system property. Existing YAML snapshots are orthogonal (they're for the replay proxy). Unit tests for the binding layer use a test `.so`/`.dylib` with a minimal C ABI surface. The full E2E suite should be run under both subprocess and InProcess transports in CI, mirroring the Rust PR's approach.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:366
COPILOT_CLI_PATHis an override for the CLI entrypoint included inargv_json, not a path toruntime.node. The .NET implementation resolves the native library separately beside that entrypoint. Repurposing this variable as the library path would makehost_starttry to launch the cdylib as its embedded-host program.
1. `COPILOT_CLI_PATH` environment variable → explicit path to the runtime binary
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:545
- A dedicated InProcess CI job that silently skips when its required native artifact is missing can remain green while exercising no in-process behavior. Since the preceding step provisions the artifact and the plan forbids skipped tests, these jobs should fail fast when
runtime.nodeis unavailable.
**Gating criteria:** CI runs InProcess E2E tests on linux-x64 and darwin-arm64. Tests are skipped gracefully when `runtime.node` is not available.
- Files reviewed: 6/7 changed files
- Comments generated: 1
- Review effort level: Medium
| <archive> | ||
| <manifestEntries> | ||
| <Copilot-Runtime-Platform>darwin-arm64</Copilot-Runtime-Platform> | ||
| <Copilot-Runtime-Version>${project.version}</Copilot-Runtime-Version> |
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (7)
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:120
- This proposed interface still models the obsolete ABI:
hostStartdropsenv_json,connectionOpendropsext_source,ext_name, andconn_token, and the callback maps nativesize_tto Javaint. Following this sketch would prevent environment/auth metadata from reaching the host and can misdeclare the callback on 64-bit platforms. Update it to match the five signatures documented at lines 26–34.
int hostStart(String entrypoint, String args);
boolean hostShutdown(int serverHandle);
int connectionOpen(int serverHandle, OutboundCallback callback, Pointer userData);
boolean connectionWrite(int connectionHandle, byte[] data);
boolean connectionClose(int connectionHandle);
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:306
- In-process FFI bypasses only the SDK↔runtime process transport; the runtime still makes outbound CAPI network requests. The Rust in-process suite starts the replay proxy, passes
COPILOT_API_URL/proxy environment into the host, and reuses snapshots (rust/tests/e2e/support.rs:289-307,786-903). Running the real cdylib “not the replay proxy” would instead hit live services and make these tests nondeterministic.
**Recommendation:** InProcess E2E tests use the real `runtime.node` binary (not the replay proxy). They run only in CI environments where the binary is available, gated by a Maven profile or system property. Existing YAML snapshots are orthogonal (they're for the replay proxy). Unit tests for the binding layer use a test `.so`/`.dylib` with a minimal C ABI surface. The full E2E suite should be run under both subprocess and InProcess transports in CI, mirroring the Rust PR's approach.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:482
- Packaging only
runtime.nodeis insufficient while the active runtime still launches the residual TypeScript worker:host_startreceives a CLI entrypoint, and both existing hosts bundle/resolve that executable alongside the cdylib (rust/src/ffi.rs:230-306,dotnet/src/FfiRuntimeHost.cs:73-101). Without also provisioning that platform CLI binary, Java still requires an externalCOPILOT_CLI_PATHand does not meet the plan's stated goal. Decide whether classifier JARs include both artifacts or implementation waits until the residual worker is gone.
**What:** New `copilot-sdk-java-runtime` Maven module that packages `runtime.node` binaries into classifier JARs.
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:11
- Python is no longer part of this “build time” statement. Its runtime library is downloaded lazily only when in-process transport is selected (or by the explicit download command), as documented in
python/copilot/_cli_download.py:376-387. Distinguish build-time acquisition from Python's runtime provisioning so the note accurately scopes the exposure.
Every non-Java SDK in the monorepo downloads `@github/copilot-{platform}` tarballs directly from `registry.npmjs.org` at build time:
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:23
- This mitigation inventory is stale. Python honors
COPILOT_NPM_REGISTRY_URLfor both packument and tarball URLs (_cli_version.py:145-162) and refuses to load unless npm Subresource Integrity is available and verifies (_cli_download.py:325-360,418-432). Sending the current text would incorrectly report two missing Python mitigations.
- .NET is the only SDK that makes the registry URL configurable (`CopilotNpmRegistryUrl` property), so enterprise consumers can point at a private/mirrored registry.
- Rust verifies SHA-512 integrity after download.
- Go and Python have no registry override and no integrity verification visible in the code I examined.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:512
- A dedicated in-process CI job should fail when its required
runtime.nodeis missing. Skipping would turn a broken package/download/profile configuration—the exact integration this job is meant to validate—into green CI and conflicts with the no-skipping test discipline above.
**Gating criteria:** CI runs InProcess E2E tests on linux-x64 and darwin-arm64. Tests are skipped gracefully when `runtime.node` is not available.
1917-java-embed-rust-cli-runtime-remove-before-merge/1917-embed-cli-runtime-ignorance-reduction-plan.md:268
- This section reverted to the old
entrypoint + argsABI description. Both reference implementations buildargv_json(including the residual CLI entrypoint and arguments) and pass a separate optionalenv_json;runtime.nodeis loaded independently. Keeping these questions sends the spike toward a contract that the corrected table has already ruled out.
The .NET PR passes an `entrypoint` path and `args`. The Rust PR similarly passes entrypoint and args as byte buffers. We need to understand:
1. What is the `entrypoint` parameter? Is it the path to the `runtime.node` binary itself, or a path to a Node.js entry script?
2. What are the `args`? JSON-formatted startup options? CLI-style flags?
3. Does the host need the `runtime.node` file path passed as entrypoint, or does it use the loaded library's own location?
- Files reviewed: 6/7 changed files
- Comments generated: 3
- Review effort level: Medium
|
|
||
| - **[napi](#references) front door** — loaded by a Node.js process as a native addon (current CLI path). | ||
| - **[C ABI](#references) front door** — a fixed set of approximately 12 `extern "C"` lifecycle and transport entry points (`copilot_runtime_server_create`, `copilot_runtime_connection_open`, etc.) that any language can call in-process via [FFI](#references) ([JNA](#references) for Java, Python/cffi, C#/`DllImport`, Go/purego) **without a Node.js process**. All API methods travel as JSON-RPC data through this fixed transport; the export list never changes as the method set grows. | ||
| - **[C ABI](#references) front door** — a fixed set of 5 `extern "C"` lifecycle and transport entry points that any language can call in-process via [FFI](#references) ([JNA](#references) for Java, Python/cffi, C#/`DllImport`, Go/purego) **without a Node.js process**. All API methods travel as JSON-RPC data through this fixed transport; the export list never changes as the method set grows. The 5 entry points are: |
| 3. Which CI runner platforms need InProcess test coverage? (linux-x64 and darwin-arm64 minimum?) | ||
| 4. Should InProcess tests be gated behind a `runtime.node` availability check to avoid failing when the binary isn't present? | ||
|
|
||
| **Recommendation:** Modify the existing `java-sdk-tests.yml` to add InProcess test jobs on linux-x64 and darwin-arm64 runners. Native binaries are downloaded from the `copilot-agent-runtime` release artifacts. InProcess tests run as a separate Maven profile. |
| **What:** `FfiRuntimeHost` class that manages the full lifecycle: `host_start` → `connection_open` → duplex stream bridging → `connection_close` → `host_shutdown`. Provides `InputStream`/`OutputStream` compatible with `JsonRpcClient`. | ||
|
|
||
| **Files to create:** | ||
|
|
||
| - `java/src/main/java/com/github/copilot/ffi/FfiRuntimeHost.java` | ||
|
|
- Supersedes Transport enum recommendation - Adopt .NET RuntimeConnection pattern: sealed base + 4 subtypes (StdioRuntimeConnection, TcpRuntimeConnection, UriRuntimeConnection, InProcessRuntimeConnection) - setConnection(RuntimeConnection) on CopilotClientOptions, nullable - Backward compat: null connection infers from legacy useStdio/cliUrl/cliPath - Package: com.github.copilot.rpc
- New FfiRuntimeHost class manages FFI lifecycle (host_start, connection_open, duplex streams, connection_close, host_shutdown) - Parallel to CliServerManager, not an extension — zero mechanics overlap - Package: com.github.copilot.ffi - Shared arg/env building: extract static helper if duplication warrants it - CopilotClient.startCoreBody() dispatches on RuntimeConnection subtype
…iOutputStream - JsonRpcClient.fromStreams(InputStream, OutputStream) — one-line factory method - Read side: QueueInputStream from 3.4 spike (BlockingQueue fed by JNA callback) - Write side: FfiOutputStream wraps connection_write (trivial OutputStream) - PipedInputStream explicitly rejected (3.4 spike proved thread-affinity failure) - Cleanup: FfiRuntimeHost owns native lifecycle, not JsonRpcClient - No spike needed — 3.4 spike proved the hard part
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: cd4b9349-0650-4f20-9efc-13afc7946487
Add the isolated Spike 3.8 experiment for the callback-dependent runtime ABI using JNA 5.19.1 and a Rust native fixture. - verify one synchronous callback and five Rust-thread callbacks on the JVM - build and run a GraalVM Native Image executable with application-specific proxy, reflection, and JNI reachability metadata - demonstrate that JNA callback upcalls fail before entering Java callback code in the tested Windows x64 GraalVM 25.0.4 configuration - document the exact toolchain and platform constraints so the result is not extrapolated to untested systems - resolve Question 3.8 with actionable JNA version, artifact, optionality, consumer setup, and Native Image support decisions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The original four questions in 3.9 targeted a parameter ("entrypoint") that
does not exist in the actual ABI signature, and covered only host_start while
leaving connection_open's three nullable metadata buffers (ext_source,
ext_name, conn_token) completely unaddressed.
Rewrite 3.9 as "C ABI parameter semantics" with eleven numbered questions
spanning all five functions:
- argv_json: full valid-argument set and --embedded-host semantics
- env_json: complete key inventory (auth token, proxy URLs, log level, others)
- argv_json / env_json nullability and zero-length handling
- host_start failure and error-retrieval companion function
- ext_source and ext_name semantics, encoding, and null-safety
- conn_token: per-connection vs. global auth, format
- user_data: confirm null is safe; Java should use closure capture not the
void-pointer cookie mechanism
- multiple concurrent connections per server handle
- wire frame format for connection_write and on_outbound
- buffer lifetime for connection_write
Name the spike spike-3-9-deep-entrypoint-questions and require a complete
call-by-call reference as the deliverable.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…arams, one-lib-per-process
- Add spike-3-9-c-abi-parameter-semantics/ with complete call-by-call parameter reference derived from all 5 SDK implementations - Add inline resolution to 3.9 in the ignorance reduction plan - Answers all 11 open questions: argv/env format, nullability, wire framing (LSP Content-Length), buffer lifetime, ext_source/ext_name/ conn_token (all null), user_data (null safe), one connection per server
- No error message channel in C ABI; construct diagnostic strings - Use IllegalStateException (matching .NET InvalidOperationException) - Native crash = process termination; do not enable JNA protected mode - java.util.logging at FINE for startup/teardown, WARNING for callbacks - Callback try-catch wrapping (Go/Python pattern, better than .NET) - JNA UncaughtExceptionHandler as secondary defense - AutoCloseable.close() never throws; best-effort teardown
Adds spike-3-11-replay-proxy-and-in-process with a standalone Maven Java program (JDK 17 baseline, JNA 5.19.1) that proves the complete InProcess transport flow on win32-x64: host_start (1.1 s) -> connection_open -> LSP-framed ping write -> on_outbound callback (Thread-0) -> pong parse -> connection_close -> host_shutdown Key artifacts: - InProcessEnvGuard.java: calls SetEnvironmentVariableW (Windows) or setenv() (Unix) via JNA to mutate the native process env block so the loaded runtime.node reads COPILOT_API_URL and friends. Required for replay-proxy redirection with InProcess transport. - CopilotRuntimeLibrary.java: JNA interface for the 5 C ABI functions; size_t -> Java long for win64 correctness. - QueueInputStream.java: callback -> InputStream bridge, no extra framing. - InProcessSpikeMain.java: full end-to-end flow against the real runtime.node, verified PASS on Windows 11 / JDK 25.0.2. Resolves 3.11 in the ignorance-reduction plan with five definitive answers: 1. Replay proxy works with InProcess (COPILOT_API_URL via InProcessEnvGuard). 2. Real runtime.node in CI (DRI decision confirmed). 3. No mocking needed (DRI decision confirmed). 4. Existing YAML snapshots reused for both transports. 5. Full E2E suite runs twice: subprocess + InProcess (-Pinprocess profile, forkCount=1, serial execution). Bonus finding: @github/copilot-win32-x64@1.0.69-0 (currently in nodejs/node_modules) is missing copilot_runtime_host_start; 1.0.73 (pinned in package-lock.json) has both the old and new server_* API. The copilot-native Maven module downloads 1.0.73 via npm pack.
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (16)
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:23
- This states the opposite of the current Python implementation. Python honors
COPILOT_NPM_REGISTRY_URL(python/copilot/_cli_version.py:145-162) and fails closed unless the downloaded runtime passes npm Subresource Integrity verification (python/copilot/_cli_download.py:418-432). As written, this security note would give its recipient materially incorrect information.
1917-java-embed-rust-cli-runtime-remove-before-merge/20260722-prompts-3_1-and-3_2-gaps.md:41 - This resolution contradicts both the evidence immediately above it and the repository: Java is
1.0.9-SNAPSHOT, while@github/copilotand its platform packages are1.0.73. Keeping this conclusion would make the native packaging design download a nonexistent runtime version; retain the independent-version mapping described in lines 29-35.
The npm package version and the SDK version are the **same version**. `@github/copilot-linux-x64@1.0.9-preview.0` and `com.github:copilot-sdk-java:1.0.9-preview.0` — same string.
So there's no version mapping at all. The `copilot-native` module downloads `@github/copilot-<platform>@${project.version}` from npm. One version, everywhere. No extra property needed.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/InProcessEnvGuard.java:216
- Both Unix failure paths continue after
setenv/unsetenvfails. In particular, a failed proxy-URL override can silently allow authenticated traffic to reach the real endpoint. Treat either nonzero result as fatal and roll back all environment changes already made by this guard.
if (rc != 0) {
LOG.warning("[InProcessEnvGuard] setenv() failed for key=" + name + " rc=" + rc);
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-linux-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:102
UNKNOWNcurrently falls through to the glibc classifier. If/proc/self/exeis unavailable or the ELF probe cannot identify its interpreter, this deterministically selects the wrong native binary on musl instead of reporting that detection failed. Only map an explicitGLIBCresult tolinux-*; rejectUNKNOWN.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-darwin-arm64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:99- This treats every Linux result other than explicit
MUSL—includingUNKNOWNafter an I/O/parsing failure—as glibc. That can select and load an incompatible binary on musl. Require an explicitGLIBCresult and fail when libc cannot be determined.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-win32-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:108 - The
elsepath mapsLinuxLibc.UNKNOWNtolinux-*, so an unreadable or unrecognized interpreter silently becomes glibc. This violates the deterministic selection requirement and can load an incompatible binary on musl; fail unless detection explicitly returnsGLIBCorMUSL.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:28 - These Rust exports return a one-byte
bool, but JNA's default Javabooleanmapping is a 32-bit C integer. The declarations can misread false results; bind the ABI return asbyteand convert it in Java wrappers. The same mismatch affects all three boolean-returning methods in this interface.
boolean host_shutdown(int serverHandle);
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:30
- The fixture exports Rust
bool(one byte), while JNA maps Javabooleanas a 32-bit native integer by default. This makes all three boolean-returning declarations ABI-incompatible and can hide native failures; use one-byte return values and Java wrappers.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:28 - These methods bind one-byte Rust
boolreturns as JNAboolean, whose default native representation is a 32-bit integer. False returns may be misread, undermining the spike's result checks. Bind asbyteand convert in wrapper code.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:30 - JNA's Java
booleanuses a 32-bit native integer, but the Rust fixture exports one-byteboolvalues. All three declarations therefore have the wrong ABI and can report failures as successes; expose byte returns and convert them in the provider methods.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:28 - There is no JDK 25 FFM path in this project:
NativeBindingProvideris only present under the baseline source tree and explicitly uses JNA on every JDK; the sole Java 25 overlay isReaderThreadFactory. The run instructions, expected logs, and result table therefore describe behavior this JAR cannot produce. Either add the missing FFM overlay or rewrite all FFM claims to describe JNA plus a virtual reader thread.
## Run on JDK 25 (FFM path)
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:28
- This project has no FFM binding or Java 25
NativeBindingProvideroverlay; JDK 25 only swaps in a virtual-threadReaderThreadFactory, while callbacks still use JNA. Consequently these instructions and the result table promise an FFM path and log output that cannot occur. Implement the missing FFM path or update every FFM reference to the actual JNA behavior.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/rust-dll/README.md:21 - The fixture now exports six functions because
callback_oncewas added atsrc/lib.rs:32, but this count and table still list only five. Include the synchronous control export so the fixture documentation matches the Graal experiment.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackSpikeMain.java:174 - A callback timeout is only logged, so this verification program proceeds through cleanup and exits successfully even though its core callback/queue experiment failed. Make the timeout fail the run, as the Graal spike already does.
LOG.warning("Timed out waiting for callbacks! activeCallbacks=" + activeCallbacks.get());
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/CallbackMrJarMain.java:133
- The timeout path still returns success, allowing this spike to claim completion when native callbacks never arrived. Throw on timeout so the result can be used as a reliable verification.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/CallbackMrJarMain.java:133 - This timeout only emits a warning and the process exits 0, so the MR-JAR experiment can be reported as successful without receiving its callbacks. Fail the run on timeout.
- Files reviewed: 70/76 changed files
- Comments generated: 3
- Review effort level: Medium
| * <p><b>bool return:</b> JNA maps Java {@code boolean} to C {@code _Bool}/ | ||
| * {@code bool} (1 byte). The Rust {@code #[repr(C)]} bool ABI is stable. |
| if (!ok) { | ||
| LOG.warning("[InProcessEnvGuard] SetEnvironmentVariableW failed for key=" + name); | ||
| } |
| LOG.info("[shutdown] Calling copilot_runtime_connection_close..."); | ||
| boolean closed = lib.copilot_runtime_connection_close(connectionId); | ||
| LOG.info("[shutdown] connection_close returned " + closed); | ||
|
|
||
| LOG.info("[shutdown] Calling copilot_runtime_host_shutdown..."); | ||
| boolean shutdown = lib.copilot_runtime_host_shutdown(serverId); | ||
| LOG.info("[shutdown] host_shutdown returned " + shutdown); |
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (20)
1917-java-embed-rust-cli-runtime-remove-before-merge/20260722-prompts-3_1-and-3_2-gaps.md:41
- This resolution contradicts the repository's actual versioning: the Java SDK is
1.0.9-SNAPSHOT(java/pom.xml:8-10) while@github/copilotis1.0.73(nodejs/package-lock.json:702-703). Deriving the npm package version from${project.version}would request a different or nonexistent runtime package; retain an explicit runtime-version pin separate from the Maven artifact version.
The npm package version and the SDK version are the **same version**. `@github/copilot-linux-x64@1.0.9-preview.0` and `com.github:copilot-sdk-java:1.0.9-preview.0` — same string.
So there's no version mapping at all. The `copilot-native` module downloads `@github/copilot-<platform>@${project.version}` from npm. One version, everywhere. No extra property needed.
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:23
- The Python findings are stale: both its packument and tarball URLs honor
COPILOT_NPM_REGISTRY_URL(python/copilot/_cli_version.py:145-162), and runtime provisioning fails closed when npm SRI is unavailable or mismatched (python/copilot/_cli_download.py:421-432). As written, this note would send an incorrect supply-chain assessment.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/CopilotRuntimeLibrary.java:28 - JNA's default Java
booleanmapping is a 32-bit Cint, not the one-byte C/Rustboolclaimed here. Binding Rustextern "C" fn -> boolthis way is an ABI mismatch and can misread return values on supported platforms. Use byte-sized return declarations with explicit!= 0conversion (or a custom type mapper) for all three boolean exports.
* <p><b>bool return:</b> JNA maps Java {@code boolean} to C {@code _Bool}/
* {@code bool} (1 byte). The Rust {@code #[repr(C)]} bool ABI is stable.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-win32-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:108
UNKNOWNcurrently falls through to the glibc classifier. If/procis unavailable, ELF parsing fails, or the interpreter is unrecognized, this selects a potentially incompatible glibc library rather than detecting the platform. Fail closed unless libc is explicitlyGLIBCorMUSL.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-linux-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:100UNKNOWNis treated as glibc here. Any failure to read or parse/proc/self/exetherefore selectslinux-*even though libc was not determined, which can load an incompatible native binary. Return a classifier only for explicitGLIBCorMUSL; otherwise fail closed.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-darwin-arm64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:97- This copy also defaults an
UNKNOWNlibc result to glibc. That converts a detection failure into selection of a potentially incompatible library. Require an explicitGLIBCresult before returninglinux-*, and throw forUNKNOWN.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/InProcessEnvGuard.java:132 nativeSetEnvonly logs OS failures, so the constructor still reports the guard as active and the runtime can start without replay-proxy redirection or credential suppression; a failedCOPILOT_API_URLupdate may send E2E traffic to the real service. Make mutations fail closed and roll back any entries already changed if construction fails.
nativeSetEnv(name, value);
LOG.info("[Env] " + name + "=" + (name.contains("TOKEN") ? "<redacted>" : value)
+ " (saved previous: " + (previous == null ? "null" : "<present>") + ")");
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:30
- These Java
booleandeclarations use JNA's 32-bit Cintmapping, but the Rust fixture exports one-byte Rust/C booleans. That is an ABI mismatch for every boolean method in this interface and can make the spike report incorrect results. Bind byte-sized returns and convert explicitly.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:28 - The JNA interface maps Java
booleanas a 32-bit Cint, while the Rust DLL exports Rust/Cbool. This ABI mismatch affects all boolean exports in the interface; use byte-sized return values and explicit conversion.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:30 - These Java
booleandeclarations are ABI-incompatible with the Rust fixture's one-byteboolreturns because JNA maps Javabooleanto a 32-bit Cint. Change every boolean export to a byte-sized binding and convert in the wrappers.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:28 - JNA maps Java
booleanto a 32-bit Cint, not the Rust fixture's one-bytebool. All boolean methods in this interface therefore use the wrong ABI and can return unreliable values. Use byte-sized declarations and explicit conversion.
boolean host_shutdown(int serverHandle);
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:56
- The table claims a JDK 25 FFM overlay, but the only overlay is
ReaderThreadFactory;NativeBindingProviderremains the baseline JNA implementation on every JDK. Record JNA plus virtual-reader behavior here.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:56 - The JDK 25 row documents an FFM
NativeBindingProvideroverlay that does not exist. Both JDKs use JNA; the MR-JAR changes only the reader thread from platform to virtual.
| JVM | Class loaded | Callback mechanism | Thread behavior |
| ------ | ---------------------------------- | ------------------ | ------------------------------ |
| JDK 17 | `NativeBindingProvider` (baseline) | JNA `Callback` | New Java thread per invocation |
| JDK 25 | `NativeBindingProvider` (overlay) | FFM upcall stub | Executes on native thread |
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/rust-dll/README.md:25
- The fixture now exports six functions because
callback_oncewas added, but the README still says five and omits that control path from the table.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:32 - The JDK 25 implementation still uses
NativeBindingProvider's JNA callback; onlyReaderThreadFactoryis overlaid to create a virtual reader thread. This run section describes an FFM implementation that is not present and even adds an unnecessary FFM native-access flag.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:32 - This copy also has no FFM binding: JDK 25 overlays only the reader-thread factory while callbacks remain JNA-based. The heading, command flag, and expected logs describe code that is not in the JAR.
## Run on JDK 25 (FFM path)
```sh
java --enable-native-access=ALL-UNNAMED ^
-Djava.library.path=../rust-dll/target/debug ^
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/CallbackMrJarMain.java:134
- A callback timeout is only logged, after which the spike proceeds to close the connection and exits successfully. This turns the callback behavior being tested into a false-positive pass; fail the run on timeout.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/CallbackMrJarMain.java:134 - This timeout path still allows the spike to continue cleanup and exit 0, so a failed callback experiment is reported only as a warning. Throw on timeout to make the result reliable.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackSpikeMain.java:175 - The core JNA callback spike exits successfully after this timeout and prints a completion summary, producing a false positive when native callbacks fail. Treat timeout as a failed experiment.
} else {
LOG.warning("Timed out waiting for callbacks! activeCallbacks=" + activeCallbacks.get());
}
1917-java-embed-rust-cli-runtime-remove-before-merge/20260722-prompts-3_1-and-3_2-gaps.md:15
- This resolution repeats the invalid version coupling:
${project.version}is the Java SDK version, not the independently pinned npm runtime version (java/pom.xml:8-10is1.0.9-SNAPSHOT, whilenodejs/package-lock.json:702-703is1.0.73). Thenpm packcommand needs a dedicated runtime-version value.
**Resolution:** Skip `package.json` entirely for the native module. Use `exec-maven-plugin` to run `npm pack @github/copilot-<platform>@<version>` for each platform, then extract with `tar`. Version comes from `${project.version}` (SDK and npm versions are identical). Integrity is verified by a build step that reads the `integrity` field from the monorepo's `nodejs/package-lock.json` for each platform package and checks the SHA-512 of the downloaded `.tgz`, mirroring Rust's `resolve_version_and_integrity` → `cached_download` → verify pattern in `build/in_process.rs`.
- Files reviewed: 70/76 changed files
- Comments generated: 0 new
- Review effort level: Medium
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (31)
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/CopilotRuntimeLibrary.java:28
- JNA's default
booleanmapping is a 32-bit Cint, but the runtime ABI returns Rustbool/C_Bool(one byte). This can misread false/error returns. Bind all three boolean-returning entry points asbyte(or use a 1-byteTypeMapper) and convert withresult != 0in Java.
* <p><b>bool return:</b> JNA maps Java {@code boolean} to C {@code _Bool}/
* {@code bool} (1 byte). The Rust {@code #[repr(C)]} bool ABI is stable.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:30
- This interface maps Rust
boolreturns to JNAboolean, which JNA marshals as a 32-bit Cint; Rust's C ABI uses a one-byte_Bool. The mismatch makes false/error returns unreliable. Use a one-byte return mapping for every boolean-returning function and convert it in the wrappers.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:30 - This interface maps Rust
boolreturns to JNAboolean, which JNA marshals as a 32-bit Cint; Rust's C ABI uses a one-byte_Bool. The mismatch makes false/error returns unreliable. Use a one-byte return mapping for every boolean-returning function and convert it in the wrappers.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:28 - The Rust fixture returns a one-byte C-ABI
bool, whereas JNA marshals Javabooleanas a 32-bit Cint. This return signature is therefore ABI-incompatible and can turn a false result into an arbitrary value. Apply a one-byte mapping to all boolean returns in this interface.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:28 - The Rust fixture returns a one-byte C-ABI
bool, whereas JNA marshals Javabooleanas a 32-bit Cint. This return signature is therefore ABI-incompatible and can turn a false result into an arbitrary value. Apply a one-byte mapping to all boolean returns in this interface.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-win32-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:108 UNKNOWNis treated as glibc here. On a musl host where/proc/self/execannot be read or parsed, that deterministically selects the wrong native library. Only producelinux-*after detecting GLIBC; otherwise report that libc detection failed.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-linux-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:102UNKNOWNcurrently falls through to the glibc classifier. If/procis unavailable or ELF parsing fails on a musl system, Java will select and load the incompatible glibc library. Fail closed unless libc was positively identified as GLIBC or MUSL.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-darwin-arm64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:99UNKNOWNcurrently falls through to the glibc classifier. If/procis unavailable or ELF parsing fails on a musl system, Java will select and load the incompatible glibc library. Fail closed unless libc was positively identified as GLIBC or MUSL.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/InProcessEnvGuard.java:208- A failed
SetEnvironmentVariableWonly emits a warning, so execution can continue with the original environment. In particular, failure to set the replay-proxy URL can send requests and credentials to the real service. Fail closed and roll back any variables already changed by this constructor.
if (!ok) {
LOG.warning("[InProcessEnvGuard] SetEnvironmentVariableW failed for key=" + name);
}
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/InProcessEnvGuard.java:217
- Ignoring a
setenvfailure lets the runtime continue without the requested proxy/auth isolation and may direct a replay test to the live service. Treat this as construction failure and roll back earlier environment changes.
if (rc != 0) {
LOG.warning("[InProcessEnvGuard] setenv() failed for key=" + name + " rc=" + rc);
}
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/InProcessEnvGuard.java:222
- Ignoring an
unsetenvfailure can leave HMAC credentials active even though this guard promises to suppress them for replay-proxy traffic. Fail construction/restoration and roll back consistently when deletion fails.
if (rc != 0) {
LOG.warning("[InProcessEnvGuard] unsetenv() failed for key=" + name + " rc=" + rc);
}
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:23
- These mitigations are outdated: Python now honors
COPILOT_NPM_REGISTRY_URL(python/copilot/_cli_version.py:145-162) and fails closed unless npm Subresource Integrity verifies (python/copilot/_cli_download.py:421-432). As written, this note gives the recipient an incorrect supply-chain assessment.
1917-java-embed-rust-cli-runtime-remove-before-merge/20260722-prompts-3_1-and-3_2-gaps.md:41 - This resolution contradicts the repository's actual versioning:
java/pom.xmlis version1.0.9-SNAPSHOT, while the runtime packages innodejs/package-lock.jsonare1.0.73. Using${project.version}as the npm runtime version therefore requests a package that need not exist. Keep a separate pinned runtime version and update the earlier resolution that also relies on${project.version}.
The npm package version and the SDK version are the **same version**. `@github/copilot-linux-x64@1.0.9-preview.0` and `com.github:copilot-sdk-java:1.0.9-preview.0` — same string.
So there's no version mapping at all. The `copilot-native` module downloads `@github/copilot-<platform>@${project.version}` from npm. One version, everywhere. No extra property needed.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:4
- This README describes an FFM implementation on JDK 25, but the only JDK 25 overlay is
ReaderThreadFactory;NativeBindingProviderexplicitly uses JNA on every JDK and says FFM is deferred. The title, run instructions, expected logs, and behavior table should describe JNA with platform versus virtual reader threads.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:4 - This README describes an FFM implementation on JDK 25, but the only JDK 25 overlay is
ReaderThreadFactory;NativeBindingProviderexplicitly uses JNA on every JDK and says FFM is deferred. The title, run instructions, expected logs, and behavior table should describe JNA with platform versus virtual reader threads.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/pom.xml:16 - The artifact metadata claims that JDK 25 supplies an FFM
NativeBindingProvider, but this project only overlaysReaderThreadFactoryand uses the baseline JNA binding on all JDKs. Correct the name and description so published/generated metadata does not report an implementation that is absent.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/pom.xml:16 - The artifact metadata claims that JDK 25 supplies an FFM
NativeBindingProvider, but this project only overlaysReaderThreadFactoryand uses the baseline JNA binding on all JDKs. Correct the name and description so published/generated metadata does not report an implementation that is absent.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/rust-dll/README.md:21 - This fixture exports six functions because
callback_oncewas added, but the README still says five and omits it from the table. This makes the documented ABI disagree with bothlib.rsand the Java binding.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/CallbackMrJarMain.java:134 - A callback timeout is only logged, after which the spike exits successfully and even prints a completion summary. That allows the experiment to be treated as passing when native callbacks never arrived. Throw or exit nonzero on timeout.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/CallbackMrJarMain.java:134 - A callback timeout is only logged, after which the spike exits successfully and even prints a completion summary. That allows the experiment to be treated as passing when native callbacks never arrived. Throw or exit nonzero on timeout.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/README.md:25 - The build command changes into a sibling directory that does not exist; this project is named
java-program-that-invokes-rust-dll-jdk17. Following the README fromrust-dlltherefore fails before Maven runs.
cd ../java-program-that-invokes-rust-dll
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackSpikeMain.java:175
- A callback timeout only produces a warning, so this verification program still exits zero and reports the spike complete despite missing callbacks. Make timeout a failing condition as the Graal spike already does.
if (completed) {
LOG.info("All callbacks completed. activeCallbacks=" + activeCallbacks.get());
} else {
LOG.warning("Timed out waiting for callbacks! activeCallbacks=" + activeCallbacks.get());
}
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/QueueInputStream.java:69
- The EOF/zero-length returns happen before argument validation, so this does not honor the
InputStreamcontract for null buffers or invalid ranges. Validate first, then handle zero-length and EOF.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/QueueInputStream.java:69 - The EOF/zero-length returns happen before argument validation, so this does not honor the
InputStreamcontract for null buffers or invalid ranges. Validate first, then handle zero-length and EOF.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/QueueInputStream.java:82 - The EOF/zero-length returns happen before argument validation, so this does not honor the
InputStreamcontract for null buffers or invalid ranges. Validate first, then handle zero-length and EOF.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/QueueInputStream.java:82 - The EOF/zero-length returns happen before argument validation, so this does not honor the
InputStreamcontract for null buffers or invalid ranges. Validate first, then handle zero-length and EOF.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:29 - The Rust fixture declares this length as
usize, which is 64-bit on every target in #1917, but this JNA signature marshals a 32-bitint. The callback length has the same mismatch. Uselongconsistently for bothsize_tparameters so the spike exercises the actual ABI.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:29 - The Rust fixture declares this length as
usize, which is 64-bit on every target in #1917, but this JNA signature marshals a 32-bitint. The callback length has the same mismatch. Uselongconsistently for bothsize_tparameters so the spike exercises the actual ABI.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:52 - Rust's
connection_writetakesusize len, but Javaintis only 32 bits on the x64/arm64 targets under test;OutboundCallback.invokehas the same mismatch. Map bothsize_tvalues as Javalongand update their callers.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:60 - Rust's
connection_writetakesusize len, but Javaintis only 32 bits on the x64/arm64 targets under test;OutboundCallback.invokehas the same mismatch. Map bothsize_tvalues as Javalongand update their callers.
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:11 - “At build time” is not true for every SDK: Python's in-process runtime is downloaded on demand by
_cli_download.py, whereas Rust/Go/.NET perform build or bundling downloads. Describe these as direct registry downloads without assigning a single lifecycle phase.
- Files reviewed: 75/81 changed files
- Comments generated: 2
- Review effort level: Medium
| $env:AZURE_STORAGE_KEY = (Invoke-Az -SensitiveOutput -Arguments @( | ||
| "storage", "account", "keys", "list", | ||
| "--resource-group", $ResourceGroup, | ||
| "--account-name", $StorageAccount, | ||
| "--query", "[0].value", |
| public int read(byte[] b, int off, int len) throws IOException { | ||
| if (eof) return -1; | ||
| if (len == 0) return 0; |
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (14)
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:23
- These mitigation claims are stale: Python now honors
COPILOT_NPM_REGISTRY_URL(python/copilot/_cli_version.py:145-162) and refuses to load a runtime unless npm Subresource Integrity is available and verifies (python/copilot/_cli_download.py:421-432). As written, this note incorrectly tells its recipient that only .NET is configurable and that Python has no integrity verification.
1917-java-embed-rust-cli-runtime-remove-before-merge/20260722-prompts-3_1-and-3_2-gaps.md:41 - This resolution is contradicted by the repository: the Java SDK is
1.0.9-SNAPSHOT(java/pom.xml:8-10) while the runtime packages are1.0.73(nodejs/package-lock.json:702-721). Using${project.version}for npm downloads will therefore request runtime versions that do not exist; retain a separate pinned runtime version.
The npm package version and the SDK version are the **same version**. `@github/copilot-linux-x64@1.0.9-preview.0` and `com.github:copilot-sdk-java:1.0.9-preview.0` — same string.
So there's no version mapping at all. The `copilot-native` module downloads `@github/copilot-<platform>@${project.version}` from npm. One version, everywhere. No extra property needed.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/rust-dll/README.md:21
- This fixture exports six functions, not five:
callback_onceis also exported byrust-dll/src/lib.rs:30-40and is used as the synchronous control in the GraalVM test. Omitting it makes the fixture documentation incomplete.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:28 - The implementation does not have an FFM path:
NativeBindingProviderexplicitly uses JNA on every JDK and says FFM is deferred, while the JDK 25 overlay only replacesReaderThreadFactory. The heading, expected logs, and behavior table therefore describe code that this spike cannot execute; document JNA + virtual threads for JDK 25 instead.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:28 - The implementation does not have an FFM path:
NativeBindingProviderexplicitly uses JNA on every JDK and says FFM is deferred, while the JDK 25 overlay only replacesReaderThreadFactory. The heading, expected logs, and behavior table therefore describe code that this spike cannot execute; document JNA + virtual threads for JDK 25 instead.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/CopilotRuntimeLibrary.java:28 - JNA maps Java
booleanto a 4-byte Cint, not to the runtime's 1-byte Rust/Cbool; the .NET binding explicitly usesUnmanagedType.U1for the same ABI (dotnet/src/FfiRuntimeHost.cs:364-387). Consequently the three boolean-returning declarations below can read undefined upper return-register bytes and misreport native failures as success. Bind these returns asbyte(or add an explicit 1-byte type mapper) and convert to boolean in Java.
* <p><b>bool return:</b> JNA maps Java {@code boolean} to C {@code _Bool}/
* {@code bool} (1 byte). The Rust {@code #[repr(C)]} bool ABI is stable.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/QueueInputStream.java:68
- This override violates
InputStream.read(byte[], off, len): a zero-length read must return0, including after EOF, and invalid buffer ranges must be rejected before the EOF result. Validate the range first, then handlelen == 0beforeeof.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/QueueInputStream.java:81 - This override violates
InputStream.read(byte[], off, len): a zero-length read must return0, including after EOF, and invalid buffer ranges must be rejected before the EOF result. Validate the range first, then handlelen == 0beforeeof.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/QueueInputStream.java:68 - This override violates
InputStream.read(byte[], off, len): a zero-length read must return0, including after EOF, and invalid buffer ranges must be rejected before the EOF result. Validate the range first, then handlelen == 0beforeeof.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/QueueInputStream.java:81 - This override violates
InputStream.read(byte[], off, len): a zero-length read must return0, including after EOF, and invalid buffer ranges must be rejected before the EOF result. Validate the range first, then handlelen == 0beforeeof.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/QueueInputStream.java:103 - This override violates
InputStream.read(byte[], off, len): a zero-length read must return0, including after EOF, and invalid buffer ranges must be rejected before the EOF result. Validate the range first, then handlelen == 0beforeeof.
public int read(byte[] b, int off, int len) throws IOException {
if (eof) return -1;
if (len == 0) return 0;
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-linux-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:102
UNKNOWNcurrently falls through to the glibc classifier. If/procis unavailable, the 2 KiB probe is insufficient, or the interpreter path is unrecognized on a musl JVM, this deterministically selects the incompatible glibc library rather than reporting that detection failed. Only map explicitGLIBC/MUSLresults and rejectUNKNOWN.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-darwin-arm64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:99UNKNOWNcurrently falls through to the glibc classifier. If/procis unavailable, the 2 KiB probe is insufficient, or the interpreter path is unrecognized on a musl JVM, this deterministically selects the incompatible glibc library rather than reporting that detection failed. Only map explicitGLIBC/MUSLresults and rejectUNKNOWN.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-win32-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:108UNKNOWNcurrently falls through to the glibc classifier. If/procis unavailable, the 2 KiB probe is insufficient, or the interpreter path is unrecognized on a musl JVM, this selects the incompatible glibc library rather than reporting that detection failed. Only map explicitGLIBC/MUSLresults and rejectUNKNOWN.
- Files reviewed: 75/81 changed files
- Comments generated: 2
- Review effort level: Medium
| <maven.compiler.source>17</maven.compiler.source> | ||
| <maven.compiler.target>17</maven.compiler.target> |
| for (String key : List.of("COPILOT_HMAC_KEY", "CAPI_HMAC_KEY")) { | ||
| String previous = System.getenv(key); | ||
| if (previous != null && !previous.isEmpty()) { | ||
| saved.add(Map.entry(key, previous)); | ||
| nativeSetEnv(key, null); // delete |
Fills in the four 3.12 sub-question answers derived from earlier ignorance-reduction sessions: 1. java-sdk-tests.yml gets a new java-sdk-inprocess job (separate job, not matrix entry; existing job unchanged); mirrors Rust test-inprocess. 2. Native binaries via npm pack in copilot-native generate-resources phase (3.2 Resolution); CI job builds that module first. 3. ubuntu-latest + macos-latest only; Windows excluded matching Rust and .NET precedents (napi-oop peer-shutdown crash). 4. No runtime availability check; gated by -Pinprocess Maven profile; IllegalStateException on missing binary (3.5 Resolution). Also adds a DRI decision note at the top of Phase 4: linux-x64 is the first and exclusive implementation target until the full stack is green on ubuntu-latest, matching the Copilot Coding Agent runner and local CLI.
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (34)
1917-java-embed-rust-cli-runtime-remove-before-merge/20260722-prompts-3_1-and-3_2-gaps.md:41
- This resolution reverses the correct observation above it. The Java SDK is currently
1.0.9-SNAPSHOT(java/pom.xml:8-10), while the runtime packages are1.0.73(nodejs/package-lock.json:702-721), so using${project.version}would request a nonexistent or unrelated npm runtime release. Keep a separate runtime-version pin and package that runtime under the SDK artifact version.
The npm package version and the SDK version are the **same version**. `@github/copilot-linux-x64@1.0.9-preview.0` and `com.github:copilot-sdk-java:1.0.9-preview.0` — same string.
So there's no version mapping at all. The `copilot-native` module downloads `@github/copilot-<platform>@${project.version}` from npm. One version, everywhere. No extra property needed.
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:16
- Python's registry is no longer hardcoded: both its packument and tarball URL builders honor
COPILOT_NPM_REGISTRY_URL(python/copilot/_cli_version.py:145-162). This entry should describe the default plus that override rather than reporting the old behavior.
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:23 - These mitigation conclusions are outdated. Python now supports a registry override and refuses to load the runtime unless npm Subresource Integrity is available and verifies (
python/copilot/_cli_download.py:325-360, 421-432); leaving this claim in the security note materially misstates the current exposure.
1917-java-embed-rust-cli-runtime-remove-before-merge/supply-chain-security-to-stephentoub.md:7 - Use the possessive contraction here.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/CopilotRuntimeLibrary.java:28 - The bool mapping documented here is incorrect: JNA's default Java
booleanmapping is a 32-bit Cint, while these Rust/C ABI functions return a 1-bytebool(the .NET binding explicitly usesUnmanagedType.U1atdotnet/src/FfiRuntimeHost.cs:364-387). The three JNA return declarations therefore do not match the ABI; bind them asbyte(or use a 1-byte type mapper) and test nonzero at call sites.
* <p><b>bool return:</b> JNA maps Java {@code boolean} to C {@code _Bool}/
* {@code bool} (1 byte). The Rust {@code #[repr(C)]} bool ABI is stable.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-win32-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:108
UNKNOWNcurrently falls through to the glibc classifier. If/procis unavailable or ELF parsing fails on a musl host, this deterministically selects the wrong native binary instead of reporting that detection failed. Only selectlinux-*for an explicitGLIBC; fail fast forUNKNOWN.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/InProcessEnvGuard.java:208- Native environment mutation failures are only logged, so the test continues after failing to set the token or replay-proxy URL. That can turn an intended replay run into an unauthenticated failure or a request to the live API. Fail the guard construction and restore any variables already changed when
SetEnvironmentVariableWfails.
if (!ok) {
LOG.warning("[InProcessEnvGuard] SetEnvironmentVariableW failed for key=" + name);
}
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:32
- This section describes an FFM implementation that is not present. The JDK 25 overlay only replaces
ReaderThreadFactorywith a virtual-thread implementation;NativeBindingProviderremains the baseline JNA class on both JDKs. Update the JDK 25 run instructions and comparison table to say “JNA + virtual thread” and remove--enable-native-access.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/pom.xml:16 - The project metadata says JDK 25 swaps in FFM and a different
NativeBindingProvider, but the only versioned class isReaderThreadFactory; native binding remains JNA. This metadata will misdescribe the produced artifact.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/README.md:32 - This copied README still describes a JDK 25 FFM path, but this spike explicitly defers FFM and only overlays
ReaderThreadFactory. Update the JDK 25 section and table to document JNA with a virtual reader thread, and remove the unnecessary native-access flag.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/pom.xml:16 - The artifact metadata claims the JDK 25 variant uses FFM and replaces
NativeBindingProvider, but no such overlay exists; both variants use JNA and differ only in reader-thread type.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/rust-dll/README.md:21 - This fixture now exports six functions because
callback_oncewas added, but the README still says five and omits it from the table. Document the synchronous callback control so the fixture contract matchessrc/lib.rs.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/dependency-reduced-pom.xml:1 - This is generated Shade output, not source. The research log itself says
dependency-reduced-pom.xmlshould remain untracked, and committing it creates a stale second project definition that will diverge frompom.xml. Delete it and configure/extend.gitignoreso packaging cannot re-add it.
<?xml version="1.0" encoding="UTF-8"?>
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/dependency-reduced-pom.xml:1
- This generated Shade POM should not be committed. It duplicates the authoritative
pom.xml, is regenerated by builds, and the accompanying research notes explicitly intended to exclude it. Delete the file and ignoredependency-reduced-pom.xml.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-jdk17/dependency-reduced-pom.xml:1 - This is generated Maven Shade output and should remain untracked; otherwise it is a stale duplicate of the real project configuration. Delete it and add
dependency-reduced-pom.xmlto this spike's ignore rules.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:28 - This JNA interface maps Rust's 1-byte
boolreturns as Javaboolean, which JNA marshals as a 32-bit Cint. That is an ABI mismatch forhost_shutdown,connection_write, andconnection_close; usebytereturn values and treat nonzero as success.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackTestLibrary.java:28 - The fixture returns Rust/C ABI 1-byte bools, but JNA's default Java
booleanmapping is a 32-bit native integer. Correct all three bool-returning declarations tobyteand compare their result with zero at the callers.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:30 - These Java
booleanreturns do not match the Rust fixture's 1-byteboolABI because JNA maps Java boolean to a 32-bit C integer. Bind each bool result asbyte, then convert withresult != 0in the provider methods.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/NativeBindingProvider.java:30 - JNA marshals Java
booleanas a 32-bit C integer, not the fixture's 1-byte Rust/Cbool. The three bool-returning methods therefore have mismatched signatures; bind them asbyteand convert nonzero results in the wrappers.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackSpikeMain.java:175 - A callback timeout is the failure condition this spike is meant to detect, but this branch only logs a warning and then exits successfully with a “complete” summary. Throw here (after cleanup via
finally) so automation cannot record a failed callback experiment as passing.
if (completed) {
LOG.info("All callbacks completed. activeCallbacks=" + activeCallbacks.get());
} else {
LOG.warning("Timed out waiting for callbacks! activeCallbacks=" + activeCallbacks.get());
}
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/CallbackMrJarMain.java:134
- The experiment still exits zero after callback timeout, so it can report completion even when the behavior under test failed. Make timeout fatal while preserving native cleanup in a
finallyblock.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/CallbackMrJarMain.java:134 - This copied spike treats callback timeout as a warning and continues to a successful process exit. Since callback delivery is the assertion under test, timeout must produce a nonzero failure after running cleanup.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/java-inprocess-e2e-win32-x64/src/main/java/com/github/copilot/spike/inprocess/QueueInputStream.java:103 InputStream.readmust validateb/off/len, and a zero-length request returns0even at EOF. This implementation does neither because it checks EOF first and returns before touching invalid zero-length arguments.
public int read(byte[] b, int off, int len) throws IOException {
if (eof) return -1;
if (len == 0) return 0;
1917-java-embed-rust-cli-runtime-remove-before-merge/dd-3038503-prepare-devbox/02-create-storage-upload-blob.ps1:100
- This leaves the storage account key in the invoking PowerShell process after uploads complete or fail, allowing unrelated later commands to inherit it. Scope the key usage with
try/finallyand removeEnv:AZURE_STORAGE_KEYin thefinallyblock.
$env:AZURE_STORAGE_ACCOUNT = $StorageAccount
$env:AZURE_STORAGE_KEY = (Invoke-Az -SensitiveOutput -Arguments @(
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-linux-x64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:102
UNKNOWNcurrently falls through to the glibc classifier. A read/parse failure on a musl system would therefore loadlinux-*, exactly the wrong cdylib. Return the glibc classifier only for explicitGLIBC, and fail fast forUNKNOWN.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-6-platform-detection-darwin-arm64/src/main/java/com/github/copilot/spike/platform/PlatformDetector.java:99UNKNOWNcurrently falls through to the glibc classifier. On a musl system where/proc/self/execannot be read, this silently chooses an incompatible runtime. Require explicitGLIBCbefore returninglinux-*, and fail fast forUNKNOWN.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/QueueInputStream.java:82- This violates
InputStream.read(byte[], off, len): a zero-length read must return0even after EOF, and invalid/null buffer arguments must be rejected before returning. Validate the range, then handlelen == 0, then test EOF.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/QueueInputStream.java:82 - The bulk-read contract requires argument validation and requires
len == 0to return0, including at EOF. The current order returns-1at EOF and accepts invalid zero-length ranges.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/QueueInputStream.java:69 - This bulk-read implementation checks EOF before the mandatory zero-length case and never validates
b/off/len. That breaks theInputStreamcontract for zero-length reads at EOF and invalid ranges.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-8-graal-research/java-program-that-invokes-rust-dll-mr-jar-17-25/src/main/java/com/github/copilot/spike/mrjar/QueueInputStream.java:69 - Honor the
InputStreambulk-read contract by validating the buffer/range and returning0forlen == 0before checking EOF. As written, EOF returns-1for an empty read and invalid empty ranges are silently accepted.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/src/main/java/com/github/copilot/spike/CallbackSpikeMain.java:17 - The class-level description says this spike uses Piped streams, but the implementation uses
QueueInputStreamthroughout and specifically rejects Piped streams. Rename this description to avoid documenting the opposite of what is tested.
* Spike 3.4 — JNA callback threading with PipedInputStream/PipedOutputStream bridging.
1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-4-jna-callback-and-threading/java-program-that-invokes-rust-dll-jdk17/pom.xml:16
- The POM says the program pipes through
PipedInputStream/PipedOutputStream, but the code usesQueueInputStreambecause Piped streams are the rejected alternative. Correct the artifact metadata.
<description>
Minimal program to test JNA callback invocation from a native (Rust) thread,
piping data through PipedInputStream/PipedOutputStream.
</description>
1917-java-embed-rust-cli-runtime-remove-before-merge/djl_temp3.txt:1
- This opaque base64 payload is an unexplained temporary artifact and has no relationship to the Java runtime embedding work. Keeping it in the base branch adds unreviewable content with no build or documentation use; remove it or replace it with a named, readable research fixture and provenance.
cGx1Z2lucyB7CiAgICBhaS5kamwuamF2YVByb2plY3QKICAgIGFpLmRqbC5w
1917-java-embed-rust-cli-runtime-remove-before-merge/djl_temp4.txt:1
- This second opaque base64
djl_tempartifact is not referenced by the spike and cannot be meaningfully reviewed in this form. Remove it from the PR, or commit the decoded source under a descriptive name with an explanation if it is genuinely required.
cm9vdFByb2plY3QubmFtZSA9ICJkamwiCgpwbHVnaW5zIHsKICAgIGlkKCJv
- Files reviewed: 75/81 changed files
- Comments generated: 3
- Review effort level: Medium
| if (rc != 0) { | ||
| LOG.warning("[InProcessEnvGuard] setenv() failed for key=" + name + " rc=" + rc); | ||
| } | ||
| } else { | ||
| int rc = libc.unsetenv(name); | ||
| if (rc != 0) { | ||
| LOG.warning("[InProcessEnvGuard] unsetenv() failed for key=" + name + " rc=" + rc); | ||
| } |
| $env:AZURE_STORAGE_ACCOUNT = $StorageAccount | ||
| Write-DebugMessage "Requesting a storage account access key (the key will not be logged)." | ||
| $env:AZURE_STORAGE_KEY = az storage account keys list ` |
| @@ -0,0 +1,33 @@ | |||
| # Rust test DLL for spike 3.4 | |||
Base branch for work in #1917 .