[herin049]

Description

Relocated from main OpenTelemetry Python repo.

External profilers and observability tools (such as the OpenTelemetry eBPF Profiler) operate outside the instrumented process and have no access to resource attributes configured inside OpenTelemetry SDKs. OTEP-4719 (Process Context Sharing) introduces a standard mechanism for OpenTelemetry SDKs to publish resource attributes for access by out-of-process readers

This PR introduces the opentelemetry-process-context package as a Rust backed Python extension (built with maturin/pyo3) that implements the process outlined in OTEP-4719. It publishes the SDK's resource attributes to
a memory region that any external reader with access to /proc/<pid>/maps and /proc/<pid>/mem can discover.

Native Code

While a large portion of this logic in this package could be implemented without native code, practically it makes more sense to keep it as native code for the following reasons:

• Some of the syscalls used aren't exposed natively by Python (e.g. madvise), (although, ctypes could be used)
• Synchronization with atomics/memory barriers is required during publication which is best done natively
• By using the prost Rust crate, we eliminate the runtime dependency on protobuf
• Keeping the implementation native reduces runtime overhead (although updates are rare)
• Future proposals (e.g. thread local context, which will very likely be accepted as an OTEP) will required native code anyways

Python API

The introduced Python API has a very light surface area, consisting of two methods to publish and unpublish the process context.

def publish_context(resource: Resource, attributes: Attributes) -> None: ...
def unpublish_context() -> None: ...

Compatibility

The original OTEP explicitly mentions that this functionality is Linux only. However, I have tried to implement this package in such a way that limited testing/development can be performed natively on any Posix compatible system (e.g. MacOS). However, the full functionality of OTEP-4719 is only available on Linux and Windows is explicitly not supported.

Feature	Linux	macOS
Backing region	memfd_create("OTEL_CTX") -> mmap(MAP_PRIVATE)	Anonymous mmap
Reader discoverability	/proc/<pid>/maps shows memfd:OTEL_CTX	Not discoverable (no /proc)
Fork protection	madvise(MADV_DONTFORK)	Skipped
Mapping name	prctl(PR_SET_VMA_ANON_NAME, "OTEL_CTX")	Skipped
Publish timestamp clock	CLOCK_BOOTTIME	CLOCK_MONOTONIC
Production use	Fully supported	Development/testing only

Fixes opentelemetry-python#5291

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

uv run tox -e py314-test-process-context

Does This PR Require a Contrib Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[opentelemetry-pr-dashboard]

Pull request dashboard status

Waiting on reviewers · refreshed 2026-11-05 00:45 UTC

Review the latest changes.

Status above doesn't look right?

• Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
• Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

[herin049]

/dashboard route:reviewers

[opentelemetry-pr-dashboard]

@herin049, this pull request was routed to reviewers. The handoff remains active across pushes until newer actionable human feedback arrives. Top-level feedback through this request will not return; unresolved review threads remain open.

[samuelhnrq]

Nice work on this. I've built a prototype of the OTEP 4947 thread-context side for Python, including the otel_thread_ctx_v1 TLS slot. It passes an end-to-end test against open-telemetry/opentelemetry-ebpf-profiler#1869: samples carry the right trace and span IDs for both threads and asyncio tasks. But its on C and needs a little more cleanup more on that later...

A few things I learned along the way that may be useful here:

• The thread context needs one thing from the process context: a threadlocal.schema_version attribute in the published record (currently tlsdesc_v1_dev). Without it the profiler never looks for the TLS slot. A way to pass extra attributes to publish_context would be enough for a separate package to plug in.
• In my POC I wrote the TLS slot in C because the spec's requirements map directly onto it: __attribute__((visibility("default"))) __thread void *otel_thread_ctx_v1; plus -mtls-dialect=gnu2. Stable Rust can't select the TLS dialect, so in Rust it takes per-arch inline asm and a linker version script. Datadog has packaged that as libdd-otel-thread-ctx (Apache-2.0), so it's solvable either way.

You mention thread-local context as the next native piece... Do you see OTEP 4947 landing in this package or a separate one? I'd rather contribute than ship a parallel package, whichever shape works best.

[herin049]

Nice work on this. I've built a prototype of the OTEP 4947 thread-context side for Python, including the otel_thread_ctx_v1 TLS slot. It passes an end-to-end test against open-telemetry/opentelemetry-ebpf-profiler#1869: samples carry the right trace and span IDs for both threads and asyncio tasks. But its on C and needs a little more cleanup more on that later...

A few things I learned along the way that may be useful here:

• The thread context needs one thing from the process context: a threadlocal.schema_version attribute in the published record (currently tlsdesc_v1_dev). Without it the profiler never looks for the TLS slot. A way to pass extra attributes to publish_context would be enough for a separate package to plug in.
• In my POC I wrote the TLS slot in C because the spec's requirements map directly onto it: __attribute__((visibility("default"))) __thread void *otel_thread_ctx_v1; plus -mtls-dialect=gnu2. Stable Rust can't select the TLS dialect, so in Rust it takes per-arch inline asm and a linker version script. Datadog has packaged that as libdd-otel-thread-ctx (Apache-2.0), so it's solvable either way.

You mention thread-local context as the next native piece... Do you see OTEP 4947 landing in this package or a separate one? I'd rather contribute than ship a parallel package, whichever shape works best.

Yep, I have unpublished changes that will be submitted in a follow-up once this is merged for TLS. It will require C and will be integrated via build.rs.

[ivoanjo]

👋 Hey folks, I worked on the OTEP/spec, amazing to see this moving forward, thought I'd chime in with two small notes :D

[herin049]

/dashboard route:reviewers

[opentelemetry-pr-dashboard]

@herin049, this pull request was routed to reviewers. The handoff remains active across pushes until newer actionable human feedback arrives. Top-level feedback through this request will not return; unresolved review threads remain open.