Skip to content

Kubernetes Job cleanup leaks headless Services #370

Description

@ethanbates-ez

Background

The Kubernetes Job launcher creates a headless Service to provide stable DNS names for indexed Pods. Job cleanup deletes only the Job; the separate Service has no ownerReferences and survives indefinitely with its generated EndpointSlice. Repeated executions therefore accumulate Kubernetes API objects until cluster-wide informer caches can exhaust memory. This reproduces on Kubernetes 1.34 for Job-backed executions.

Acceptance Criteria

  • Headless Services are lifecycle-owned by their Kubernetes Jobs.
  • Cleanup removes Services created before ownership metadata existed.
  • Failed Service creation does not leave a runnable Job.

Additional Context

Reproduction

  1. Launch a Job-backed container execution.
  2. Allow the execution to complete and run launcher cleanup.
  3. Observe that the Job is deleted but its same-named headless Service remains.

Expected behavior: deleting or cleaning up the Job also removes the headless Service and generated EndpointSlice.

Proposed fix: #368

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions