Skip to content

Add LiteLLM activity sample - #343

Merged
brianstrauch merged 3 commits into
temporalio:mainfrom
Abhinav0905:litellm-workflow-sample
Aug 11, 2026
Merged

Add LiteLLM activity sample#343
brianstrauch merged 3 commits into
temporalio:mainfrom
Abhinav0905:litellm-workflow-sample

Conversation

@Abhinav0905

Copy link
Copy Markdown
Contributor

What changed

  • Added a standalone litellm_activity sample with serializable Workflow inputs.
  • Kept LiteLLM network I/O in a Temporal Activity so Workflow replay remains deterministic.
  • Added Activity timeouts, Temporal-controlled retries, provider configuration, and setup documentation.
  • Added deterministic Activity and Workflow tests with no live provider calls.
  • Added the LiteLLM dependency group, package metadata, root sample listing, and lockfile entries.

Why

LLM provider calls are nondeterministic network operations and must run outside Workflow code. This sample demonstrates the Activity boundary directly while making timeouts and retry behavior visible in Temporal Event History.

Closes #239.

User impact

Users can run the sample against any LiteLLM-supported provider by setting provider credentials on the Worker and selecting a model with LITELLM_MODEL. Credentials are not passed through the Workflow or stored in Event History.

Validation

  • Ruff import ordering and lint checks
  • Ruff formatting check
  • MyPy on the new sample and tests
  • pytest tests/litellm_activity (2 passed)

cc @brianstrauch

@CLAassistant

CLAassistant commented Aug 2, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Comment thread litellm_activity/activities.py Outdated
Comment thread litellm_activity/activities.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new litellm_activity sample that demonstrates the Temporal best practice of performing nondeterministic LLM/provider network calls inside an Activity (not Workflow code), along with dependency wiring and deterministic tests to validate behavior without live provider calls.

Changes:

  • Introduces the litellm_activity sample (Workflow + Activity + worker/starter) with Activity timeouts and Temporal-managed retries.
  • Adds deterministic unit tests for both the Activity wrapper and the Workflow-to-Activity boundary.
  • Registers the new sample in packaging metadata and documentation, and adds a litellm dependency group plus lockfile entries.

Reviewed changes

Copilot reviewed 11 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
uv.lock Adds lockfile entries for the new litellm dependency group.
README.md Adds the new litellm_activity sample to the repo’s sample index.
pyproject.toml Registers the litellm dependency group and includes litellm_activity as a package.
litellm_activity/init.py Declares the new sample package.
litellm_activity/activities.py Implements the LiteLLM provider call as a Temporal Activity with a client timeout and LiteLLM retries disabled.
litellm_activity/shared.py Adds a serializable request dataclass shared across starter/workflow/activity.
litellm_activity/workflow.py Adds a Workflow that calls the Activity with timeouts and a bounded retry policy.
litellm_activity/worker.py Adds a runnable Worker hosting the workflow and activity.
litellm_activity/starter.py Adds a runnable starter to execute the workflow with CLI/env-provided inputs.
litellm_activity/README.md Documents setup, provider configuration, running, and tests for the new sample.
tests/litellm_activity/init.py Initializes the test package for the new sample.
tests/litellm_activity/activity_test.py Unit-tests the Activity wrapper via monkeypatched LiteLLM call (no network).
tests/litellm_activity/workflow_test.py Tests the Workflow execution path with a mocked Activity implementation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread litellm_activity/README.md Outdated
Comment thread litellm_activity/activities.py Outdated
Comment thread litellm_activity/worker.py Outdated
Comment thread litellm_activity/starter.py Outdated
Comment thread tests/litellm_activity/activity_test.py Outdated
Comment thread litellm_activity/README.md
@Abhinav0905
Abhinav0905 force-pushed the litellm-workflow-sample branch from 7a2f3a7 to d197583 Compare August 8, 2026 20:44
@Abhinav0905
Abhinav0905 force-pushed the litellm-workflow-sample branch from d197583 to cda04e0 Compare August 8, 2026 20:45
@Abhinav0905

Abhinav0905 commented Aug 8, 2026 via email

Copy link
Copy Markdown
Contributor Author

@brianstrauch

Copy link
Copy Markdown
Member

Looks good @Abhinav0905. Are you still working on this PR? It's still marked as draft, but it feels complete to me.

Copy link
Copy Markdown
Contributor Author

Thanks! The implementation is complete and all checks are passing. I left it in draft by mistake; it’s ready for review.

@Abhinav0905
Abhinav0905 marked this pull request as ready for review August 10, 2026 21:38
@Abhinav0905
Abhinav0905 requested review from a team as code owners August 10, 2026 21:38
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@brianstrauch
brianstrauch merged commit 87c4177 into temporalio:main Aug 11, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Add sample demonstrating how to make liteLLM calls from a workflow

5 participants