feat(prompts): add capture_errors option for error tracking by andrewm4894 · Pull Request #427 · PostHog/posthog-python

andrewm4894 · 2026-02-06T18:05:31Z

This is just an exploratory one. As a user i kinda might want posthog errors if we fail to handle prompt as expected. I dont think anything too crazy going on here but worth discussing.

Summary

Adds capture_errors parameter to Prompts.__init__() (default False, fully backward compatible)
When True and a PostHog client is provided, prompt fetch failures are reported via client.capture_exception() before the existing fallback logic runs
If capture_exception() itself fails, it is silently swallowed and never affects prompt resolution

Motivation

When using prompts with fallbacks, fetch failures are silently handled -- the caller gets a fallback string and the error is only logged at WARNING level. With capture_errors=True, these failures surface in PostHog error tracking, making it easy to spot API issues, auth problems, or missing prompts without configuring Python logging.

Usage

from posthog import Posthog
from posthog.ai.prompts import Prompts

posthog = Posthog('phc_xxx', host='https://us.posthog.com', personal_api_key='phx_xxx')
prompts = Prompts(posthog, capture_errors=True)

# Fetch failures now show up in PostHog error tracking
template = prompts.get('my-prompt', fallback='default prompt')

Changes

posthog/ai/prompts.py -- new capture_errors param, _maybe_capture_error() helper
posthog/test/ai/test_prompts.py -- 7 new tests covering all branches

Test plan

capture_exception called on fetch failure with fallback
capture_exception called on fetch failure with stale cache
capture_exception called when error is re-raised (no fallback/cache)
NOT called when capture_errors=False (default)
NOT called when no client provided (direct personal_api_key mode)
NOT called on successful fetch
capture_exception failure does not affect fallback resolution
All 30 existing tests pass unchanged

…error tracking When capture_errors=True and a PostHog client is provided, prompt fetch failures are reported via client.capture_exception(). This surfaces prompt API errors in PostHog error tracking without affecting the existing fallback behavior (stale cache → fallback → raise).

github-actions · 2026-02-06T18:08:28Z

posthog-python Compliance Report

Date: 2026-02-06 18:12:57 UTC
Duration: 143240ms

⚠️ Some Tests Failed

26/29 tests passed, 3 failed

Capture Tests

⚠️ 26/29 tests passed, 3 failed

View Details

Test	Status	Duration
Format Validation.Event Has Required Fields	✅	514ms
Format Validation.Event Has Uuid	✅	1507ms
Format Validation.Event Has Lib Properties	✅	1506ms
Format Validation.Distinct Id Is String	✅	1505ms
Format Validation.Token Is Present	✅	1506ms
Format Validation.Custom Properties Preserved	✅	1506ms
Format Validation.Event Has Timestamp	✅	1506ms
Retry Behavior.Retries On 503	✅	7799ms
Retry Behavior.Does Not Retry On 400	✅	3504ms
Retry Behavior.Does Not Retry On 401	✅	3507ms
Retry Behavior.Respects Retry After Header	❌	7202ms
Retry Behavior.Implements Backoff	❌	20574ms
Retry Behavior.Retries On 500	✅	7069ms
Retry Behavior.Retries On 502	✅	6630ms
Retry Behavior.Retries On 504	✅	7315ms
Retry Behavior.Max Retries Respected	✅	20018ms
Deduplication.Generates Unique Uuids	✅	1007ms
Deduplication.Preserves Uuid On Retry	✅	7362ms
Deduplication.Preserves Uuid And Timestamp On Retry	✅	12656ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	6916ms
Deduplication.No Duplicate Events In Batch	✅	1501ms
Deduplication.Different Events Have Different Uuids	✅	1505ms
Compression.Sends Gzip When Enabled	✅	1506ms
Batch Format.Uses Proper Batch Structure	✅	1506ms
Batch Format.Flush With No Events Sends Nothing	✅	1004ms
Batch Format.Multiple Events Batched Together	✅	1504ms
Error Handling.Does Not Retry On 403	✅	3508ms
Error Handling.Does Not Retry On 413	✅	3505ms
Error Handling.Retries On 408	❌	6511ms

Failures

retry_behavior.respects_retry_after_header

Retry delay too short: 693ms < 2500ms

retry_behavior.implements_backoff

First retry delay too short: 90ms < 100ms

error_handling.retries_on_408

Expected at least 2 requests, got 1

Radu-Raicea

Any way to support both instantiation methods?

greptile-apps

_{2 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-02-06T18:15:42Z

+        self._client = posthog if posthog is not None else None
+        self._capture_errors = capture_errors



Unchecked capture_exception call

_maybe_capture_error() assumes the provided posthog object has a capture_exception method. With capture_errors=True, passing a PostHog client version (or a stub) that lacks capture_exception will raise AttributeError, which then gets swallowed by the broad except Exception, meaning errors silently won’t be reported (and it’s hard to diagnose). Consider guarding the call (e.g., hasattr(self._client, "capture_exception")) and/or logging at least once at WARNING when the method is missing so users know why capture isn’t happening.

Prompt To Fix With AI

This is a comment left during a code review. Path: posthog/ai/prompts.py Line: 97:99 Comment: **Unchecked capture_exception call** `_maybe_capture_error()` assumes the provided `posthog` object has a `capture_exception` method. With `capture_errors=True`, passing a PostHog client version (or a stub) that lacks `capture_exception` will raise `AttributeError`, which then gets swallowed by the broad `except Exception`, meaning errors silently won’t be reported (and it’s hard to diagnose). Consider guarding the call (e.g., `hasattr(self._client, "capture_exception")`) and/or logging at least once at WARNING when the method is missing so users know why capture isn’t happening. How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2026-02-06T18:15:43Z

+class TestPromptsCaptureErrors(TestPrompts):
+    """Tests for the capture_errors option."""
+
+    @patch("posthog.ai.prompts._get_session")
+    def test_capture_exception_called_on_fetch_failure_with_fallback(


Tests are not parameterized

The new TestPromptsCaptureErrors cases are all the same shape (setup fetch failure/success → call get() → assert on capture_exception). This increases repetition and makes it easier to miss scenarios when adding new branches. The repo preference is parameterized tests; these can be collapsed into a single parameterized test that covers (fallback vs stale cache vs re-raise vs success) and toggles (capture_errors on/off, client present/absent).

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review. Path: posthog/test/ai/test_prompts.py Line: 512:516 Comment: **Tests are not parameterized** The new `TestPromptsCaptureErrors` cases are all the same shape (setup fetch failure/success → call `get()` → assert on `capture_exception`). This increases repetition and makes it easier to miss scenarios when adding new branches. The repo preference is parameterized tests; these can be collapsed into a single parameterized test that covers (fallback vs stale cache vs re-raise vs success) and toggles (`capture_errors` on/off, client present/absent). <sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub> How can I resolve this? If you propose a fix, please make it concise.

rafaeelaudibert · 2026-02-19T03:20:28Z

We've updated our release process. We require sampo now. Please rebase on master and check README to understand what should be done.

richardsolomou · 2026-04-03T08:05:47Z

@andrewm4894 is this still relevant?

andrewm4894 · 2026-04-03T08:07:14Z

@andrewm4894 is this still relevant?

Yeah I think it kinda is worth supporting. Was going to sense check the sampo stuff on it and update to get it ready at some point

github-actions · 2026-04-14T09:58:35Z

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

marandaneto · 2026-04-16T10:37:08Z

@@ -1 +1,4 @@
-VERSION = "7.11.2"
+VERSION = "7.8.4"


you dont need to do this

https://github.com/PostHog/posthog-python/blob/main/RELEASING.md

marandaneto · 2026-04-16T10:42:26Z

+            capture_errors: If True and a PostHog client is provided, prompt fetch
+                failures are reported to PostHog error tracking via capture_exception().


so we are reporting our own errors?
what's the final user gonna do with this when those errors are in their error tracking tab?

andrewm4894 self-assigned this Feb 6, 2026

andrewm4894 added the team/llm-analytics label Feb 6, 2026

chore: bump version to 7.8.4

d110b58

andrewm4894 requested a review from a team February 6, 2026 18:07

style: apply ruff formatting

52d6cf0

Radu-Raicea reviewed Feb 6, 2026

View reviewed changes

greptile-apps bot reviewed Feb 6, 2026

View reviewed changes

github-actions bot added the stale label Apr 14, 2026

richardsolomou removed the stale label Apr 14, 2026

Base automatically changed from master to main April 16, 2026 08:09

marandaneto requested a review from a team as a code owner April 16, 2026 08:09

marandaneto reviewed Apr 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(prompts): add capture_errors option for error tracking#427

feat(prompts): add capture_errors option for error tracking#427
andrewm4894 wants to merge 3 commits intomainfrom
feat/prompts-capture-errors

andrewm4894 commented Feb 6, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Feb 6, 2026 •

edited

Loading

Failures

Uh oh!

Radu-Raicea left a comment

Uh oh!

greptile-apps bot left a comment

Uh oh!

greptile-apps bot Feb 6, 2026

Uh oh!

greptile-apps bot Feb 6, 2026

Uh oh!

rafaeelaudibert commented Feb 19, 2026

Uh oh!

richardsolomou commented Apr 3, 2026

Uh oh!

andrewm4894 commented Apr 3, 2026

Uh oh!

github-actions bot commented Apr 14, 2026

Uh oh!

marandaneto Apr 16, 2026

Uh oh!

marandaneto Apr 16, 2026

Uh oh!

marandaneto Apr 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		self._client = posthog if posthog is not None else None
		self._capture_errors = capture_errors

		capture_errors: If True and a PostHog client is provided, prompt fetch
		failures are reported to PostHog error tracking via capture_exception().

Conversation

andrewm4894 commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Usage

Changes

Test plan

Uh oh!

github-actions bot commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

posthog-python Compliance Report

⚠️ Some Tests Failed

Capture Tests

Failures

Uh oh!

Radu-Raicea left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

rafaeelaudibert commented Feb 19, 2026

Uh oh!

richardsolomou commented Apr 3, 2026

Uh oh!

andrewm4894 commented Apr 3, 2026

Uh oh!

github-actions bot commented Apr 14, 2026

Uh oh!

marandaneto Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

marandaneto Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

marandaneto Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

andrewm4894 commented Feb 6, 2026 •

edited

Loading

github-actions bot commented Feb 6, 2026 •

edited

Loading