Skip to content

refactor: [SDK-5065] remove the otel observability path and OpenTelemetry dependency - #2724

Open
abdulraqeeb33 wants to merge 13 commits into
mainfrom
ar/sdk-5065
Open

refactor: [SDK-5065] remove the otel observability path and OpenTelemetry dependency#2724
abdulraqeeb33 wants to merge 13 commits into
mainfrom
ar/sdk-5065

Conversation

@abdulraqeeb33

@abdulraqeeb33 abdulraqeeb33 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Description

One Line Summary

Deletes the legacy OpenTelemetry observability path, the :otel Gradle module and the entire io.opentelemetry dependency tree, making the multiplatform logger pipeline unconditional — and restores the disk-retention and export-retry behavior OTel had been providing by default, as shared KMP code.

Closes SDK-5065.

Details

Motivation

The logger module has been validated in production, so the OpenTelemetry pipeline it was built to replace is now dead weight. Keeping both meant:

  • Shipping two of everything — two ANR detectors, two crash reporters, two platform providers, two lifecycle managers — behind a startup feature-flag branch that made the early-init sequence hard to reason about.
  • Keeping io.opentelemetry on every integrator's classpath. That tree is the source of the recurring R8 Missing class failures behind SDK-4820 (dontwarn rule) and SDK-5006 (shading work), and of the Failed resolution of: Lio/opentelemetry/contrib/disk/buffering/storage/impl/FileStorage crash-reporting failures seen in production on 5.9.3–5.9.5 and 5.7.x/5.8.x.

Net: +1,859 / −5,916 lines across 89 files.

Scope

Deleted

  • The :otel Gradle module in its entirety, plus its settings.gradle include/substitution and the :core dependency.
  • All six OpenTelemetry artifacts and the three version pins in the root build.gradle. Transitive Jackson and AutoValue leave the classpath with them, so integrator APKs shrink and the optional-class -dontwarn rules become unnecessary (see MIGRATION_GUIDE.md).
  • OtelLifecycleManager, OtelAnrDetector, AndroidOtelLogger, OneSignalCrashHandlerFactory.
  • LoggerModuleSwitch, resolveCustomLoggingEnabled, and the SDK_CUSTOM_LOGGING gate.
  • Logging.setOtelTelemetry / logToOtel, and the otel consumer R8 dontwarn rules.
  • CrashDirCleanup.kt / CrashDirCleanupTest.kt. Worth being precise about this one, since the file's history spans two PRs: feat(logger): [SDK-4939] wire Android FileLogStore for shared KMP legacy purge #2691 added it holding only the legacy-purge helpers, this PR added the retention policy to it, and then moved the whole thing into KMP. Net status in this diff is a deletion.

Renamed, not deleted (audited as shared)
The ticket flagged these for audit, and all three turned out to be used by the logger path, so they are renamed off the otel prefix rather than removed:

  • OtelPlatformProviderLoggerPlatformProvider, now implementing ILoggerPlatformProvider directly. This retires LoggerPlatformProviderAdapter, which existed only to bridge the two interfaces.
  • OtelIdResolverLoggerIdResolver (moved into the logger.android package, minus the flag resolver).
  • OtelConfig/OtelConfigEvaluator/OtelSdkSupportObservabilityConfig/ObservabilityConfigEvaluator/ObservabilitySdkSupport.

AnrConstants and AnrCheckEvaluator were also audited — both are already shared by AndroidLogAnrDetector, so they are untouched.

Deliberately unchanged: the crash directory path
The on-disk path stays {cacheDir}/onesignal/otel/crashes despite the name. Renaming it would orphan logger-owned .otlp records that an upgrading install still has pending upload — a real data-loss regression. Records left by a pre-upgrade otel session are OTLP-disk-buffering blobs nothing can read anymore; they are reclaimed by the suffix-based purge in FileLogStore.deleteUnrecognizedEntries (driven by CrashRetention.selectUnrecognized), which satisfies the ticket's "uploaded or intentionally purged" criterion. The retained name is commented at the definition site.

Restoring behavior OTel supplied implicitly

OTel's disk-buffering and OTLP exporter had bounded the crash cache and retried failed exports by default. Removing them without an explicit config change silently dropped both — nothing in the diff said so, which is precisely what made it easy to miss. Both are restored as shared KMP code so iOS gets the same guarantees rather than a second Android-only implementation:

  • Retentioncom.onesignal.logger.crash.CrashRetention / CrashRetentionPolicy, merged in OneSignal-KMP-SDK#20. Defaults: 72 h read-age ceiling, 50 records, 2 MiB budget claim, 512 KiB per-record write limit, .otlp owned suffix. Android's FileLogStore and OneSignalCrashUploaderWrapper call the shared selectors and apply the result with File I/O.

    Both bounds are enforced on every path that touches the directory (save, listReadable, deleteUnrecognizedEntries), and over-limit records are deleted rather than merely hidden from listReadable — otherwise a size-capped record would wedge the backlog forever. Future-dated records (clock skew) are reclaimed rather than treated as infinitely fresh.

  • Export retry/backoff — bounded retry on the remote log export path, merged in OneSignal-KMP-SDK#21.

The OneSignal-KMP-SDK submodule pin moves 87e87fd2 → 64ce06b3 to pick both up. Neither is visible in this repo's diff — review them in the KMP PRs.

Cross-repo sequencing

  1. KMP #20 and #21merged.
  2. This PR — bumps the submodule pin and adopts the shared selectors on Android.
  3. OneSignal-iOS-SDK#1725 — adopts the same shared policy on iOS. Independent of this PR; both consume the same pin.

No KMP release or published artifact is required. Both platforms consume the KMP module as a git submodule pinned to a commit and build it from source — Android via substitute(module('com.onesignal:kmp')).using(project(':OneSignal:kmp')) in settings.gradle, iOS via the same submodule. The pin at 64ce06b3 already contains both merged PRs, so the shared code is live in this diff.

Other behavioral changes worth reviewing

  • First-launch observability. resolveCustomLoggingEnabled() returned false when there was no cached config — i.e. on every first launch after install or upgrade. Deleting otel without addressing that would have left first-run sessions with no observability at all. Removing the switch entirely fixes this: LoggerLifecycleManager comes up unconditionally, and features enable as soon as the first remote config arrives.
  • ANR stacktrace format. Removing OTel also removed the synthetic Throwable the ANR detector built, so ANR records stopped going through stackTraceToString() and lost the type: message header and \tat frame prefix. Ordinary crashes were unaffected, so the pipeline would have emitted two formats and anything parsing exception.stacktrace as a Java stacktrace — frame extraction, grouping, symbolication, the Grafana ^\s*at transform — would have silently stopped matching ANR records only. Both ANR paths now share buildAnrCrashData / buildBackgroundBlockCrashData over one formatJvmStacktrace helper.
  • Start retry. Components that fail to start no longer leave the SDK permanently degraded; LoggerLifecycleManager retries on the next config refresh without tearing down healthy sinks.

Testing

Unit testing

Full :core suite, spotlessCheck and detekt pass across every module. Diff-coverage gate passes at 362/385 touched executable lines (94.0%), against a required ≥ 80%.

  • Restored and renamed the suites covering renamed classes (LoggerIdResolverTest, LoggerPlatformProviderTest, ObservabilitySdkSupportTest, ObservabilityConfigEvaluatorTest) so no coverage was lost with the otel deletions.
  • Added LoggerLifecycleManagerTest and LoggerLifecycleManagerFaultTest — the logger lifecycle manager is now the SDK's only observability path but had no direct tests, since the deleted OtelLifecycleManagerTest was the only lifecycle coverage. Asserts the config state machine by observing real UncaughtExceptionHandler registration, and covers the start-failure/retry path.
  • Added LoggingRemoteTest, replacing the deleted LoggingOtelTest. The old suite could only assert "does not crash" because OpenTelemetry's types were not visible to mocks; ILogTelemetryRemote is our own interface, so this verifies emission, filtering, exception attributes and sink-failure isolation.
  • Extended FileLogStoreTest for retention: per-record write cap, count and byte caps enforced on the crash path, expiry, future-dated reclamation, that an over-cap record is evicted rather than hidden, and that an expired record whose unlink fails is still withheld from readers.
  • Pinned the keepName write-path guarantee. selectOverflowOwned clamps its sort key to min(lastModified, now), so a just-written record can never sort strictly oldest — it lands in the top tie group, where its position comes from listFiles ordering. The guarantee is therefore about tie ordering, not age, and a single-attempt test passes roughly three times in four even with keepName dropped. The test repeats the trial to make a false pass negligible, and was verified to fail against code with the wiring removed.
  • Added an upgrade-path test to OneSignalCrashUploaderWrapperTest proving a pre-upgrade otel record in the real crash dir is reclaimed while a pending logger-owned .otlp record survives.
  • Retention selector logic itself is tested in KMP (CrashRetentionTest), so both platforms exercise one suite.

Coverage tooling fix

The changed-lines coverage gate initially failed at 11%, which was a measurement bug rather than untested code. Robolectric loads classes through its own instrumenting classloader, which strips the source-location metadata JaCoCo uses to attribute execution, so any class exercised only by a @RobolectricTest reported 0% no matter how well tested it was, while plain-JVM tests in the same module reported 96–100%.

This is pre-existing and repo-wideAndroidLogAnrDetector.kt and AndroidLogCrashHandler.kt sit at 0% on main today. It surfaced here only because renaming ~180 lines of Robolectric-only-tested code moved them into the diff-coverage denominator.

Enabling includeNoLocationClasses in jacoco.gradle fixes the attribution. No test changed — only what the report can see. Untouched Robolectric-tested classes are now measured honestly too, so repo-wide numbers reflect real coverage instead of an artifact.

Happy to split this into its own PR if reviewers would rather keep the removal isolated.

Manual testing

Acceptance criteria verified locally:

Criterion Result
No io.opentelemetry in released AARs dependencies --configuration releaseRuntimeClasspath clean for :OneSignal, :core, :notifications, :in-app-messages, :location
No io.opentelemetry in POMs publishToMavenLocal — no match in any published POM; core POM no longer depends on com.onesignal:otel
Release APK No opentelemetry entries in the built release APK
R8 full mode :app:assembleRelease -Pandroid.enableR8.fullMode=true succeeds for both GMS and Huawei flavors, zero missing-class diagnostics
Consumer R8 rules No remaining OpenTelemetry references
MIGRATION_GUIDE.md New section covering the dependency removal, the removed internal API, ProGuard cleanup, and how pre-upgrade crash records on disk are handled

Merge prerequisites

  • Flip the default so logger is the fallback — resolved by deleting the switch entirely.
  • The suspected otel-5.9.9 ANR reporting regression (SDK-5053) becomes permanently unreproducible once this lands — moot by deletion.
  • Accept the loss of the rollback path. Disabling sdk_custom_logging remotely is no longer a mitigation after this merges. Needs explicit sign-off.
  • Finish the rollout. As of 2026-08-24 the flag was ~56% of 5.9.9 installs. Needs effectively-full coverage across app-volume cohorts.
  • Migrate dashboards and alerts off otel-only attributes. telemetry.sdk.name / .version / .language disappear from log records, and anything grouping on com.onesignal.debug.internal.crash.OtelAnrDetector$ApplicationNotRespondingException must move to ApplicationNotRespondingException (SDK-5053). The ANR stacktrace-format fix above means the ^\s*at frame transform keeps working. Lives in OneSignal/infra:dashboards/sdk/.
  • Pick up the shared KMP code. Both KMP PRs are merged and the submodule pin is bumped to 64ce06b3, which contains both. No KMP release is needed — the module is built from pinned source, not consumed as a published artifact.

Follow-up (deliberately out of scope)

The SDK_CUSTOM_LOGGING enum constant lives in OneSignal-KMP-SDK, so removing it needs a KMP PR — cheap now that this PR already bumps the pin, but not worth folding in:

  • Nothing in the Android SDK gates on it after this PR — the only remaining reference is FeatureManagerTests, which uses it purely as a fixture.
  • It is currently the only APP_STARTUP flag in the catalog, so deleting it removes the sole test subject for FeatureManager's startup-latching behavior on both sides. That removal should land alongside a replacement APP_STARTUP flag or a test-only fixture.

Leaving the constant in place is harmless: the server may still send the key and FeatureManager will track it, but no behavior depends on it.

Affected code checklist

  • Notifications
  • Display
  • Open
  • Push Processing
  • Confirm Deliveries
  • Outcomes
  • Sessions
  • In-App Messaging
  • REST API requests — no endpoint change; the log-export path moves off the otel exporter to the existing OneSignalLogHttpSender, which now retries with backoff
  • Public API changes

Integrator-facing dependency change (documented in MIGRATION_GUIDE.md), but no source-compatible API change.

Checklist

Overview

  • I have filled out all REQUIRED sections above
  • PR does one thing — removes the otel path. The renames are not incidental cleanup; the shared classes could not keep compiling against the deleted :otel interfaces. The retention and retry work restores behavior the removal would otherwise have silently dropped.
  • Any Public API changes are explained in the PR details and conform to existing APIs

Testing

  • I have included test coverage for these changes, or explained why they are not needed
  • All automated tests pass, or I explained why that is not possible
  • I have personally tested this on my device, or explained why that is not possible — verified via R8 full-mode release builds and APK inspection rather than on-device; on-device crash and ANR validation should ride along with the rollout sign-off.

Final pass

  • Code is as readable as possible.
  • I have reviewed this PR myself, ensuring it meets each checklist item

Made with Cursor

…etry dependency

The multiplatform logger module is validated in production, so the legacy
OpenTelemetry pipeline it was built to replace is now dead weight. Keeping both
meant shipping two ANR detectors, two crash reporters, two platform providers and
two lifecycle managers behind a startup feature-flag branch, and it kept the
io.opentelemetry tree on every integrator's classpath — the source of the
recurring R8 "Missing class" failures in SDK-4820 and SDK-5006.

The logger pipeline is now unconditional. LoggerModuleSwitch, the
SDK_CUSTOM_LOGGING gate and resolveCustomLoggingEnabled are gone, which also
fixes the first-launch gap: with no cached config the switch defaulted to otel,
so a freshly installed app would have had no observability at all once otel was
deleted.

Code the logger path shared with otel is kept and renamed off the otel prefix
rather than deleted: OtelPlatformProvider now implements ILoggerPlatformProvider
directly (retiring the adapter), OtelIdResolver becomes LoggerIdResolver, and the
OtelConfig/OtelSdkSupport pair becomes ObservabilityConfig/ObservabilitySdkSupport.

The crash directory keeps its `onesignal/otel/crashes` path on purpose. Renaming
it would orphan logger-owned records an upgrading install still has pending;
OTel-format records left in it are reclaimed by the existing suffix-based purge.

Verified: no io.opentelemetry in any published module's releaseRuntimeClasspath
or POM, none in the release APK, and the example app minifies under R8 full mode
for both flavors with no missing-class diagnostics.

Co-authored-by: Cursor <cursoragent@cursor.com>
@abdulraqeeb33
abdulraqeeb33 requested a review from a team August 24, 2026 15:49
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

📊 Diff Coverage Report

Diff Coverage Report (Changed Lines Only)

Gate: aggregate coverage on changed executable lines must be ≥ 80% (JaCoCo line data for lines touched in the diff).

Changed Files Coverage

  • AnrCheckEvaluator.kt: 22/22 touched executable lines (100.0%) (77 touched lines in diff)
  • ObservabilitySdkSupport.kt: 4/4 touched executable lines (100.0%) (27 touched lines in diff)
  • ⚠️ OneSignalCrashHandlerFactory.kt: Not in coverage report (may not be compiled/tested)
  • OneSignalCrashUploaderWrapper.kt: 7/7 touched executable lines (100.0%) (20 touched lines in diff)
  • ⚠️ OtelAnrDetector.kt: Not in coverage report (may not be compiled/tested)
  • Logging.kt: 4/4 touched executable lines (100.0%) (12 touched lines in diff)
  • ⚠️ LoggerModuleSwitch.kt: Not in coverage report (may not be compiled/tested)
  • AndroidLogAnrDetector.kt: 0/2 touched executable lines (0.0%) (8 touched lines in diff)
    • 2 uncovered touched lines in this file
  • ⚠️ CrashDirCleanup.kt: Not in coverage report (may not be compiled/tested)
  • FileLogStore.kt: 69/72 touched executable lines (95.8%) (155 touched lines in diff)
  • LoggerIdResolver.kt: 88/97 touched executable lines (90.7%) (247 touched lines in diff)
  • ⚠️ LoggerPlatformFactory.kt: Not in coverage report (may not be compiled/tested)
  • LoggerPlatformProvider.kt: 81/83 touched executable lines (97.6%) (204 touched lines in diff)
  • ⚠️ LoggerPlatformProviderAdapter.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ AndroidOtelLogger.kt: Not in coverage report (may not be compiled/tested)
  • LoggerLifecycleManager.kt: 81/94 touched executable lines (86.2%) (173 touched lines in diff)
  • ObservabilityConfigEvaluator.kt: 20/20 touched executable lines (100.0%) (66 touched lines in diff)
  • OneSignalImp.kt: 4/4 touched executable lines (100.0%) (12 touched lines in diff)
  • ⚠️ OtelConfigEvaluator.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelLifecycleManager.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ IOtelCrashHandler.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ IOtelCrashReporter.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ IOtelLogger.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ IOtelOpenTelemetry.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ IOtelPlatformProvider.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OneSignalOpenTelemetry.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelFactory.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelLoggingHelper.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelFieldsPerEvent.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelFieldsTopLevel.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelConfigCrashFile.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelConfigRemoteOneSignal.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelConfigShared.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ IOtelAnrDetector.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelCrashHandler.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelCrashReporter.kt: Not in coverage report (may not be compiled/tested)
  • ⚠️ OtelCrashUploader.kt: Not in coverage report (may not be compiled/tested)

Overall (aggregate gate)

380/409 touched executable lines covered (92.9% — requires ≥ 80%)

Per-file detail (informational; gate is aggregate above):

  • AndroidLogAnrDetector.kt: 0.0% (2 uncovered touched lines)

📥 View workflow run

AR Abdul Azeez and others added 11 commits August 24, 2026 11:03
Robolectric loads classes through its own instrumenting classloader, which strips
the source-location metadata JaCoCo uses to attribute execution. Every class
exercised only by a @RobolectricTest therefore reported 0% coverage no matter how
well tested it was, while plain-JVM tests in the same module reported ~96-100%.
CrashDirCleanup's doc comment already alludes to this, noting that keeping the
logic free of Robolectric is what gets it "counted by Jacoco on the plain JVM".

The gap was invisible until the otel removal renamed ~180 lines of
Robolectric-only-tested code, which moved them into the diff-coverage
denominator and failed the changed-lines gate at 11%.

Enabling includeNoLocationClasses fixes the attribution. Nothing about the tests
changed, only what the report can see:

  LoggerPlatformProvider        1.2% -> 98.8%
  LoggerIdResolver              0.0% -> 90.7%
  LoggerLifecycleManager        0.0% -> 84.7%
  OneSignalCrashUploaderWrapper 0.0% -> 82.6%
  Logging                      47.0% -> 84.0%
  OneSignalImp                 28.7% -> 71.9%

Untouched Robolectric-tested classes are now measured honestly too
(AndroidLogAnrDetector 0% -> 49.5%, FileLogStore 0% -> 34.5%), so the reported
figures reflect real coverage rather than a measurement artifact.

Co-authored-by: Cursor <cursoragent@cursor.com>
…st seams

Removing :otel took its disk-buffering config with it, including the 72h
maxFileAgeForRead and the per-file/per-folder size limits. FileLogStore only
had a lower age bound, and the purge deliberately skips owned .otlp records at
any age, so a record that never uploaded — including one written while remote
logging is off, which is never even read — would be retried every launch
forever. Restore both bounds and delete over-limit records rather than merely
hiding them from listReadable.

The fold-in of OtelLifecycleManager also dropped its injectable factories,
which left the surviving pipeline's try/catch isolation, ANR start/stop, and
remote-sink wiring untestable. Restore the seams with production defaults so
runtime wiring is unchanged, and port the fault matrix.

Also correct the migration guide: the otel artifact is no longer published and
Logging.setOtelTelemetry is gone, so "no API change" was wrong.

Co-authored-by: Cursor <cursoragent@cursor.com>
…pgrade docs

Round-2 review found the accumulation caps were enforced only in save(), so an
install carrying a backlog from a build without caps — which includes the large
5.9.x cohort already on the logger path — was fully listed and re-POSTed every
launch until a new crash happened to trim it. Both bounds now run on
listReadable and deleteUnrecognizedEntries too, reclaiming before payloads are
read so an over-cap directory is never fully loaded. The crash path keeps only
a cheap bounded trim; bulk reclaim happens on the uploader's IO paths.

The byte cap also treated the first over-budget record as a cutoff, so one
oversized payload evicted the entire older backlog — the opposite of what the
cap is for. Skip it instead, and add a per-record cap so an outsized payload is
dropped alone. selectOverflowOwnedEntries now also pins the record save() just
wrote, so a backwards clock step cannot make it sort oldest and delete it.

disableFeatures cleared each field only after the teardown call returned, so a
throwing stop()/unregister() left the field set and the start guards then
treated the dead component as running for the rest of the process.

The migration guide claimed all pre-upgrade crash records are deleted. That is
true only for OTel-format records; logger-path records are uploaded normally,
and telling integrators otherwise would misdirect support.

Tests: JVM coverage for both selectors including boundary, tie-break, oversized
and keepName cases; re-enable-after-teardown-failure cases; the enable-twice
case now asserts something; and the fault suite no longer leaks a mock sink
into the global Logging object.

Co-authored-by: Cursor <cursoragent@cursor.com>
…cklog

Round-3 review found two ways the retention policy could delete crash reports
it was meant to protect.

keepName pinned the just-written record but charged its full length to the
shared budget. An oversized payload therefore started the budget over cap,
every sibling failed the remaining-budget check, and the whole backlog was
evicted -- then the uploader, which runs without keepName, dropped the
oversized record too. One bad payload destroyed everything including itself.
The test covering that path used a single-entry directory, so it could observe
the retention but never the consequence.

Separately, the cheap exit in enforceAccumulationCaps checked count and total
bytes but not the per-record cap, so a lone 600 KiB report survived save() and
was then deleted by the uploader before any upload was attempted.

Fixed at the source instead of patching the selector: save() now refuses a
payload over the per-record limit and says so, which makes "every stored record
is within the shared budget" an invariant. Size is no longer grounds for
eviction -- deleting a captured crash unread is worse than keeping it -- and
each record now claims at most the per-record cap against the budget, so an
oversized record inherited from a build without the write-time limit still gets
an upload attempt without displacing anything.

Also: startLogging never received the clear-before-teardown fix disableFeatures
got, so a throwing shutdown() stranded a dead sink that NoChange would never
replace; expired-but-undeletable records were filtered out of the byte
accounting and could hold the directory over cap indefinitely; and three
lifecycle tests spawned real ANR watchdog daemon threads that outlived the spec
and wrote into the cache dir other specs assert on.

Co-authored-by: Cursor <cursoragent@cursor.com>
applyAction committed currentConfig even when a component never came up. Since
a stable remote payload produces an identical config on the next refresh, the
evaluator returned NoChange and the dead crash handler, ANR detector or sink
stayed down for the rest of the process. enableFeatures now reports whether
everything started, the config is only committed once it did, and startLogging
is null-guarded like its siblings so a retry cannot tear down a healthy sink.

startLogging also only had half the teardown invariant: it cleared its own
field but left Logging's global pointing at the old sink while shutting it
down. Every log emitted between shutdown and the replacement being installed --
including the warn in that window -- went to a telemetry whose consumer was
already cancelled, where it queued and was never drained. On a throwing factory
the global stayed on the dead instance for the session.

Reverts the ExpiryOutcome split from the previous commit. It was added on the
theory that an expired record whose delete failed could hold the directory over
cap while invisible to the byte accounting. Writing the test disproved it:
expired records are by definition the oldest, so the selector always picks them
for eviction rather than retention, and only retained records claim budget.
Including them in the candidate set changes no outcome, so the two-set
bookkeeping was inert complexity. Kept a test that the record stays unreadable
when its delete fails, which is the part that does matter.

Also drops a tautological assertion that passed regardless of keepName now that
size is not grounds for eviction, replaces a counter mutated from six
concurrent coroutines with an AtomicInteger, stops building a throwaway
platform provider just to read a path the pure helper computes, and corrects
two KDocs that still claimed the byte cap bounds disk rather than claim.

Co-authored-by: Cursor <cursoragent@cursor.com>
Removing the OpenTelemetry path also removed the synthetic Throwable the ANR
detector used to build, so ANR records stopped being serialized via
stackTraceToString() and were hand-joined instead — no `type: message` header
and no `\tat ` frame prefix. Ordinary crashes still went through
stackTraceToString(), so the pipeline emitted two different stacktrace formats
depending on record type, and consumers that parse `exception.stacktrace` as a
Java stacktrace (frame extraction, grouping/fingerprinting, symbolication, the
Grafana `^\s*at ` transform) silently stopped matching ANR records only.

Both ANR paths now go through shared `buildAnrCrashData` /
`buildBackgroundBlockCrashData` builders backed by one `formatJvmStacktrace`
helper that emits the canonical layout.

Chose hand-formatting over re-synthesizing a Throwable for two reasons: this
runs on the ANR watchdog thread while reporting a possibly-wedged app, so
avoiding a throwable allocation and its stack fill keeps it cheap and
non-throwing; and a real exception class would put its fully-qualified name in
the header, which would no longer match the bare `exceptionType` the record
reports. Against drift, a test pins `formatJvmStacktrace` output against a real
`Throwable.stackTraceToString()`, so the ANR and crash paths cannot diverge
again without a red test.

`exceptionType` values are unchanged — this touches the `stacktrace` field only.

Co-authored-by: Cursor <cursoragent@cursor.com>
… size

CrashDirCleanup is a near-duplicate of the shared CrashRetention policy in the
KMP submodule and is slated for deletion once that lands and the pin is bumped.
This PR may merge first, so the two correctness defects are fixed here rather
than left to merge ordering. Ports commit a95117a from the KMP repo,
deliberately excluding its CrashRetentionPolicy value type: that change exists
to shorten Swift call sites, which have no Kotlin default arguments after the
Objective-C export. Android has no such boundary, so the parameter-heavy
signatures stay and the diff stays reviewable.

Reclaim records dated far enough into the future to be unrecoverable. The read
path gates on `now - lastModifiedMs >= minAgeMillis`, which a future timestamp
never satisfies, and selectExpiredOwnedEntries ignored every negative age, so
such a record was unreadable for its entire life while still holding a count
slot and budget — and it sorted newest during overflow, so it displaced genuine
records that could still have been uploaded.

The threshold is a full retention window ahead of now, not merely "in the
future". That preserves the deliberate protection against a modest backwards
clock step, which is what the negative-age handling was there for: a record
dated modestly ahead is still left to wait until the clock agrees it is old.
Clamping the timestamp for ordering alone would not have been sufficient — a
record clamped to nowMs still ranks as the newest entry and keeps its slot.

selectOverflowOwnedEntries now takes nowMs and applies the same judgement when
ordering. This is needed on Android for the same reason it is on iOS:
FileLogStore.enforceAccumulationCaps runs on the crash write path and enforces
caps without running an expiry pass first, so ordering cannot assume the zombie
has already been removed. Ordinary future dates clamp to nowMs; unrecoverable
ones sort last. The two uploader-side callers already had a `now` in scope.

Make CrashDirEntry.lengthBytes required. Budget claim is
`min(lengthBytes, maxRecordBytes)`, so the previous `= 0L` default meant a
caller that omitted the size claimed nothing and disabled the byte budget for
that record. Both production call sites already passed a real length, so this
was latent — but several test cases relied on the default, which is exactly the
hazard. Tests now pass an explicit size.

Replace the test that pinned the bug as correct. It asserted a record dated two
full retention windows into the future was correctly ignored, citing backwards-
clock protection — but two windows ahead is not a clock step, and the case it
described is an hour of skew. It is now split into a plausible one-hour
backwards step that must be left alone and a boundary case at exactly one
window, matching KMP.

Also coerce formatCrashDirInventory's maxSample to at least zero. Both callers
pass literals today, but List.take throws on a negative argument and this is a
logging helper on a crash-adjacent path.

Each new test was confirmed red against the reverted production change and
green after: reverting the expiry clause fails only "reclaims a record dated
past the window into the future"; reverting the overflow sort key fails only "a
future-dated record is evicted before any record that could still upload";
reverting the maxSample coercion fails only "treats a negative sample size as
zero". The three tests guarding the lower bound — the one-hour step, the
exactly-one-window boundary, and the modestly-future ordering case — were
confirmed red against an over-correction that reclaims any future date, since
no under-correction can fail them.

Behavior now matches the KMP implementation exactly; only the signatures
differ, which is what the comparison should find when the duplicate is deleted.

Co-authored-by: Cursor <cursoragent@cursor.com>
…licy

Moves the pin from 87e87fd to 64ce06b, picking up:

- #20 shared crash-record retention policy (CrashRetention /
  CrashRetentionPolicy / CrashDirEntry in commonMain, with 29
  commonTest cases running on both JVM and iOS)
- #21 bounded retry/backoff for remote log export

Pointer change only; Android still uses its local duplicate of the
retention logic, which the next commit removes.

Co-authored-by: Cursor <cursoragent@cursor.com>
CrashDirCleanup.kt was a near-duplicate of KMP's CrashRetention, written
only because the shared version did not exist yet. Now that it does,
Android consumes it and the local copy goes, leaving FileLogStore
responsible for nothing but File I/O — turning a directory listing into
CrashDirEntrys and applying the decisions the shared selectors return.

Pure refactor, no behaviour change. The shared bounds are identical to
the ones the deleted constants carried (72h read age, 50 records, 2 MiB
budget, 512 KiB per record, ".otlp"), and the selector bodies match
line for line, including the full-window future-date threshold and the
clamp-vs-sort-last ordering.

Shape differs deliberately: the shared API groups the bounds into a
CrashRetentionPolicy that every selector takes, so FileLogStore holds
one CrashRetention.defaultPolicy instance and passes the same one
everywhere rather than relying on per-call defaults. The inline
cheap-exit in enforceAccumulationCaps is now CrashRetention.isWithinCaps,
which shares the selector's capped accounting instead of restating it.

CrashDirCleanupTest goes with the implementation it covered: KMP's
CrashRetentionTest is a strict superset of its 22 cases, and runs them
on both JVM and iOS. FileLogStoreTest covers Android's own file I/O and
stays, asserting against the shared policy rather than copies of its
numbers.

Co-authored-by: Cursor <cursoragent@cursor.com>
Applies the review standard already applied to the KMP side: a comment
should state a constraint the code cannot show, never provenance, never a
narration of the next line, never an argument aimed at a reviewer that the
change is correct.

Cut across FileLogStore, AnrCheckEvaluator, LoggerLifecycleManager,
AndroidLogAnrDetector, OneSignalCrashUploaderWrapper and their tests:

- Provenance: references to the removed OpenTelemetry disk-buffering
  library, "mirrors the old otel behavior", "ported from the deleted otel
  equivalent". That history lives in this PR body and in the commits that
  removed the module.
- Reviewer-facing justification: paragraphs defending the one-time cost of
  a crash-path trim, the testability of the pure decision core, and why the
  crash-dir path helper is preferred over building a provider.
- Repetition: the AnrCheckResult doc comments were restated verbatim on the
  BlockClassification entries; the teardown-ordering invariant was spelled
  out in the production code and again in two test comments; the stack
  fingerprint rationale appeared in three places.

Kept the comments where the obvious reading is wrong: the byte cap bounds
the budget claim rather than disk bytes, expired names are returned even
when the unlink fails, save() must use raw Logcat because Logging.info can
run app listeners, keepName exists so save() cannot evict its own record,
and ANR stacktraces must stay byte-identical to the crash path's format.

Comments and KDoc only — no non-comment line is touched.

Co-authored-by: Cursor <cursoragent@cursor.com>
`save never evicts the record it just wrote` passed with `keepName` removed from
`enforceAccumulationCaps` entirely, so the wiring was unverified. Both sort keys
clamp to `nowMs`, so the record `save` just wrote can never sort strictly oldest;
it lands in a tie group with any backlog dated at or ahead of the clock, and its
position inside that group is whatever the filesystem happens to list. Only the
explicit reservation keeps it. Measured on the old fixture, eviction without
`keepName` was a coin flip that landed the safe way 7 times in 25, which is why a
single attempt looked green. The fixture now dates the backlog ahead of the clock
and repeats, so a false pass is vanishingly unlikely; it fails without the
reservation and passes with it.

Also restores the note explaining why the `isWithinCaps` short-circuit is what
makes a full sort acceptable on the crashing thread, and passes the policy to
`formatInventory` explicitly so every shared-selector call site reads alike.

Co-authored-by: Cursor <cursoragent@cursor.com>
@abdulraqeeb33
abdulraqeeb33 marked this pull request as ready for review August 26, 2026 19:38

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multi-model review (Opus 5, GPT 5.6 Sol, Grok 4.6) of the OTel removal and remaining logger path.

OTel deletion, leftover-reference cleanup, ANR stacktrace formatting, and the intentional {cacheDir}/onesignal/otel/crashes path look consistent with the stated intent. The untagged KMP pin is already listed as a merge prerequisite.

Act on

  • Remote disable / log-level updates can be skipped after a partial Enable (3/3). applyAction only commits currentConfig when every component starts. The evaluator then treats prior state as disabled, so a later isEnabled=false HYDRATE is NoChange and never calls disableFeatures(). The same stuck-null config makes a later level change evaluate as Enable instead of UpdateLogLevel, and if (remoteTelemetry == null) leaves shouldSend pinned at the original level. Fault tests cover retry-on-identical-enable and disable-after-full-enable, not disable or level-change after a partial start.

Consider

  • FileLogStore.save() still lists and stats the whole crash directory on the uncaught-exception thread before the cheap isWithinCaps check, including any inherited OTel backlog (1/3).
  • initialize()/start() install process-global state, then log through Logging (app listeners). A throwing listener leaves the field null, so retry can install a second UEH / ANR watchdog (1/3).

Noted / dismissed

  • Untagged KMP pin vs the publish vX.Y.Z gate — already documented as a merge blocker.
  • Write path skipping selectExpiredOwned — crash-thread cost; class KDoc overclaims both bounds on every path.
  • formatJvmStacktrace “byte-identical” vs the bare ANR type name — comment accuracy only.
Open in Web View Automation 

Sent by Cursor Automation: PR Reviews

Comment on lines +148 to +162
private fun applyAction(action: ObservabilityConfigAction, newConfig: ObservabilityConfig) {
val applied =
when (action) {
is ObservabilityConfigAction.Enable -> enableFeatures(newConfig.logLevel ?: LogLevel.ERROR)
is ObservabilityConfigAction.UpdateLogLevel -> updateLogLevel(action.newLevel)
is ObservabilityConfigAction.Disable -> {
disableFeatures()
true
}
is ObservabilityConfigAction.NoChange -> {
Logging.debug("OneSignal: logger config unchanged")
true
}
}
if (applied) currentConfig = newConfig

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Act on (3/3): holding currentConfig back until every component starts breaks the remote kill switch and log-level updates.

ObservabilityConfigEvaluator derives wasEnabled from old?.isEnabled == true. After a partial Enable (e.g. crash handler and remote sink up, ANR throws), currentConfig stays null:

  1. HYDRATE isEnabled=falseNoChangedisableFeatures() never runs. Healthy sinks keep shipping and the UEH stays installed.
  2. HYDRATE enabled at a new level → Enable again, not UpdateLogLevel. if (remoteTelemetry == null) then skips startLogging, so shouldSend stays closed over the original level. If the missing component later succeeds, committed config and the live predicate disagree for the rest of the process.

This is new relative to the old always-commit currentConfig = newConfig. LoggerLifecycleManagerFaultTest retries on an identical enable and disables after a full start, but never follows a partial start with a disable or a level change.

Track desired config separately from component health: always commit the last observed remote snapshot, call disableFeatures() whenever !new.isEnabled and anything is still up, and on Enable retry reinstall shouldSend if the requested level differs.

@fadi-george

Copy link
Copy Markdown
Contributor

Multi-model review:
Act on (consensus)

  1. Partial Enable then Disable never tears down. currentConfig stays null after a failed start, so null → disabled evaluates to NoChange and disableFeatures() never runs. Crash handler, ANR watchdog, and remote sink keep running after the backend kill switch.

  2. Enable retry commits a new log level without moving the sink. if (remoteTelemetry == null) startLogging(logLevel) skips a healthy sink, then allStarted commits the new level. A cache-at-ERROR then hydrate-at-DEBUG process ships at ERROR for the rest of its life.

Both come from the same gap: desired config vs actual component health. Treat Disable as “ensure off if anything is live,” and on Enable retry apply logLevel when it differs from the live sink.

Consider

  • Failed UpdateLogLevel detaches the sink first; a throwing factory leaves remote logging dead, and the next identical HYDRATE is NoChange so it never retries.
  • A host ILogListener that throws after initialize()/start() can leave the field null while the handler/watchdog is already installed, so the next retry chains a second one.

Noted / dismissed

Crash-export “one failed file aborts the pass” is in the KMP pin, not Android-authored code. Retention KDoc drift and the non-root unlink test are nits.

No dangling :otel / setOtelTelemetry / io.opentelemetry references. Crash dir path kept on purpose. ANR stacktrace format matches Throwable.stackTraceToString().

…iveness

LoggerLifecycleManager conflated desired config with actual component health.
currentConfig only advances once every component started, so a partial-failure
Enable leaves components running under a config that was never committed. Two
bugs followed from that gap:

- A later disabled payload evaluated as null -> disabled, which the evaluator
  reads as NoChange, so the remote kill switch never tore anything down and the
  crash handler, ANR watchdog and remote sink kept running.
- An Enable retry carrying a newer level skipped startLogging because the sink
  was healthy, then committed the new level while the sink still filtered at the
  old one, permanently for the process.

Disable now runs whenever any component is actually live, and the sink guard
compares against the level the sink was really started at rather than assuming a
live sink is at the requested level.

Also unwinds a partially started crash handler or ANR detector: initialize()
can chain onto the process-global uncaught-exception handler before throwing, so
dropping the reference let a retry install a second one and double-report. The
component is unregistered before the field is cleared, which preserves the
existing invariant that a field never points at something dead.

Co-authored-by: Cursor <cursoragent@cursor.com>
@abdulraqeeb33

Copy link
Copy Markdown
Contributor Author

Thanks — both consensus findings were real and are fixed in 209132e81. Your framing of the shared cause (desired config vs. actual component health) was the right one, so the fix reconciles against liveness rather than patching the two symptoms. One of the "Consider" items doesn't hold, with the trace below.

1. Partial Enable then Disable — confirmed and fixed

Confirmed the exact path. With currentConfig left at null by a partial-failure Enable, evaluate(old = null, new = disabled) has wasEnabled = false and isNowEnabled = false, so it misses !wasEnabled && isNowEnabled, misses wasEnabled && !isNowEnabled, and falls through to NoChange. Anything that did start keeps running through the kill switch.

applyAction now short-circuits on actual liveness before consulting the verdict:

if (!newConfig.isEnabled && isAnyFeatureLive()) {
    disableFeatures()
    currentConfig = newConfig
    return
}

private fun isAnyFeatureLive(): Boolean =
    crashHandler != null || anrDetector != null || remoteTelemetry != null

The liveness gate is also what keeps it from thrashing: after teardown nothing is live, so repeat disabled payloads fall through to NoChange as before.

Proof. disable tears down live components after a partial-failure enable. Reverting just the short-circuit:

java.lang.AssertionError: Verification failed: call 1 of 1: ILogAnrDetector(#190).stop()) was not called.

That is the kill switch being ignored. Restored, the suite is green.

2. Enable retry commits a level the sink never adopted — confirmed and fixed

Also confirmed. The guard treated any live sink as correct, so a retry entering enableFeatures(DEBUG) with a healthy ERROR sink skipped startLogging and then committed currentConfig at DEBUG. Every later identical HYDRATE is NoChange, so the process ships at ERROR for its lifetime.

The manager now tracks the level the sink was actually started at:

if (remoteTelemetry == null || activeLogLevel != logLevel) startLogging(logLevel)

Proof. an enable retry moves a live sink to the newly requested level fails when the old if (remoteTelemetry == null) guard is restored.

3. Host listener throwing after initialize()/start() — real, fixed

startCrashHandler assigned the field only after initialize(), so a handler that installed itself and then threw left the field null while chained to the global — and the next retry chained a second one. startCrashHandler and startAnrDetector now unregister/stop the partially-started component before rethrowing, so allStarted still goes false.

Worth noting how this sits against the existing invariant in disableFeatures, which clears fields before teardown. The two orders look opposite but enforce the same rule: a field never points at a component that isn't running. Going down that means clear-then-call; coming up it means undo-then-don't-publish.

Covered by a crash handler that throws after installing itself is unregistered and an ANR detector that throws after starting is stopped, both verified red without the unwind blocks.

4. Failed UpdateLogLevel never retries — doesn't hold

This one I'd push back on. updateLogLevel returns false when startLogging throws, and applyAction only commits on success:

if (applied) currentConfig = newConfig

So currentConfig retains the old level with isEnabled = true. The next HYDRATE at the new level therefore hits wasEnabled && isNowEnabled && old.logLevel != new.logLevel and evaluates to UpdateLogLevel again — it retries. It would only collapse to NoChange if the failed level had been committed, which is exactly what the guard prevents.

Added a failed log level update is retried on the next identical config (enable at ERROR → WARN with a throwing factory → WARN again, asserting a third factory call). To be clear about its status: it passes against unmodified code, so it documents existing behavior rather than defending a fix — no perturbation claim behind it.

Remote logging is left detached until a retry succeeds, which is the intended failure mode: better no sink than a dead one wired into Logging's global.

One test I deleted

I wrote a test defending the anti-thrash property and then removed it. Perturbing toward the naive alternative (dropping && isAnyFeatureLive()) left it passing — after the first teardown the fields are null, so detector?.stop() is a no-op either way and the call count is identical with or without the guard. It couldn't fail, so it went. The gate is still justified (it avoids pointless teardown and log noise on every disabled payload), but I'm not claiming coverage for that aspect.

Verification

Full :core suite 910 passing, spotlessCheck clean, detekt clean in the touched files (the two pre-existing findings in OneSignalDispatchers.kt and FeatureFlagsRefreshService.kt are unchanged).

Also worth recording, since it's the interesting part: both bugs came from ingredients that were each fixes from earlier review rounds on this PR — the conditional commit so partial failures retry, and the remoteTelemetry == null guard so retries don't tear down a healthy sink. Individually correct, jointly broken. Neither earlier round caught it because each looked only at the change in front of it; reviewing the resulting state machine as a whole is what surfaced these.

@abdulraqeeb33
abdulraqeeb33 enabled auto-merge (squash) August 28, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants