Skip to content

fix: preserve ambiguous local times in chrono conversions - #6202

Open
rwgk wants to merge 5 commits into
pybind:masterfrom
rwgk:chrono-preserve-fold
Open

rwgk wants to merge 5 commits into
pybind:masterfrom
rwgk:chrono-preserve-fold

Conversation

@rwgk

@rwgk rwgk commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Description

During a backward clock change, chrono conversions discarded datetime.fold and could collapse the two occurrences of a local time into one instant. For example, Los Angeles's two occurrences of 2026-11-01 01:30 reach C++ as the same time point, and returning the second occurrence from C++ produces fold=0 with a Python timestamp one hour early.

Use Python's datetime implementation to disambiguate local timestamps and determine the returned fold. Preserve microseconds separately from whole-second timestamps to avoid floating-point precision loss. Reconstructing a base naive datetime preserves the documented handling of tzinfo and datetime subclasses. A narrow Windows fallback retains valid conversions during the first day after the Unix epoch, where Python's backward fold-detection probes cannot be represented by the Windows runtime.

Add independent tests for both conversion directions, both folds, spring-gap normalization, half-hour and non-DST clock changes, negative timestamps, distant-date microsecond precision, input-field semantics, and early-epoch round trips. Update timezone tests to refresh and restore the process timezone with time.tzset(), and document fold and gap behavior. Use named IANA zones on GraalPy, whose Java timezone parser does not support POSIX rule strings, and check known output wall times and fold values independently of Python conversion results. On GraalPy before 25.1, run timezone-dependent tests in fresh interpreters with TZ set at startup: those releases cache their initial timezone after time.tzset(). Preserve child assertion failures and skips in the parent pytest results.

This follows the fold-based approach discussed in #3805, prompted by James Unicomb's original report in #3451, and clarifies the local-time behavior raised in #1638. The original Sydney spring-gap example cannot round-trip its nonexistent wall time through an actual instant; it follows Python's documented gap normalization.

Local validation (GCC 13.3 / C++20, with incremental scons builds for regular and free-threaded CPython 3.14.4):

  • New regression cases against the original header: 7 failures, confirming the bug.
  • Regular Python: 1,422 pytest tests passed, 4 skipped; 30 Catch2 cases / 1,657 assertions passed.
  • Free-threaded Python: 1,403 pytest tests passed, 23 skipped; 30 Catch2 cases / 1,657 assertions passed.
  • All 72 chrono tests pass in both full suites, and the standalone reproducer reports zero incorrect conversions in both environments.
  • GraalPy 25.0.3 / Python 3.12.8, matching the Ubuntu CI binary: the previous tests reproduced all 12 CI failures; all 72 chrono tests now pass in a C++20 Debug build with warnings as errors. The startup-timezone workaround passed both a serial run (about four minutes) and a final run with eight local workers.
  • GraalPy 25.3.4 / Python 3.13.14: all 72 chrono tests passed in a C++20 Debug build with warnings as errors.
  • Deliberately incorrect child assertions fail the parent, and deliberate child skips remain skips in the parent.
  • Clang-tidy 18.1.3 on test_chrono.cpp, with the full repository configuration, C++17, CPython 3.12 headers, and warnings as errors: the two modernize-use-auto diagnostics reproduce before the correction and the corrected header passes. CI uses clang-tidy 20.
  • C++14 syntax check and prek run -a --quiet passed.

The skips cover unavailable SciPy and <experimental/optional>, plus existing GIL/deadlock exclusions. Windows, PyPy, and 32-bit coverage will come from CI; Windows handling was reviewed against CPython's implementation.

Suggested changelog entry:

  • Fix chrono conversions around backward clock changes by honoring datetime.fold on input and preserving it on returned datetimes. Preserve exact microseconds and document Python-compatible normalization of nonexistent local times.

📚 Documentation preview 📚: https://pybind11--6202.org.readthedocs.build/

@rwgk
rwgk requested a review from henryiii October 11, 2026 06:10

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant