Embedded Laplace: accept rank-deficient Hessian blocks in solver 1 and fix solver fallback reporting - #3419
Open
jachymb wants to merge 6 commits into
Open
Embedded Laplace: accept rank-deficient Hessian blocks in solver 1 and fix solver fallback reporting#3419jachymb wants to merge 6 commits into
jachymb wants to merge 6 commits into
Conversation
block_matrix_sqrt used RealSchur and rejected any negative entry on the Schur diagonal. The block is a negative Hessian of a log-concave likelihood: symmetric positive semi-definite, and rank deficient whenever there are more latent variables than observations (common with a dense coefficient block). Its zero eigenvalues come out of floating point with either sign, so solver 1 failed on every such call and fell back to solver 2 after computing one Hessian for nothing. Symmetrise the block, use SelfAdjointEigenSolver, clamp eigenvalues that are negative only at rounding level and reject a block with a genuinely negative eigenvalue. The non-finite check also only fired when no entry was finite; it now fires when any entry is not. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
log_solver_fallback threw whenever the message stream was null, even with allow_fallthrough set, so callers without a stream lost the fallback entirely. And since the call sits inside std::call_once, a failure with allow_fallthrough unset after an earlier logged fallback was not reported at all and ended in a misleading "solver not valid" error. Decide first: throw with the failing solver's reason when fallthrough is disabled, otherwise log the fallback once if a stream is available. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Collaborator
|
This is cool! Another cool update would be to use Eigen's |
Co-authored-by: Steve Bronder <Stevo15025@gmail.com>
Contributor
Author
|
@SteveBronder |
…tive tolerance Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
Jenkins Console Log Machine informationDistributor ID: Ubuntu Description: Ubuntu 20.04.3 LTS Release: 20.04 Codename: focal CPU: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 43 bits physical, 48 bits virtual CPU(s): 256 On-line CPU(s) list: 0-255 Thread(s) per core: 2 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 2 Vendor ID: AuthenticAMD CPU family: 23 Model: 49 Model name: AMD EPYC 7742 64-Core Processor Stepping: 0 Frequency boost: enabled CPU MHz: 1495.989 CPU max MHz: 3416.0681 CPU min MHz: 1500.0000 BogoMIPS: 4491.56 Virtualization: AMD-V L1d cache: 4 MiB L1i cache: 4 MiB L2 cache: 64 MiB L3 cache: 512 MiB NUMA node0 CPU(s): 0-63,128-191 NUMA node1 CPU(s): 64-127,192-255 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Old microcode: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Mitigation; untrained return thunk; SMT enabled with STIBP protection Vulnerability Spec rstack overflow: Mitigation; Safe RET Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Mitigation; IBPB before exit to userspace Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es G++: g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Clang: clang version 10.0.0-4ubuntu1 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin |
Collaborator
|
Oh shoot it is still in unsupported in 5.0. They moved / fixed it on their main branch but it must have been post release |
…ce. Hides throwing conditionals behind cold compiler hint macro
SteveBronder
approved these changes
Sep 29, 2026
SteveBronder
left a comment
SteveBronder
left a comment
Collaborator
There was a problem hiding this comment.
lgtm! I made a few small cleanups for the conditional statements that throw and I think this is good to merge when tests pass
SteveBronder
approved these changes
Sep 29, 2026
Contributor
Jenkins Console Log Machine informationDistributor ID: Ubuntu Description: Ubuntu 20.04.3 LTS Release: 20.04 Codename: focal CPU: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 43 bits physical, 48 bits virtual CPU(s): 256 On-line CPU(s) list: 0-255 Thread(s) per core: 2 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 2 Vendor ID: AuthenticAMD CPU family: 23 Model: 49 Model name: AMD EPYC 7742 64-Core Processor Stepping: 0 Frequency boost: enabled CPU MHz: 1497.317 CPU max MHz: 3416.0681 CPU min MHz: 1500.0000 BogoMIPS: 4491.85 Virtualization: AMD-V L1d cache: 4 MiB L1i cache: 4 MiB L2 cache: 64 MiB L3 cache: 512 MiB NUMA node0 CPU(s): 0-63,128-191 NUMA node1 CPU(s): 64-127,192-255 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Old microcode: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Mitigation; untrained return thunk; SMT enabled with STIBP protection Vulnerability Spec rstack overflow: Mitigation; Safe RET Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Mitigation; IBPB before exit to userspace Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es G++: g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Clang: clang version 10.0.0-4ubuntu1 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI use disclosure: The code as well as this commentary was done with the help of claude Fable 5.1
This PR bnudles two kinda independent changes (improving the solver and improving fallback reporting; one per commit) but they both improve embedded Laplace implementation so I decided to put them together
Summary
block_matrix_sqrttakes the square root of each Hessian block withEigen::SelfAdjointEigenSolver, clamping eigenvalues that are negative only at rounding level, and the solver-fallback path decides between throwing and logging before itsstd::call_once. Solver 1 no longer fails on rank-deficient blocks (more latent variables than observations), which previously cost one wasted Hessian and a fallback to solver 2 on every such call, and a solver failure is now reported correctly with or without a message stream.One commit per fix:
Wis the negative Hessian of a log-concave likelihood, symmetric positive semi-definite, and rank deficient whenever a block has more latent variables than observations, which is the normal case for a dense coefficient block. The Schur-based code threw on any negative Schur diagonal entry, which such blocks produce by rounding. The tolerance is-block_size * epsilon * max(|eigenvalues|, 1); genuinely indefinite blocks still throw and trigger the fallback as before. The non-finite check fired only when no entry was finite (isFinite().any()) and now fires when any entry is not. Theunsupported/Eigen/MatrixFunctionsinclude is no longer needed.log_solver_fallbackthrew whenever the message stream was null even withallow_fallthroughset, and because it ran insidestd::call_once, a failure withallow_fallthroughunset after an earlier logged fallback was not reported at all and ended in the unrelated "solver not valid" error. Nowthrow_solver_failurethrows with the failing solver's reason when fallthrough is disabled; otherwise the fallback is logged once if a stream exists.Tests
test/unit/math/laplace/block_matrix_sqrt_test.cpp(new): positive definite blocks; rank-deficient blocks, accepted where they threw before; an indefinite block and a block with aNaNentry both throw, the latter passed the old finiteness check.test/unit/math/laplace/laplace_solver_fallback_test.cpp(new), with a likelihood that solver 1 rejects and solver 2 handles: a null stream must not throw (it did); with fallthrough disabled the call must throw with a reason, also after an earlier logged fallback in the same process (it ended in "solver not valid"); a stream receives the warning.Side Effects
block_matrix_sqrtand disabled-fallthrough error messages change; nothing matches on the old texts.internal::log_solver_fallbackloses itsallow_fallthroughparameter.Release notes
Embedded Laplace approximation: solver 1 no longer fails on rank-deficient block Hessians, a null message stream no longer turns an allowed solver fallback into an error, and a solver failure with
allow_fallthroughdisabled is always reported with its reason.Checklist
Copyright holder: Jachym Barvinek
The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
the basic tests are passing
./runTests.py test/unit)make test-headers)make test-math-dependencies)make doxygen)make cpplint)the code is written in idiomatic C++ and changes are documented in the doxygen
the new changes are tested