Skip to content

Code enhancement#162

Merged
jchiquet merged 59 commits into
masterfrom
code-enhancement
Jun 15, 2026
Merged

Code enhancement#162
jchiquet merged 59 commits into
masterfrom
code-enhancement

Conversation

@jchiquet

@jchiquet jchiquet commented Jun 10, 2026

Copy link
Copy Markdown
Member

Various enhancements in code and doc:

  • New built-in Newton optimizer (backend = "builtin") for PLN, ZIPLN and PLNPCA,
    now the default for PLN and ZIPLN. Uses the alternate parametrization of PLN (variational means embeded the covariate effects). Uses envelope-theorem Newton steps with strong Wolfe line search; does not depend on NLOPT. Substantially faster and more accurate than nlopt on large datasets with full covariance (e.g. +30 000 loglik on microcosm, p=259). CCASQ/nlopt remains competitive for PLNPCA

  • Fix critical convergence bug in PLN/PLNPCA with nlopt: premature termination due to ill-conditioned X scaling triggered the XTOL stopping criterion after very few iterations (e.g. 14 iter on barents, loglik -8520 instead of -4400). The built-in backend is immune to this bug; nlopt is also fixed via better parameter scaling.

  • ZIPLN joint VE step (backend = "builtin"): variational parameters (M, ψ, R) are now optimised jointly in a single Newton step per EM iteration, instead of sequentially. Gains up to +666 loglik on oaks compared to the nlopt sequential approach.

  • PLNPCA warm-start and shared SVD init: the collection of PLNPCA models now shares a single SVD initialisation computed once from a fast LM (init_method = "LM", default). A pre-fitted PLNfit can be supplied via inception = PLN(...) to improve convergence for large ranks (e.g. rank > sqrt(p)); see ?PLNPCA_param for details.

  • torch backend marked experimental: the torch backend is now clearly documented as experimental. It emits a message on use and is not recommended for PLNPCA (systematically lower loglik than nlopt/builtin). It remains available for PLN on diagonal/spherical covariance where it can be faster.

Other changes

jchiquet and others added 30 commits June 5, 2026 19:24
Ajoute un optimiseur Newton diagonal coordonné (sans NLopt) pour les
modèles PLN standard et ZIPLN, activable via
  PLN_param(config_optim = list(algorithm = "NEWTON"))
  ZIPLN_param(config_optim = list(algorithm = "NEWTON"))

PLN (`nlopt_optimize_newton`, `nlopt_optimize_vestep_newton`) :
- Newton diagonal pour B et M avec backtracking Armijo
- Fixed-point exact pour logS : logS* = -0.5 log(Omega_jj + A_ij)
- Guard overflow élément par élément : logS <= 0.5 log(max(1, 700-Z))
- Gains sur oaks : +38 loglik, 36% plus rapide que CCSAQ

ZIPLN (`optim_zipln_M_S_newton`) :
- Même approche, hessien diagonal ajusté pour le masque ZI
- Défaut : CCSAQ conservé (Newton donne de mauvais optima locaux sur
  données très sparse)
- Guard de covariance "covar" : substitue CCSAQ avant d'appeler NLopt

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jchiquet jchiquet marked this pull request as ready for review June 15, 2026 09:07
@jchiquet jchiquet merged commit e257f76 into master Jun 15, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant