refactor(poller): resilient dual-stack map generation - #111
somethingwithproof wants to merge 7 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Refactors the gpsmap poller workflow to load the mapped Device set once per poller cycle and reuse it across subnet renders, significantly reducing repeated DB queries and DNS lookups. It also hardens upgrade/version persistence behavior, makes artefact publication behavior explicit on failure vs empty estates, improves atomic artefact writes, and unifies icon validation between the UI dropdown and the renderers.
Changes:
- Split region processing into
gpsmap_load_devices()(single load) andgpsmap_render_region()(per-subnet render), with prefix enumeration derived from already-resolved addresses. - Add/extend tests and coverage targets for polling + upgrade behavior, and add new upgrade-path regression tests.
- Move/centralize icon enumeration logic into
gpsmap_security.phpand align it with renderer constraints.
Reviewed changes
Copilot reviewed 15 out of 16 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_upgrade.php | New regression tests for upgrade gating, version persistence, and backoff behavior. |
| tests/test_polling.php | Adds tests for single-pass loading, atomic writes, publication decisions, and direct poller entry point coverage. |
| tests/test_icons.php | Adds tests ensuring getIcons() matches renderer constraints and respects base_path. |
| tests/run.php | Adds test_upgrade.php to the test runner. |
| tests/harness.php | Ensures test tmp root symlinks include additional plugin entrypoints needed by new tests. |
| tests/coverage.php | Expands coverage gate to include polling + upgrade logic; removes deleted entry point mention. |
| setup.php | Passes persisted old-version into gpsmap_upgrade_database() instead of relying on an unset global. |
| print.php | Clarifies authentication control flow in comment. |
| includes/towerSelect.php | Removes unused/unreferenced legacy entry point. |
| includes/setup/database.php | Makes upgrade path deterministic (version persisted, retry backoff, schema verification) and adjusts table create details. |
| includes/polling/processregion.php | Introduces load/render split, prefix derivation, and device state reset between renders. |
| includes/polling/functions.php | Makes artefact writes staged+atomic while preserving destination file mode. |
| includes/polling.php | Refactors poller entry to single-load per cycle + explicit publish/withhold decisions and stats logging. |
| gpstemplates.php | Uses shared getIcons() (now in gpsmap_security.php) and escapes edit URL. |
| gpsmap_security.php | Adds getIcons() implementation consistent with renderer constraints and base_path. |
| .gitignore | Ignores local debug file under tests. |
Suppressed comments (1)
tests/test_polling.php:341
- This base_path override is immediately restored without any assertion or code that depends on it, so it has no effect (and can confuse future readers). It should be removed or turned into a real assertion.
$savedRoot = $GLOBALS['config']['base_path'];
$GLOBALS['config']['base_path'] = $root . '/no-such-root';
$GLOBALS['config']['base_path'] = $savedRoot;
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
bc05aa7 to
3fdb4a8
Compare
54b80ce to
c629293
Compare
…ubnet region() re-queried every Device and re-resolved every hostname on each call, and the poller called it once per discovered subnet prefix. On a 2000-Device install across 150 subnets that was 174 table scans and roughly 348,000 DNS lookups per cycle. Split into gpsmap_load_devices() and gpsmap_render_region() so the load runs once and every subnet renders from that set. A failed Device query withholds publication, because overwriting the artefacts with an empty set would blank the map; an estate with genuinely no mapped Devices still publishes, so a new install gets an all.xml rather than a 404. Artefact writes are staged and renamed, preserving the destination mode, so a reader never sees a truncated or unreadable document. Closes Cacti#6 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
$old was read from an unset global, so every migration re-ran on each version change. Two ALTER statements backticked a literal default, which MySQL reads as an identifier, so they had been failing silently. And the version was recorded before the migrations ran, so a failed ALTER left the schema behind while the plugin reported itself current. The schema is now verified with db_column_exists() rather than trusting helper return values, both version records are written together only on success, and a failure backs off instead of re-running an ALTER on host from every page view. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
getIcons() accepted my-icon.png while the JavaScript emitter rejected it, so the icon appeared in the Map Template dropdown, saved cleanly, and then never rendered. Both sides now use gpsmap_icon_identifier(), the path is built from base_path so poller and CLI callers resolve it, and a missing icon directory no longer prints a PHP warning into the Map Templates form. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Nothing in the plugin calls it, so it was an authenticated route to host_template that no page uses. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
c629293 to
8a221fb
Compare
|
Superseded by #121, which ports these changes onto the Cacti repo directly (resolving the merge conflicts from the CI harmonization work in #119/#120) so they can be reviewed and merged. All credit for the implementation remains with @somethingwithproof — thank you for the contribution! |
The CI failures traced back to two problems introduced while porting PR #111 onto the harmonized Pest suite: - tests/bootstrap-unit.php only symlinked class/ and includes/ into the scratch Cacti root, so setup.php's own INFO/gpsmap_security.php reads (via $config['base_path']) failed in the upgrade lifecycle probe. It also lacked db_table_exists(), api_plugin_is_enabled(), cacti_escapeshellarg(), exec_background(), register_process_start()/ unregister_process(), and an enhanced db_column_exists()/ db_fetch_assoc_prepared()/db_execute_prepared() the refactored runtime code (DNS cache, thold integration) now depends on. - tests/Unit/FunctionsTest.php, tests/Security/{SecurityTest,IconsTest}.php and tests/Integration/PollingTest.php were carried over from before PR #111, but that PR also substantially rewrote the fork's own tests/test_*.php to match its new subnet validation, DNS caching, IPv6 drilldown, thold integration and atomic-write/pruning behavior. Ported those updates in, using the fork's own validated assertions. - PHPUnit's failOnWarning does not honor "@" suppression on the expected-failure paths several tests exercise (an unwritable directory, a missing icon folder, a rename onto an existing directory). Added tests/bootstrap-unit.php's gpsmap_test_silence() helper, which installs a temporary swallow-all error handler for the duration of those specific calls, and used it in place of "@".
* refactor(poller): load the Device set once per cycle instead of per subnet region() re-queried every Device and re-resolved every hostname on each call, and the poller called it once per discovered subnet prefix. On a 2000-Device install across 150 subnets that was 174 table scans and roughly 348,000 DNS lookups per cycle. Split into gpsmap_load_devices() and gpsmap_render_region() so the load runs once and every subnet renders from that set. A failed Device query withholds publication, because overwriting the artefacts with an empty set would blank the map; an estate with genuinely no mapped Devices still publishes, so a new install gets an all.xml rather than a 404. Artefact writes are staged and renamed, preserving the destination mode, so a reader never sees a truncated or unreadable document. Closes #6 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(upgrade): record the version only after the schema work succeeds $old was read from an unset global, so every migration re-ran on each version change. Two ALTER statements backticked a literal default, which MySQL reads as an identifier, so they had been failing silently. And the version was recorded before the migrations ran, so a failed ALTER left the schema behind while the plugin reported itself current. The schema is now verified with db_column_exists() rather than trusting helper return values, both version records are written together only on success, and a failure backs off instead of re-running an ALTER on host from every page view. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(icons): offer only names the map can draw getIcons() accepted my-icon.png while the JavaScript emitter rejected it, so the icon appeared in the Map Template dropdown, saved cleanly, and then never rendered. Both sides now use gpsmap_icon_identifier(), the path is built from base_path so poller and CLI callers resolve it, and a missing icon directory no longer prints a PHP warning into the Map Templates form. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * chore: remove the unreferenced towerSelect endpoint Nothing in the plugin calls it, so it was an authenticated route to host_template that no page uses. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * feat(poller): add resilient dual-stack map generation Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * style: apply current Cacti PHP formatting Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(poller): preserve maps while DNS cache warms Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: align ported tests with the poller refactor's actual behavior The CI failures traced back to two problems introduced while porting PR #111 onto the harmonized Pest suite: - tests/bootstrap-unit.php only symlinked class/ and includes/ into the scratch Cacti root, so setup.php's own INFO/gpsmap_security.php reads (via $config['base_path']) failed in the upgrade lifecycle probe. It also lacked db_table_exists(), api_plugin_is_enabled(), cacti_escapeshellarg(), exec_background(), register_process_start()/ unregister_process(), and an enhanced db_column_exists()/ db_fetch_assoc_prepared()/db_execute_prepared() the refactored runtime code (DNS cache, thold integration) now depends on. - tests/Unit/FunctionsTest.php, tests/Security/{SecurityTest,IconsTest}.php and tests/Integration/PollingTest.php were carried over from before PR #111, but that PR also substantially rewrote the fork's own tests/test_*.php to match its new subnet validation, DNS caching, IPv6 drilldown, thold integration and atomic-write/pruning behavior. Ported those updates in, using the fork's own validated assertions. - PHPUnit's failOnWarning does not honor "@" suppression on the expected-failure paths several tests exercise (an unwritable directory, a missing icon folder, a rename onto an existing directory). Added tests/bootstrap-unit.php's gpsmap_test_silence() helper, which installs a temporary swallow-all error handler for the duration of those specific calls, and used it in place of "@". * fix: gpsmap_test_icons() must remove directory fixture entries recursively A prior test can leave a directory (e.g. Nested.png) in the icon fixture folder to exercise 'hides directories with image-like names'. The next test's gpsmap_test_icons() call then tried to unlink() it, which fails on a directory, and PHPUnit's failOnWarning does not tolerate that even when suppressed with '@'. Use gpsmap_test_rmtree() instead, which already handles files, symlinks and directories. * fix: shared set_config_option() stub must persist into gpsmap_stub_settings It was a no-op, so read_config_option() (which reads gpsmap_stub_settings) never saw values written by the real source (e.g. plugin_gpsmap_dns_last_success in includes/dns.php), failing 'dns worker: a completed worker records liveness' and similar assertions in PollingTest. Matches the isolated probes' own set_config_option() overrides in UpgradeTest/DnsCliTest. * ci: warm gpsmap DNS worker before poller run to avoid cold-start warning * ci: bump actions/checkout and actions/upload-artifact to node24 releases --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: Thomas Vincent <thomasvincent@gmail.com>
Summary
The poller now loads the Device set once and publishes resilient dual-stack snapshots:
/16,/32,/48IPv6 drill-downalertand uses their Down iconIssue coverage and overlap
This PR is the single implementation PR for the related poller concerns. No separate PR duplicates these changes:
PR #102 changes workflows only and does not overlap these runtime changes.
Linux Docker validation
git diff --check: cleanThe repository has no
composer.json; none was created or edited.