perf(tags): bulk-propagate inherited tags + gate child post_save on create#14812
Open
valentijnscholten wants to merge 1 commit intoDefectDojo:devfrom
Open
perf(tags): bulk-propagate inherited tags + gate child post_save on create#14812valentijnscholten wants to merge 1 commit intoDefectDojo:devfrom
valentijnscholten wants to merge 1 commit intoDefectDojo:devfrom
Conversation
9015ac8 to
64b0287
Compare
Maffooch
approved these changes
May 6, 2026
64b0287 to
d2c4fc5
Compare
blakeaowens
approved these changes
May 7, 2026
Jino-T
approved these changes
May 7, 2026
…reate Replaces the per-row `.save()` loop in `propagate_tags_on_product_sync` with bulk SQL through the existing tag-utils helpers. For every child model (Engagement/Test/Finding/Endpoint/Location), reads current inherited_tags in one query, computes the per-child diff against the Product's tags, and applies adds/removes via `bulk_add_tag_mapping` and the new `bulk_remove_tags_from_instances` helper. Both `tags` and `inherited_tags` fields are kept in sync. Also gates the per-child `inherit_tags_on_instance` post_save handler on `created=True`. The previous behavior fired on every save (create OR update), repeatedly re-applying inherited tags to children whose tag state had not changed. Sticky enforcement on user-driven tag edits is unchanged (still handled by `make_inherited_tags_sticky` on m2m_changed). Pinned query-count baselines from PR DefectDojo#14811 drop accordingly: product_tag_add -> 100 findings : 4758 -> 91 (~52x fewer queries) product_tag_remove -> 100 findings : 4540 -> 53 (~85x fewer queries) Sticky and child-creation paths are unchanged in this PR. Phase B targets those (centralized inheritance module + drop the duplicate `inherited_tags` TagField).
d2c4fc5 to
bd62203
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is going to be multi stage optimization to keep the PRs a little smaller to ease review.
This PR basically adds another batch tag method to apply inherited tags more efficiently.
Summary
obj.save()loop indojo/product/helpers.py:propagate_tags_on_product_syncwith bulk SQL through the existing tag-utils helpers. For every child model (Engagement / Test / Finding / Endpoint / Location), reads currentinherited_tagsin one query, computes the per-child diff against the Product's tags, and applies adds/removes viabulk_add_tag_mappingand the newbulk_remove_tags_from_instanceshelper. Bothtagsandinherited_tagsfields are kept in sync.bulk_remove_tags_from_instancestodojo/tag_utils.py, symmetric tobulk_add_tags_to_instances. Decrements Tagulous tag counts to avoid drift, deletes through-model rows in one DELETE per tag-batch, invalidates prefetch caches.inherit_tags_on_instancepost_save handler oncreated=True. The previous behavior fired on every save (create OR update), repeatedly re-applying inherited tags to children whose tag state had not changed. Sticky enforcement on user-driven tag edits is unchanged (still handled bymake_inherited_tags_stickyonm2m_changed).unittests/test_tag_inheritance_perf.pyto reflect the reductions.Query-count impact
Sticky and child-creation paths are unchanged in this PR. Phase B targets those (centralized inheritance module + drop the duplicate
inherited_tagsTagField).Verification
unittests.test_tag_inheritance- 36 tests pass (5 skipped for v3 feature flag)unittests.test_tag_utils_bulk- 29 tests passunittests.test_tag_inheritance_perf- 6 tests pass with the lowered pinsBehavior preserved
inherited_tagsfield semantics unchanged - still a duplicateTagFieldkept in lock-step withtags. Phase B replaces it with a JSON column.make_inherited_tags_stickym2m_changed handler.