Implement skeleton display - #120
Draft
C-Achard wants to merge 18 commits into
Draft
Conversation
This was referenced Aug 27, 2026
C-Achard
added
enhancement
Contributor
There was a problem hiding this comment.
Pull request overview
Adds configurable skeleton overlays to the main preview and centralizes overlay rendering.
Changes:
- Adds skeleton configuration, topology resolution, and rendering.
- Propagates pose metadata from DLCLive and adds GUI controls.
- Adds broad unit and GUI coverage for the new pipeline.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
dlclivegui/config.py |
Adds skeleton visualization settings. |
dlclivegui/display/display.py |
Generates keypoint colors. |
dlclivegui/display/overlays.py |
Composes and caches overlays. |
dlclivegui/gui/main_window.py |
Integrates skeleton controls and rendering. |
dlclivegui/gui/misc/color_dropdowns.py |
Adds skeleton color selection. |
dlclivegui/gui/qt_display/utils.py |
Converts frames to Qt pixmaps. |
dlclivegui/services/dlc_processor.py |
Attaches skeleton metadata to poses. |
tests/display/conftest.py |
Adds a frame factory fixture. |
tests/display/test_display.py |
Tests keypoint color generation. |
tests/display/test_draw.py |
Tests drawing and skeleton rendering. |
tests/display/test_overlays.py |
Tests overlay composition and caching. |
tests/display/test_skeleton.py |
Tests topology resolution. |
tests/gui/main_window/test_main_display.py |
Tests main-window integration. |
tests/gui/ui_blocks/test_ui_blocks.py |
Tests skeleton color controls. |
tests/services/test_dlc_processor.py |
Tests metadata extraction. |
tests/test_skeleton_config.py |
Tests configuration defaults and validation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+675
to
+678
| raw_bodyparts = cfg.get("bodyparts") | ||
| if not isinstance(raw_bodyparts, (list, tuple)): | ||
| logger.info("DLCLive configuration does not define bodyparts.") | ||
| return |
C-Achard
force-pushed
the
cy/skeleton-final-wiring
branch
from
August 28, 2026 13:11
d41045f to
50932d3
Compare
Add internal state for keypoint names and skeleton definitions, plus a helper that extracts this metadata from `dlc.cfg` after initialization. The loader validates config shape (bodyparts and skeleton edges), rejects duplicate bodypart names and unknown edge references, and derives a stable skeleton ID from the DLC task name.
DLCLiveProcessor now includes keypoint names, skeleton ID, and skeleton edges in emitted PosePacket data, and loads pose metadata from the DLC config during initialization. In the GUI, pose callbacks now refresh a cached resolved skeleton based on packet metadata, avoid repeated resolution work with a signature check, and deduplicate warning logs when skeleton resolution fails.
Introduce a new `display.overlays` module with typed dataclass settings for pose and bounding-box overlays, plus a `render_overlays` function that composes drawing calls on a copied frame. Also add a reusable `DisplayTransform` dataclass in `display.py` to centralize point mapping with offset and scale.
Move main window frame rendering to shared display utilities by building `OverlaySettings`, calling `render_overlays`, and converting frames with a new `frame_to_pixmap` helper. Add `SkeletonMetadata`/`SkeletonResolver` caching support and an `OverlayRenderer` runtime-state container to prepare for stable skeleton resolution across frames. Also fix module path header comments in display utilities.
Cleans up display-layer dead code by removing `DisplayTransform`, `OverlayRenderer`, and the cached `SkeletonResolver`/`SkeletonMetadata` path that was no longer used. `OverlaySettings` now explicitly includes skeleton settings, `render_overlays` uses a clearer `overlay_settings` argument name, and the main window stops calling the obsolete skeleton refresh hook in the pose callback.
Introduce an `OverlayRenderer` class to centralize pose, skeleton, and bounding-box composition with runtime state for skeleton resolution caching and warning deduplication. The renderer now resolves packet skeleton metadata on demand, surfaces resolution/render warnings once, and supports gradient skeleton coloring via a new `keypoint_colors_bgr` helper in `display.py`.
Refactors overlay drawing to use `OverlayRenderer` end-to-end in the main window, including pose, skeleton, and bounding-box settings from UI state. Adds skeleton controls (visibility, color mode/color, thickness), wires them into visualization/config sync, and surfaces renderer warnings in logs/status bar with deduped messaging. Also resets cached overlay runtime state on DLC/session resets and clears pose metadata fields before reloading DLC config to prevent stale skeleton/keypoint metadata from leaking between runs.
Extend visualization config with skeleton-related defaults (`show_pose`, `show_skeleton`, and `skeleton_style`) and tighten bbox color typing to `BGR`. Add reusable skeleton color combo helpers that support both solid colors and a keypoint-gradient mode, including gradient swatch icon rendering, combo population, and style get/set mapping.
Remove the unused `visible` field from `SkeletonStyle` and clear the overlay renderer runtime state during live-session reset. This keeps runtime overlay data from leaking across resets and aligns skeleton styling with the current rendering flow.
Introduce a new test module covering VisualizationSettings and SkeletonStyle behavior, including default/legacy config handling, model round-tripping, validation bounds for thickness and gradient steps, and effective thickness scaling behavior with and without zoom scaling.
Renames `tests/utils/test_display.py` to `tests/display/test_display.py` to keep display-related tests grouped under the display test module. No test logic was changed.
Expand unit coverage in display-related tests by adding assertions for `keypoint_colors_bgr` behavior (requested count, zero/one cases, determinism, and negative input validation). Add a new `test_skeleton.py` suite that validates skeleton name-to-index resolution, duplicate/missing keypoint handling, and packet-based skeleton topology resolution when keypoint names or edges are absent or provided.
Refactors display unit tests by moving bbox/keypoint/pose and most skeleton rendering coverage into a new `tests/display/test_draw.py`, while keeping tiling/color tests in `test_display.py`. Adds a shared `test_frame` fixture in `tests/display/conftest.py` to remove duplicated frame helpers and standardize test data creation. Also extends skeleton tests in `test_display.py` with gradient-mode behavior checks, including missing color handling and successful gradient rendering.
Introduces a new `tests/display/test_overlays.py` suite covering overlay rendering behavior, including frame immutability, skeleton metadata warning deduplication/reset, skeleton-only rendering, and skeleton resolution caching. Refactors the shared `test_frame` fixture into a typed frame-factory callable for clearer test setup, and moves skeleton gradient rendering assertions from `test_display.py` into `test_draw.py` to keep skeleton drawing coverage consolidated.
Introduce a new test module for main window display behavior focused on skeleton overlays. The tests cover default skeleton UI values, restoring skeleton settings from config, collecting visualization settings from UI, delegating frame rendering to the overlay renderer, clearing overlay runtime state when inference starts, and an end-to-end renderer check that verifies skeleton lines are drawn without mutating the input frame.
Adds a new `tests/gui/ui_blocks/test_ui_blocks.py` suite to validate skeleton color dropdown behavior. The tests cover presence of the gradient option, round-tripping both gradient and solid styles, fallback to the default solid color when an unknown color is provided, and icon creation when enabled.
Expand `test_dlc_processor.py` with focused unit tests for `_load_pose_metadata_from_dlc_config()`. The new cases verify successful loading of bodyparts/skeleton/task metadata, clearing of stale metadata when config is empty, support for bodyparts without skeleton data, and validation failures for duplicate bodyparts and skeleton links to unknown bodyparts.
Keep a dedicated `SkeletonStyle` state on the main window and sync it when loading visualization settings, so non-UI-exposed fields (`gradient_steps`, `scale_with_zoom`) are preserved instead of being pulled from stale config data. The UI-to-settings path now updates and reuses this state consistently. Also renames `test_frame` factory parameters in display tests (`height/width/channels` -> `h/w/c`) to match current call patterns.
C-Achard
force-pushed
the
cy/skeleton-final-wiring
branch
from
August 28, 2026 13:35
50932d3 to
9fb68bf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds skeleton display to main window and refactors main window preview rendering.