Skip to content

fix: keep overhanging tools, detect paper on dark backgrounds, mask every object - #226

Merged
jasonmadigan merged 2 commits into
mainfrom
worktree-triage-fixes
Sep 17, 2026
Merged

jasonmadigan merged 2 commits into
mainfrom
worktree-triage-fixes

Conversation

@jasonmadigan

@jasonmadigan jasonmadigan commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes for the September triage in one PR: #212, #213, #223, and the #214 cleanup.

#212 tools crossing the sheet edge (ai_tracer.py). Local and remote saliency run on the paper crop from _detect_paper_rect, and everything outside the crop was background. Two changes:

  • _detect_paper_rect merges bright fragments of the same tone that line up with the largest one across a gap of at most 25% of that fragment's long side, so a tool splitting the sheet no longer leaves the crop on one half.
  • _saliency_within grows the crop by 25% on any side the mask touches and runs again, at most twice, so an overhanging tool is kept whole. A grown attempt whose foreground balloons past 4x the previous one is treated as saliency flipping onto the sheet and the previous crop wins. Remote providers can therefore receive a crop larger than the paper, in up to three requests; README says so.

Gemini does not crop and is unchanged.

#213 corners on a dark background (image_processor.py). A U2-Net mask over TOOL_MASK_MAX_FRACTION (40%) of the frame is treated as the sheet and ignored, and a masked miss retries unmasked instead of returning no corners.

#223 mask prompt (ai_tracer.py). MASK_PROMPT_PRO asks for every object, and prompt choice uses _STENCIL_PROMPT_MODELS rather than _NEEDS_ALIGNMENT. TRACERS auto-detect behaviour documented in README and docs/usage/tracing.md. The effect on Gemini output is not verified here, only the wording.

#214 finger hole rebuilds. FingerHole rebuilds use model_copy, ScaledFingerHole construction goes through from_finger_hole, and drawer_service.py, which had no callers, is deleted. On sync a hole the placement has not seen yet keeps the library depth_override instead of None.

User-visible behaviour changes: overhanging and sheet-crossing tools trace correctly with local and remote tracers; dark-background photos get detected corners; Gemini prompt wording.

Known limits: a same-tone white surface beside the sheet can still be merged into the crop; a masked corner miss now retries unmasked, which can return a wrong quad rather than none (corners are shown and draggable).

Verification:

  • venv/bin/python -m pytest: 804 passed (22 new tests across five files)
  • ruff check backend/: clean
  • end to end on the Tools that touch or cross the sheet edge may be traced incorrectly #212 photo at the app's 2048 upload size with real U2-Net and isnet: corners detected, paper rect is exactly the sheet, final mask bbox matches an uncropped run, one polygon traced
  • frontend not touched, frontend suite not run; e2e via CI

…very object

Saliency tracers ran on a crop clipped to the paper rect, so a tool
overhanging the sheet came back cut flat at the paper edge, and a tool
crossing the sheet split the bright region so the crop kept only the
larger half and the tracer masked the printed ruling. The crop now merges
sheet fragments of the same tone that line up across a tool-sized gap,
and grows on any side the mask touches before running again. Gemini does
not crop and is unchanged.

Paper corner detection blacked out the U2-Net mask before looking for the
sheet, but on a dark background the sheet itself is the salient object,
so the upload came back with no corners. A mask over 40% of the frame is
now ignored and a masked miss retries unmasked.

The mask prompt for dimension-accurate Gemini models asked for one tool
while the contour tracer emits a polygon per object; it now asks for
every object, and prompt choice no longer piggybacks on the alignment
set. TRACERS auto-detect behaviour is documented.

Finger holes were rebuilt with hand-written argument lists at six sites,
so a field survived only where someone remembered it. Rebuilds now go
through model_copy or ScaledFingerHole.from_finger_hole, a test walks
FingerHole.model_fields, and the never-called drawer_service module is
removed.

Closes #212
Closes #213
Closes #223
Closes #214

Signed-off-by: Jason Madigan <jason@jasonmadigan.com>
A grown crop shows more background, which can flip saliency onto the
sheet; that mask touches every edge and would have won unconditionally.
A grown attempt whose foreground exceeds four times the previous one now
loses to the previous crop. The fragment merge gap is fixed to the largest
fragment's long side so chained merges cannot walk across the frame.
Contour points are cast to int32 before drawContours. A library hole not
yet on a placement keeps the library depth_override on sync. README says
the grown crop reaches remote providers as extra requests.

Signed-off-by: Jason Madigan <jason@jasonmadigan.com>
@jasonmadigan
jasonmadigan merged commit 9271bdc into main Sep 17, 2026
6 checks passed
@jasonmadigan
jasonmadigan deleted the worktree-triage-fixes branch September 17, 2026 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant