Repository navigation
fix: keep overhanging tools, detect paper on dark backgrounds, mask every object - #226
Merged
Merged
Conversation
…very object Saliency tracers ran on a crop clipped to the paper rect, so a tool overhanging the sheet came back cut flat at the paper edge, and a tool crossing the sheet split the bright region so the crop kept only the larger half and the tracer masked the printed ruling. The crop now merges sheet fragments of the same tone that line up across a tool-sized gap, and grows on any side the mask touches before running again. Gemini does not crop and is unchanged. Paper corner detection blacked out the U2-Net mask before looking for the sheet, but on a dark background the sheet itself is the salient object, so the upload came back with no corners. A mask over 40% of the frame is now ignored and a masked miss retries unmasked. The mask prompt for dimension-accurate Gemini models asked for one tool while the contour tracer emits a polygon per object; it now asks for every object, and prompt choice no longer piggybacks on the alignment set. TRACERS auto-detect behaviour is documented. Finger holes were rebuilt with hand-written argument lists at six sites, so a field survived only where someone remembered it. Rebuilds now go through model_copy or ScaledFingerHole.from_finger_hole, a test walks FingerHole.model_fields, and the never-called drawer_service module is removed. Closes #212 Closes #213 Closes #223 Closes #214 Signed-off-by: Jason Madigan <jason@jasonmadigan.com>
A grown crop shows more background, which can flip saliency onto the sheet; that mask touches every edge and would have won unconditionally. A grown attempt whose foreground exceeds four times the previous one now loses to the previous crop. The fragment merge gap is fixed to the largest fragment's long side so chained merges cannot walk across the frame. Contour points are cast to int32 before drawContours. A library hole not yet on a placement keeps the library depth_override on sync. README says the grown crop reaches remote providers as extra requests. Signed-off-by: Jason Madigan <jason@jasonmadigan.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes for the September triage in one PR: #212, #213, #223, and the #214 cleanup.
#212 tools crossing the sheet edge (
ai_tracer.py). Local and remote saliency run on the paper crop from_detect_paper_rect, and everything outside the crop was background. Two changes:_detect_paper_rectmerges bright fragments of the same tone that line up with the largest one across a gap of at most 25% of that fragment's long side, so a tool splitting the sheet no longer leaves the crop on one half._saliency_withingrows the crop by 25% on any side the mask touches and runs again, at most twice, so an overhanging tool is kept whole. A grown attempt whose foreground balloons past 4x the previous one is treated as saliency flipping onto the sheet and the previous crop wins. Remote providers can therefore receive a crop larger than the paper, in up to three requests; README says so.Gemini does not crop and is unchanged.
#213 corners on a dark background (
image_processor.py). A U2-Net mask overTOOL_MASK_MAX_FRACTION(40%) of the frame is treated as the sheet and ignored, and a masked miss retries unmasked instead of returning no corners.#223 mask prompt (
ai_tracer.py).MASK_PROMPT_PROasks for every object, and prompt choice uses_STENCIL_PROMPT_MODELSrather than_NEEDS_ALIGNMENT.TRACERSauto-detect behaviour documented in README and docs/usage/tracing.md. The effect on Gemini output is not verified here, only the wording.#214 finger hole rebuilds.
FingerHolerebuilds usemodel_copy,ScaledFingerHoleconstruction goes throughfrom_finger_hole, anddrawer_service.py, which had no callers, is deleted. On sync a hole the placement has not seen yet keeps the librarydepth_overrideinstead of None.User-visible behaviour changes: overhanging and sheet-crossing tools trace correctly with local and remote tracers; dark-background photos get detected corners; Gemini prompt wording.
Known limits: a same-tone white surface beside the sheet can still be merged into the crop; a masked corner miss now retries unmasked, which can return a wrong quad rather than none (corners are shown and draggable).
Verification:
venv/bin/python -m pytest: 804 passed (22 new tests across five files)ruff check backend/: clean