Repository navigation
Simplify and harden the chunked MicroSSIM RI-factor objective - #54
Conversation
Replace three hand-written five-field SSIMElements rebuilds (chunk views, pooled buffers, test layouts) with _map_arrays, pass alpha to _chunked_mean instead of wrapping each call in a lambda, allocate the pooled buffers with np.empty_like rather than get_array_module, and drop the single-chunk .mean() branch and C1_last/C2_last locals that the loop already covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
Preallocating the pooled buffers turned an empty (0, H, W) stack into a TypeError from SSIMElements; the list + concatenate code raised ValueError. Check up front so the error names the problem. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
Use the shared gpu_available fixture and cubic.cuda.ascupy instead of importorskip + cp.asarray (CuPy without a device errored instead of skipping), factor the pool hook into a helper, and tighten the bound from 12 to 9 pooled arrays (measured 6.3 after the fix, 25.3 before). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
Chunks split only along axis 0, so a pool whose single slice exceeds _CHUNK_ELEMS (a (1, 4096, 4096) fit) was reduced whole and still built ~15 element-sized temporaries. Recurse into oversized indices so every chunk holds at most _CHUNK_ELEMS pixels for any layout, without copies. Measured on an A40, (1, 1024, 1024) float32 with a 2**16 chunk: objective temporaries 15.0 -> 0.94 element arrays, alpha unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
Sum chunks on the device and sync once per evaluation instead of once per chunk, upcasting each chunk with astype(float64) so the accumulation is float64 as documented. CuPy's sum(dtype=float64) on float32 measured ~13x slower per chunk (2.7 vs 0.2 ms) and doubled fit time, so the explicit upcast is deliberate. Measured on an A40, 576x640x960 float32: fit 50.6 s at 12.86 GB peak, alpha 1.3615524173 (unchanged). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
Reviewer's GuideThis PR hardens MicroSSIM RI-factor fitting against GPU memory spikes by simplifying pooled-element construction, enforcing bounded recursive chunking for all array layouts, and reducing synchronization overhead during chunked reductions, while adding regression and GPU peak-memory coverage. Sequence diagram for pooled MicroSSIM element constructionsequenceDiagram
participant Caller
participant Fit as get_global_ri_factor
participant Compute as compute_ssim_elements
participant Pool as Pooled SSIMElements
participant Solver as get_ri_factor
Caller->>Fit: get_global_ri_factor(gt, pred)
Fit->>Compute: compute_ssim_elements(gt[0], pred[0])
Compute-->>Fit: first slice elements
Fit->>Pool: np.empty_like arrays with shape (n, ...)
loop each slice
Fit->>Compute: compute_ssim_elements(gt[i], pred[i])
Compute-->>Fit: slice elements
Fit->>Pool: copy five element fields into pooled arrays
end
Fit->>Pool: dataclasses.replace(C1, C2)
Fit->>Solver: get_ri_factor(pooled elements)
Solver-->>Caller: RI factor
Flow diagram for bounded MicroSSIM RI-factor reductionflowchart TD
E[SSIMElements] --> C{Chunk exceeds limit?}
C -->|No| S[Split along axis 0]
C -->|Yes| R[Recursively split each axis-0 item]
R --> S
S --> M[_S_map or _dS_map]
M --> D[astype float64 and sum on device]
D --> A[Accumulate chunk sums]
A --> H[One host sync per evaluation]
H --> O[Mean objective or derivative]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="cubic/metrics/microssim/ri_factor.py" line_range="114" />
<code_context>
- C2=elements.C2,
- )
+ n = elements.ux.shape[0]
+ row = elements.ux.size // n
+ if row > _CHUNK_ELEMS:
+ for i in range(n):
</code_context>
<issue_to_address>
**issue (bug_risk):** `_iter_chunks` raises `IndexError` when `get_ri_factor` receives scalar (0-D) SSIM element arrays because it accesses `elements.ux.shape[0]`; the removed single-chunk branch explicitly supported `elements.ux.ndim == 0`.
**Triggers:** When callers construct or receive `SSIMElements` whose fields are scalar arrays.
**Suggested fix:** Handle 0-D elements before indexing the first axis, for example by yielding the scalar elements as one chunk.
```suggestion
if elements.ux.ndim == 0:
yield elements
return
n = elements.ux.shape[0]
```
</issue_to_address>Sourcery assessment
Approval pending. 1 finding to address first.
Blocking findings: cubic/metrics/microssim/ri_factor.py:114
Dropping the single-chunk .mean() branch made _iter_chunks index axis 0 unconditionally, so get_ri_factor on scalar element arrays raised IndexError where it previously reduced them. Yield 0-D elements as one chunk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The implementation is coherent, device-agnostic, and thoroughly covers the corrected behavior.
Review effort: Balanced
Findings: None
What changed in this PR
Hardens MicroSSIM RI-factor fitting by bounding objective memory for arbitrary array layouts while preserving CPU/GPU behavior.
Changes:
- Adds recursive chunking and device-side float64 accumulation.
- Restores validation for empty stacks.
- Expands correctness and GPU peak-memory tests.
| File | Description |
|---|---|
cubic/metrics/microssim/ri_factor.py |
Refactors element mapping, chunking, accumulation, and empty-stack handling. |
tests/metrics/microssim/test_ri_factor.py |
Covers scalar layouts, oversized slices, empty stacks, and GPU memory bounds. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The chunked objective divides by the element row size and pixel count, so empty arrays such as (0,), (3, 0) or (0, 5, 5) raised a bare ZeroDivisionError from get_ri_factor. Before the chunking they reached the documented RuntimeError from bracketing a NaN objective. Reject them up front with a clear ValueError instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
…hunks The existing chunking test builds float64 elements, where the upcast is a no-op, so a revert to float32 chunk sums passed every CPU test. The new float32 case fails on that revert (0.47415616313 vs 0.47415615877 at rel 1e-12). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
cubic.cuda imports without CuPy, so the two GPU tests need no local import for it; only the cupy imports stay guarded inside the GPU-only helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
The loop computed slice 0 ahead of time and then skipped recomputing it through an `if i:` check on the index. Store slice 0 explicitly and loop over the remaining slices instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
get_global_ri_factor kept the last slice's five element arrays alive for the whole bisection although only their C1 / C2 floats are used, about 12 MB of dead CuPy pool per 640x960 float32 slice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
A bare "ri_factor.py:123-131" read as this file, where those lines are the chunk iterator; cite juglab/microssim as the module docstring does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
Ships the MicroSSIM RI-factor fit memory fix (f4ca22a, #54) so the dynacell eval environment, pinned to 0.9.0a1, can pick it up. Still a pre-release: the single-image FRC/FSC questions from 0.9.0a1 are open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
Follow-up to f4ca22a (MicroSSIM RI-factor fit GPU OOM), from a
/simplifypass. One concern per commit:_map_arrayshelper replaces three five-fieldSSIMElementsrebuilds;alphais passed to_chunked_meaninstead of wrapped in lambdas;np.empty_likereplacesget_array_module; the dead single-chunk branch andC1_last/C2_lastlocals are gone.ValueErroragain (preallocation had turned it into aTypeError).gpu_available+ascupy, a shared pool-hook helper, and a tighter bound (9 pooled arrays; measured 6.3 after, 25.3 before)._CHUNK_ELEMSare split recursively. Axis-0-only chunking reduced a(1, 4096, 4096)pool whole. Measured on an A40,(1, 1024, 1024)float32, 2**16 chunk: objective temporaries 15.0 -> 0.94 element arrays. The new test fails on f4ca22a.astype(float64).sum()is used deliberately: CuPy'ssum(dtype=float64)on float32 measured 2.7 vs 0.2 ms per 4M-element chunk and doubled fit time.Measurements (A40, synthetic 640x960 float32)
Alpha is identical to 10 decimals at N = 48, 192, 576.
Checks (local, on the final tree)
ruff check .andruff format --check .: cleanuv run mypy --ignore-missing-imports cubic/: no issues in 45 filespytest: 802 passed, 2 skipped (GPU tests ran on an A40)Not changed
MicroSSIM.fitstill holds full-stackgt_norm/pred_normcopies through the solve, about 22% of the remaining 12.86 GB peak. Normalizing per slice would need aget_global_ri_factorAPI change, so it is left for a separate PR.🤖 Generated with Claude Code
https://claude.ai/code/session_011hEQ1TZJUbmhYKcC14RYHT
Summary by Sourcery
Harden and optimize chunked MicroSSIM RI-factor fitting to reduce GPU memory usage while preserving numerical results and improving validation.
Bug Fixes:
Enhancements:
Tests: