Illustrative Case Study (Good Case)

astropy__astropy-12907

separability_matrix incorrect for nested CompoundModels — test retrieval anchors the correct entry, trace identifies the & combiner, and reranking pinpoints _cstack as the GT edit location.

← Back to Case Studies   |   Homepage

Outcome

Hit @1 (GT function)astropy/modeling/separable.py::_cstack
Core idea: the erroneous “bottom-right 2×2 all-True” block matches the ndarray branch behavior in _cstack, making symptom-level alignment decisive in reranking.

0) Metadata (Repro Anchors)

1) Input

Symptom. astropy.modeling.separable.separability_matrix behaves inconsistently on nested CompoundModels: flat composition preserves block-diagonal structure, while nested composition incorrectly makes two Linear1D models appear coupled.

Issue excerpt (original, abridged)
Modeling's separability_matrix does not compute separability correctly for nested CompoundModels

cm = Linear1D(10) & Linear1D(5)
separability_matrix(cm) -> diagonal (expected)

separability_matrix(Pix2Sky_TAN() & Linear1D(10) & Linear1D(5)) -> expected block structure

separability_matrix(Pix2Sky_TAN() & cm) -> bottom-right 2×2 becomes all True (unexpected)

2) Step 1 — Relevant Test Retrieval (Two-Phase)

BM25 retrieves many unrelated tests due to cross-module noise; the relevant separability tests are not top-ranked. The LLM then selects a coherent test set aligned with separability semantics.

BM25 Top-10 (lexical filtering)

The relevant test_separable appears only around top-23 in this instance, indicating vocabulary mismatch / noise.

astropy/table/tests/test_masked.py::TestTableInit::test_mask_false_if_input_mask_not_true
astropy/table/tests/test_masked.py::TestTableInit::test_mask_false_if_no_input_masked
astropy/modeling/tests/test_models.py::test_custom_separability_matrix
astropy/table/tests/test_table.py::Test__Astropy_Table__::test_simple_1
astropy/table/tests/test_init_table.py::test_init_and_ref_from_dict
astropy/modeling/tests/test_core.py::test_compound_model_with_bounding_box_true_and_single_output
astropy/table/tests/test_init_table.py::test_init_and_ref_from_multidim_ndarray
astropy/modeling/tests/test_core.py::test_model_with_bounding_box_true_and_single_output
astropy/utils/tests/test_diff.py::test_diff_values_false
astropy/io/votable/tests/table_test.py::TestVerifyOptions::test_pedantic_false
...
(astropy/modeling/tests/test_separable.py::test_separable ~ top-23)
LLM-selected Td (semantic selection)
astropy/modeling/tests/test_separable.py::test_separable
astropy/modeling/tests/test_separable.py::test_custom_model_separable
astropy/modeling/tests/test_separable.py::test_coord_matrix
astropy/modeling/tests/test_separable.py::test_cdot
astropy/modeling/tests/test_separable.py::test_cstack

Why test retrieval matters: the issue reports an incorrect boolean matrix pattern without a stacktrace or named function. Tests provide executable anchors that constrain where matrix assembly actually occurs.

3) Step 2 — Trace-Guided Analysis (Execution Causal Skeleton)

Trace-guided analysis narrows the search to the internal separability pipeline and surfaces the operator combiner for &.

Coverage / call graph (test → suspect chain)
astropy/modeling/tests/test_separable.py::test_separable →
  • astropy/modeling/separable.py::is_separable
  • astropy/modeling/separable.py::separability_matrix

astropy/modeling/separable.py::is_separable →
  • astropy/modeling/core.py::CompoundModel.n_inputs
  • astropy/modeling/core.py::CompoundModel.n_outputs
  • astropy/modeling/separable.py::_separable

astropy/modeling/separable.py::separability_matrix →
  • astropy/modeling/core.py::CompoundModel.n_inputs
  • astropy/modeling/core.py::CompoundModel.n_outputs
  • astropy/modeling/separable.py::_separable

astropy/modeling/separable.py::_separable →
  • astropy/modeling/separable.py::_separable
  • astropy/modeling/core.py::CompoundModel._calculate_separability_matrix
  • astropy/modeling/core.py::Mapping._calculate_separability_matrix
  • astropy/modeling/core.py::Polynomial2D._calculate_separability_matrix
  • astropy/modeling/core.py::Rotation2D._calculate_separability_matrix
  • astropy/modeling/core.py::Scale._calculate_separability_matrix
  • astropy/modeling/core.py::Shift._calculate_separability_matrix
  • astropy/modeling/mappings.py::Mapping.n_outputs
  • astropy/modeling/separable.py::_cdot
  • astropy/modeling/separable.py::_coord_matrix
  • astropy/modeling/separable.py::_cstack

Suspicious locations (trace-guided). Candidates include _separable (dispatch) and _cstack (combiner for &).

4) Step 3 — Contextual Refinement

IssueExec retrieves structure around astropy/modeling/separable.py so reranking can distinguish the router layer (_separable) from the matrix assembly logic (_cstack).

5) Step 4 — Final Reranking (Ranked Edit Locations)

Candidate contexts (abridged)

Candidate 1: _separable (dispatch)

elif isinstance(transform, CompoundModel):
    sepleft = _separable(transform.left)
    sepright = _separable(transform.right)
    return _operators[transform.op](sepleft, sepright)

Candidate 2 (GT): _cstack (combiner for &)

if isinstance(right, Model):
    cright = _coord_matrix(right, 'right', noutp)
else:
    cright = np.zeros((noutp, right.shape[1]))
    cright[-right.shape[0]:, -right.shape[1]:] = 1

Final ranked edit locations L* (Top-k).

  1. astropy/modeling/separable.py::_cstack (GT)
  2. astropy/modeling/separable.py::_separable

Symptom-level alignment: in nested compounds, right often becomes an ndarray via recursion; the ndarray branch in _cstack fills the corresponding block with 1 (all True), exactly matching the erroneous bottom-right 2×2 all-True pattern.

6) Baseline (SWE-agent) — Failure Summary

Key contrast: IssueExec reuses existing tests and (offline) traces, avoiding brittle “must build everything inside the agent” dependency.

7) Step 5 — Patch Generation

With astropy/modeling/separable.py::_cstack identified as the Top-1 edit location, we apply a targeted fix to the ndarray branch responsible for composing separability blocks under the & operator. The goal is to preserve dependency structure when right is already a matrix produced by nested recursion, instead of collapsing it into an all-True block.

In this instance, nested CompoundModel composition turns the right side into an ndarray that already encodes internal separability. Filling the bottom-right block with ones erases that structure, producing the reported “bottom-right 2×2 all-True” artifact. The patch copies the matrix into the corresponding block to retain the correct pattern.

Resolved patch (at _cstack)
diff --git a/astropy/modeling/separable.py b/astropy/modeling/separable.py
--- a/astropy/modeling/separable.py
+++ b/astropy/modeling/separable.py
@@ -242,7 +242,12 @@ def _cstack(left, right):
         cright = _coord_matrix(right, 'right', noutp)
     else:
         cright = np.zeros((noutp, right.shape[1]))
-        cright[-right.shape[0]:, -right.shape[1]:] = 1
+        # When `right` is already a matrix (e.g. from nested CompoundModel recursion),
+        # preserve its internal dependency structure instead of filling with ones.
+        r0, r1 = right.shape
+        block = cright[-r0:, -r1:]
+        np.copyto(block, right, casting="no")

     return np.hstack([cleft, cright])

Outcome: the fix removes the spurious all-True coupling in nested compositions and resolves the instance under the SWE-bench Lite setup.

8) Takeaway

This is the canonical win: tests anchor the semantics, traces isolate the & combiner, and reranking pinpoints the exact faulty branch in _cstack based on the observed matrix-pattern symptom.