Repository navigation
fix(cascade): seed a delete with its keys when its restriction is another query, and keep a % in it - #1611
fix(cascade): seed a delete with its keys when its restriction is another query, and keep a % in it#1611dimitri-yatsenko wants to merge 3 commits into
Conversation
MilagrosMarin
left a comment
There was a problem hiding this comment.
Thanks @dimitri-yatsenko — I traced the % round trip and it holds up: restrict() appends make_condition() output, which doubles every % (condition.py), so the seed's conditions really are compiled SQL and undoing one escape before the cascade re-applies them is symmetric. _has_subquery also fails safe — a false positive falls back to keys(), which selects the same rows. I haven't run the suite locally.
Requested change
✅ test_seed_condition_with_a_percent_sign doesn't assert what its data implies — see the inline comment. The loop carries an expected label_id per case but never checks it, so the test pins the row count, not which row matched.
Minor
_has_subquery (diagram.py:163) strips single- and double-quoted text, so a PostgreSQL identifier "select" is handled, but a MySQL backtick identifier `select` isn't — the two backends differ here. The failure mode is safe (falls back to keys()), so this is just tightening:
text = re.sub(r"'[^']*'|\"[^\"]*\"|`[^`]*`", "", text) # quoted text and identifiers
Suggestion
This widens seed materialization from dj.Top (rare) to any seed restricted by a query (common — (Subject & (Session & "quality = 0")).delete()). keys() pulls every selected primary key into Python and compiles it back to an OR list, so a large delete now carries that cost where it previously didn't. I haven't measured it; worth a line in the _seed_restriction docstring or the delete() docs so the cost is discoverable.
Before merge
- On head
e9e1d67I see no workflow runs or check-runs and the status ispending. - Base is
fix/cascade-overdelete, so this retargets tomasterwhen #1592 lands; onmaint/2.3, #1593 goes before #1612 as you noted.
The subquery detection, the Top/subquery branch ordering, and the three parametrized seed cases all read correct to me. Let me know if anything needs adjusting!
| (Label & {"name": "50%"}, 1), | ||
| (Label & "name = '5%%'", 3), | ||
| ]: | ||
| assert len(seed) == 1 |
There was a problem hiding this comment.
| assert len(seed) == 1 | |
| assert seed.to_arrays("label_id").tolist() == [label] |
label is bound at line 263 but never used, so each case only asserts that exactly one row matched — not that it matched the right one. If name = '5%%' selected label 1 instead of label 3, this would still pass. The test does catch the original bug (double-escaping makes len(seed) 0), but it's weaker than the data suggests.
Ruff won't flag the unused binding either: select is ["E", "W", "F", "C90"], and pyflakes doesn't report unused loop variables — that's bugbear's B007, which isn't enabled.
A delete restricted by another query, such as `(Subject & (Session & "quality = 0")).delete()`, failed. The cascade applies the seed's compiled conditions again to a fresh copy of the table through `restrict()`, which collects column names from the whole condition text, subquery included, and rejects every name the seed table lacks. The delete raised `Attribute ... is not found in query` before deleting anything. On MySQL the backquoted schema name in every subquery tripped the same check, so even `(Subject & Session).delete()` raised. On PostgreSQL, a seed whose subquery names only the seed's columns got past the check but still did nothing. The cascade deletes children before parents, so by the time it reached the seed, the rows its subquery reads were gone. `(Subject & Session).delete()` matched no subject, returned 0 and rolled back. A seed whose conditions contain a subquery is now replaced by the primary keys it selects, as a `dj.Top` seed already is. The traversal and the delete then work from the rows the restriction selected when the delete started. The new tests delete subjects restricted by a query on their sessions, by its negation, and by the session table itself. They check the preview counts and the rows left afterwards, on both backends. All six cases fail on the previous code.
A delete restricted by a condition containing %, such as `(Label & "name = '50%'").delete()` or a dict `{"name": "50%"}`, deleted nothing and returned 0. The seed's conditions are compiled SQL, in which each % is already escaped as %% for the database driver. The cascade applies them again through `restrict()`, which escaped every % a second time, so the condition looked for the literal text `50%%` and matched no row. A LIKE pattern kept working only because `%%` and `%` match the same strings there.
The seed's conditions now have that first escape undone before the cascade applies them again, so they select the same rows as the restriction on its own.
The new test checks the preview counts for a string condition, a dict and a condition holding `%%`, and deletes with a LIKE pattern, on both backends. It fails before this change.
…sert the matched row Review follow-up. The subquery check strips quoted text before looking for SELECT; it now strips MySQL's backquoted identifiers too, as it already did PostgreSQL's double-quoted ones, so a column named `select` no longer makes a seed look like a subquery (that case was safe, since the seed's keys select the same rows). The percent-sign test now checks which row each seed selects, not only how many. The seed helper's docstring notes that replacing a seed by its keys reads them all into one condition.
e9e1d67 to
94d1105
Compare
|
Thanks, @MilagrosMarin. Pushed 94d1105.
This PR moves to 2.4. Unlike #1592, it fixes deletes that fail or do nothing, not ones that lose data, and there is a workaround: |
A delete restricted by another query, such as
(Subject & (Session & "quality = 0")).delete(), failed. The cascadeapplies the seed's compiled conditions again to a fresh copy of the table
through
restrict(), which collects column names from the whole conditiontext, subquery included, and rejects every name the seed table lacks. The
delete raised
Attribute ... is not found in querybefore deletinganything. On MySQL the backquoted schema name in every subquery tripped the
same check, so even
(Subject & Session).delete()raised.On PostgreSQL, a seed whose subquery names only the seed's columns got past
the check but still did nothing. The cascade deletes children before
parents, so by the time it reached the seed, the rows its subquery reads
were gone.
(Subject & Session).delete()matched no subject, returned 0and rolled back.
A seed whose conditions contain a subquery is now replaced by the primary
keys it selects, as a
dj.Topseed already is. The traversal and thedelete then work from the rows the restriction selected when the delete
started.
The new tests delete subjects restricted by a query on their sessions, by
its negation, and by the session table itself. They check the preview
counts and the rows left afterwards, on both backends. All six cases fail
on the previous code.
This PR is stacked on #1592 (it changes the seed helper that PR adds), so its base is
fix/cascade-overdelete; it moves tomasterwhen #1592 merges.Closes #1610.
Second commit: keep a % in a delete's restriction (#1636)
A delete restricted by a condition containing %, such as
(Label & "name = '50%'").delete()or a dict{"name": "50%"}, deleted nothing and returned 0. The seed's conditions are compiled SQL, in which each % is already escaped as %%; the cascade applies them again throughrestrict(), which escaped every % a second time. The seed's conditions now have that first escape undone before the cascade applies them again. The new test checks a string condition, a dict and a condition holding%%, on both backends, and fails before this change.Closes #1636.