Skip to content

Run an aggregate on the right of a data service @Where comparison as a subquery - #16835

Open
borinquenkid wants to merge 2 commits into
fix/h7-criteria-query-args-8.0.xfrom
fix/service-where-aggregate-subquery-8.0.x
Open

borinquenkid wants to merge 2 commits into
fix/h7-criteria-query-args-8.0.xfrom
fix/service-where-aggregate-subquery-8.0.x

Conversation

@borinquenkid

@borinquenkid borinquenkid commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Fixes #16807

@Where({ amount > avg(amount) }) in a data service generated select avg(amount) from t where amount > (select amount from t), which the database rejects ('Scalar subquery contains more than one row'). DetachedCriteriaTransformer translated the aggregate into a bare closure; inside a service the enclosing criteria resolved its projections call, so the aggregate landed on the outer query and the subquery lost it.

The aggregate is now built as an explicit new DetachedCriteria(X).build { projections { ... } }, the way the .of { } form already was. The property projection form that feeds the gtAll family is unchanged.

  • WhereServiceAggregateScenarioSpec runs >, >=, <, <=, == max, == min and a combination with a method argument on both the H5 and H7 example apps. All 7 fail on H7 without the change.
  • Verified: datamapping-core 1813, H5 core 1167, H7 core 4470 tests and both integration suites pass. Mongo, Neo4j and graphql are left to CI.

The issue was marked for Grails 9 on the assumption that it depends on the classic domain binder. It does not: the fix is in the shared AST transformer in grails-datamapping-core. Stacked on the chain below it. Its base is the head of #16823, the top of the #16816 chain (the earlier base, the closed #16825, was replaced by #16876, which has merged); the functional spec needs the ScenarioItem fixture from #16816, so this one stays in the stack.

https://claude.ai/code/session_01Dmki4SQGwRSj8pRJ5KCWyX

Copilot AI balanced review requested due to automatic review settings November 3, 2026 03:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

 comparison as a subquery

amount > avg(amount) in a @where closure of a data service was translated into
a bare closure that the enclosing criteria resolved its projections calls
against, so the aggregate landed on the outer query and the subquery lost it:
select avg(amount) from t where amount > (select amount from t). The aggregate
is now built as an explicit DetachedCriteria subquery, as the .of { } form
already was. WhereServiceAggregateScenarioSpec runs it on H5 and H7.

Fixes #16807

Claude-Session: https://claude.ai/code/session_01Dmki4SQGwRSj8pRJ5KCWyX
@borinquenkid
borinquenkid force-pushed the fix/service-where-aggregate-subquery-8.0.x branch from a4a1797 to d0eae47 Compare November 7, 2026 16:04
@borinquenkid
borinquenkid changed the base branch from fix/tck-where-ast-8.0.x to fix/h7-criteria-query-args-8.0.x November 7, 2026 16:04

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approving. An aggregate on the right of a comparison uses the declared type, not the runtime type.

// declared in a service (@Where), and the aggregate then lands on the outer query.
rightExpression = new MethodCallExpression(
new ConstructorCallExpression(getParameterizedDetachedCriteriaClassNode(null),
new ArgumentListExpression(new ClassExpression(this.currentClassNode))),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A helper typed as DetachedCriteria of Animal is called with a criteria of Cat, and the query is weight greater than avg(weight). The average includes every animal, not only the cats. Use the criteria that was passed in.

…taken over the criteria that runs it

The subquery of weight > avg(weight) was built from the class the code declares for the criteria. A
helper typed DetachedCriteria<Animal> and called with the criteria of a Cat therefore averaged every
animal, and min and max were taken over every animal as well, so cats were compared with the animals
of the other classes. The subquery is built from the class of the criteria the closure runs against,
read with the new getTargetClass(), which needs no datastore: a criteria built in memory, as the where
specs do, has no persistent entity to ask. It is the declared class wherever the two are the same, as
in a data service @where.

Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
@borinquenkid

Copy link
Copy Markdown
Member Author

@jamesfredley Reproduced, and fixed in the new commit. WhereAggregateHelperSpec has your case on Hibernate 5 and Hibernate 7: a helper typed DetachedCriteria<Animal> and called with the criteria of a Cat, where the animals weigh 1, 2 and 3 and the cats 10, 20 and 30. The average of every animal is 11 and that of the cats is 20; both rows fail without the change.

The transformer built the subquery of weight > avg(weight) from the class that the code declares for the criteria, so the helper averaged every animal and found cats 20 and 30 above it, where the cats alone have only 30. min and max were taken over the same rows: weight == min(weight) on the cats found no cat, because the lightest animal is 1. The subquery is built from the class of the criteria the closure runs against now, which is read when it runs through a new public getTargetClass(): asking the criteria for its persistent entity instead broke WhereQueryOperatorSpec, which builds its criteria in memory with no datastore. Where the declared class and the real one are the same, as in a data service @Where, nothing changes: the 7 scenarios of WhereServiceAggregateScenarioSpec pass on both example applications.

The Hibernate 7 module (4,631 tests), the Hibernate 5 module (1,184), grails-datamapping-core (1,877) and the in-memory TCK (706) pass, and so does codeStyle.

Branch policy: the new public types and throws that line-check lists came with the base branches and were already in this PR. This commit adds no type and no throw; it adds one public method, AbstractDetachedCriteria.getTargetClass(), which mirrors AbstractCriteriaBuilder.getTargetClass() and needs no datastore. Where the PR belongs is still open; I have not moved it.

If you find another shape that does not reproduce first time, a failing spec on a throwaway branch, or the few lines you ran, would help me a lot.

@matrei matrei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at d076adf (base fix/h7-criteria-query-args-8.0.x, #16823's head branch). This PR's own change is the 9 files GitHub shows, measured from 21573cb, the #16823 commit it last merged; #16823's head is 20510a5 now. The PR is labelled no ci, so CI has not run on it.

What I ran locally:

  • A throwaway spec with a data service whose @Where closures compare qty with avg, max and min of qty, the same comparisons as a where query, and James's helper typed DetachedCriteria<Animal> and given the criteria of a Cat. The cats weigh 10, 20 and 30 and the other animals 1, 2 and 3, so the average of the cats is 20 and that of every animal 11:

    Query Hibernate 5, 7.0.x Hibernate 5, 8.0.x Hibernate 7, 8.0.x this head, Hibernate 5 and 7
    data service @Where({ qty > avg(qty) }), and >=, == max, == min and one with an argument SQLGrammarException SQLGrammarException, with the aggregate in the outer select QueryArgumentException or ClassCastException the right rows
    where { qty > avg(qty) }.list() and where { qty == max(qty) } the right rows the right rows QueryArgumentException or ClassCastException the right rows
    the helper with weight > avg(weight), given the cats [30] [30] QueryArgumentException [30]
    the helper with weight == min(weight), given the cats [10] [10] ClassCastException [10]
    the helper with weight > avg(weight).of { weight > 0 }, given the cats [20, 30] [20, 30] QueryArgumentException [30]

    The SQLGrammarException is H2's Values of types "INTEGER" and "ROW()" are not comparable.

  • This PR's diff applied to 7.0.x and to 8.0.x: it applies without conflicts.

Needs a fix

  1. Branch: the fix goes to 7.0.x. The bug is in DetachedCriteriaTransformer in grails-datamapping-core, and Hibernate 5 has it on 7.0.x: every data service method of the table fails there. By the rule in the dev@ thread, the fix goes to the oldest branch that has the bug and is merged up. Hibernate 7 cannot run an aggregate comparison on 8.0.x at all, in a where query either, until #16816 is merged, so the Hibernate 7 specs need #16816: they can follow on 8.0.x after the merge-up, on top of #16816. For 7.0.x:

    • the example-app specs use ScenarioItem from #16816. A data service spec in grails-data-hibernate5-core, built with datastore.getService(...) as DataServiceSpec does, or a domain class of the Hibernate 5 app's own, would do.
    • WhereAggregateHelperSpec extends HibernateGormDatastoreSpec, which the Hibernate 5 tests on 7.0.x do not have.
  2. getTargetClass() is new public API on the patch line, and compiled applications would call it. The transformer writes the call into every query with an aggregate on the right of a comparison, so application classes compiled against the release call it, and it cannot be removed without breaking them. The patch rule asks for no new public methods on 7.0.x or 8.0.x. Outside a data service, the bare form already takes the aggregate over the criteria the closure runs against on 7.0.x: the helper gives [30] there, because AbstractDetachedCriteria.add turns a closure value into new DetachedCriteria(targetClass).build(closure). What fails in a data service is the projections call that the transformer adds inside that closure: it does not reach the subquery's criteria. On 8.0.x it lands on the enclosing query (the outer select avg(...)), and on 7.0.x the subquery returns whole rows. A fix that keeps the bare closure and gets that call to the subquery's criteria would need no new method. I tried calling it on the closure's delegate: that gives the right rows everywhere but in the data service, where delegate is the service implementation, so I do not have a version to suggest. If there is no way without the method, please say in the description why the patch line needs it.

  3. The .of { } form changes a result. On 7.0.x and 8.0.x, weight > avg(weight).of { weight > 0 } in the helper averages over every animal, the class the code declares ([20, 30]). At this head it averages over the cats ([30]), as the bare form does. Making the two forms agree is reasonable, but it is a change to a result that is not part of #16807, so the rule in the dev@ thread asks for a second reviewer, a description of the change and an upgrade note that says how to get the old result, for example with a subquery built for the parent class. I would keep it out of this fix, in a pull request of its own.

  4. Description. Please update it to match this head:

    • The branch and the stacking change with item 1. You are right that the bug does not depend on the domain binder; it does not depend on Hibernate 7 either, as Hibernate 5 fails the same way.
    • It says that the 7 scenarios fail on Hibernate 7 without the change; on Hibernate 7 they need #16816 as well, and on Hibernate 5 they fail without the change too.
    • The second commit, getTargetClass() and the .of { } change appear only in your comment of 11-17.
    • "Mongo, Neo4j and graphql are left to CI", but with no ci CI does not run. The transformer is shared, so the MongoDB module's tests should run before the merge: locally, or with run ci on the 7.0.x pull request.

Suggestions

  1. WhereAggregateHelperSpec passes on 7.0.x and 8.0.x with Hibernate 5 without this PR, since the bare form takes the class of the criteria there. It guards against the first commit of this PR, which is useful; it could say so.
  2. The example-app specs can use a bare @Integration: each app has one Application class, and the annotation finds it.

Verified as correct

  • At this head, the five data service methods and the where queries of the table return the right rows on Hibernate 5 and 7, and James's helper takes the average and the minimum over the cats.
  • The property projection that feeds gtAll and its siblings is unchanged.

@borinquenkid

Copy link
Copy Markdown
Member Author

Thanks for the table. Status on your items, before I change the PR:

  1. 7.0.x: I tried the fix on 7.0.x (explicit new DetachedCriteria(...).build(closure) for the aggregate, in a data service @Where). It does not work there yet: the compiled closure casts the service implementation to DetachedCriteria (GroovyCastException), whichever class expression I give the constructor, and 7.0.x lacks two things 8.0.x has (appliedWhereTransform, the scope recompute of Nested criteria subqueries bind to the outer criteria under @CompileStatic (Groovy 4) #16425, and DELEGATE_FIRST in projections). Porting those is a separate change, so I will not push a half fix to the patch line.
  2. getTargetClass(): agreed that it cannot be new public API on a patch line. Without it the bare form keeps the criteria's class, so the helper results stay as they are today.
  3. .of { }: I will take that out of this PR.
  4. Description / tests: will follow with the branch change; MongoDB tests will run before the merge.

Until the data service problem on 7.0.x is understood, this PR stays as it is.

https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

4 participants