Skip to content

Materialize deferred criteria on a copy before translating queries and subqueries - #16820

Open
borinquenkid wants to merge 25 commits into
fix/h7-where-query-functions-8.0.xfrom
fix/datastore-wherelazy-thread-safe-8.0.x
Open

borinquenkid wants to merge 25 commits into
fix/h7-where-query-functions-8.0.xfrom
fix/datastore-wherelazy-thread-safe-8.0.x

Conversation

@borinquenkid

@borinquenkid borinquenkid commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Summary

Resolves two lifecycle defects with deferred criteria (whereLazy, buildLazy). Fixes #16814.

Stack: the base of this PR is #16818's branch, and #16818 is not in #16816. This PR therefore carries #16818's changes and lands together with #16821, which repairs the Hibernate 7 SubqueryAliasSpec > Test subquery with root alias failure that #16818's fail-fast causes (tracked in #16884). Merge them as one unit: no single merge of this PR leaves the stack green.

What this head does

  • The deferred closure runs once, on first use, under a lock, on the criteria itself, and what it adds stays part of it. A query reads a copy (materializeLazyCriteria()), and a criteria derived later (where, max, offset, withConnection, buildLazy) carries the result instead of running the closure again. buildLazy on a deferred criteria keeps the earlier closure.
  • Reading getCriteria(), getProjections(), getOrders(), getAlias() and the other getters runs the closure. Before, a getter on a deferred criteria returned what was there without it.
  • clone() copies junctions and association nodes through one identity map, builds its own projection list, and copies the root alias, so a derived criteria keeps the alias. A copy made by the thread that is running the closure does not carry it, so buildLazy { sort('name') } cannot recurse.
  • Deferred subqueries are materialized before they are translated on Hibernate 7, Hibernate 5 (at getHibernateDetachedCriteria) and in bulk statements, so a restriction in a buildLazy subquery is no longer dropped.
  • applyLazyCriteria() is not on the QueryableCriteria interface and is protected. QueryableCriteria.materializeLazyCriteria() is a default method for query implementations, so existing implementations stay compatible.
  • DetachedCriteriaDeferredQuerySpec has a row for each way into the builder and for each way the closure can derive from itself; the reviewer scenarios are pinned on Hibernate 5 and 7. The shared change was run against the simple, MongoDB (core, bson) and Neo4j suites.

Known

  • 7.0.x has the same race and the same dropped subquery restriction, but only an explicit buildLazy builds a deferred criteria there, so a 7.0.x port can wait for someone who hits it.
  • The stack has not had a CI run; it carries no ci. The combined branch needs one before the first of these merges into 8.0.x.

https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL

Handle QueryableCriteria values in GreaterThan, GreaterThanEquals,
LessThan, and LessThanEquals criteria within PropertyCriterionHandler
by building a JPA Subquery expression instead of converting the criteria
as a literal value. Infer the subquery return type from its projections
for aggregate, identifier, and property projections.

Fixes #16812
…ed paths, and filter nulls in inList

- Copy subquery projections into a local ProjectionList in PropertyCriterionHandler and SubqueryCriterionHandler instead of mutating the caller's QueryableCriteria in-place
- Throw ConfigurationException when property paths cannot be resolved in PropertyComparisonCriterionHandler, PropertyNameCriterionHandler, and SubqueryCriterionHandler
- Filter null elements in In criteria across PropertyCriterionHandler and FunctionCallingCriterionHandler and return a false predicate (disjunction) when no non-null values remain

Fixes #16815
…d subqueries

Introduce QueryableCriteria.materializeLazyCriteria() and implement it in
AbstractDetachedCriteria and DetachedCriteria so that deferred closures
(whereLazy / buildLazy) are applied to a cloned instance rather than
mutating the shared criteria in place.

Materialize deferred criteria in PropertyCriterionHandler.buildSubquery,
ExistsCriterionHandler, and JpaQueryBuilder so that lazy subqueries retain
their restrictions when translated.

Fixes #16814

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Deferred criteria still share mutable association nodes and ignore the effective root alias, risking accumulated restrictions, cross-thread mutations, and broken alias-qualified queries.

2 open issues
What changed in this PR

This PR addresses deferred-criteria lifecycle defects in GORM by materializing lazy definitions on copies before query execution and subquery translation. It builds on the earlier subquery-projection immutability work.

Changes:

  • Adds a materialization contract and copy-based handling for deferred criteria.
  • Integrates materialization into query execution, bulk operations, dynamic finders, and Hibernate/JPA subqueries.
  • Adds regression coverage for deferred restrictions, copy behavior, pagination, and bulk operations.
File Description
grails-datastore-core/​src/​test/​groovy/​org/​grails/​datastore/​mapping/​query/​jpa/​JpaQueryBuilderFunctionSpec.groovy Tests deferred root materialization and interface defaults; updates the subquery stub.
grails-datastore-core/​src/​main/​groovy/​org/​grails/​datastore/​mapping/​query/​jpa/​JpaQueryBuilder.java Materializes root criteria and subqueries before JPQL translation.
grails-datastore-core/​src/​main/​groovy/​org/​grails/​datastore/​mapping/​query/​api/​QueryableCriteria.java Adds default deferred-criteria lifecycle methods.
grails-datamapping-core/​src/​main/​groovy/​org/​grails/​datastore/​gorm/​query/​criteria/​AbstractDetachedCriteria.groovy Adds copy-based materialization and uses it for dynamic finders.
grails-datamapping-core/​src/​main/​groovy/​grails/​gorm/​DetachedCriteria.groovy Uses materialized criteria for execution and bulk operations.
grails-data-hibernate7/​core/​src/​test/​groovy/​grails/​gorm/​tests/​detachedcriteria/​DetachCriteriaSubquerySpec.groovy Adds Hibernate-backed deferred-criteria regression tests.
grails-data-hibernate7/​core/​src/​main/​groovy/​org/​grails/​orm/​hibernate/​query/​PropertyCriterionHandler.java Materializes property subqueries before reading projections and restrictions.
grails-data-hibernate7/​core/​src/​main/​groovy/​org/​grails/​orm/​hibernate/​query/​ExistsCriterionHandler.java Materializes existence subqueries before translation.

Review effort: Balanced

if (pending == null) {
return this
}
AbstractDetachedCriteria<T> materialized = clone()

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in e9e5626. clone() now copies every DetachedAssociationCriteria node, including nested ones, through copyAssociationCriteria, so the copy's criteria list and its association map share one new node per association, and that node keeps its alias. DetachedAssociationCriteria.newInstance() keeps the node type on copy. A deferred createAlias() or a chained .eq(...) applied to a materialized copy therefore never reaches the original. The new clone-isolation features in DetachedCriteriaCloneSpec cover shared nodes, nested nodes, and a deferred closure that mutates a node.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e9e5626: clone() now copies the association nodes so a deferred createAlias() no longer touches the original.

- Copy DetachedAssociationCriteria nodes, including nested ones, in AbstractDetachedCriteria.clone() so a deferred closure never mutates nodes shared with the original
- Override newInstance() in DetachedAssociationCriteria so a copied node keeps its type, association and alias
- Create the query with the alias of the materialized criteria in DetachedCriteria.withPopulatedQuery()
- Add clone isolation specifications for association nodes
… on H5 and H7

Add integration scenarios to the Hibernate 5 and Hibernate 7 grails-hibernate test apps
that compare a property with avg, max and min subqueries (>, >=, <, <=, ==, !=), use a
subquery in an in list, and combine one with another criterion, through where queries
and the criteria builder. Both modules return the same rows, and the scenarios fail
when the greater-than, less-than, equal or not-equal subquery branch is changed.
@borinquenkid borinquenkid added this to the grails:8.0.1 milestone Nov 4, 2026

@jdaugherty jdaugherty left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is skipped on this stack, so I ran it locally at 6f0d3e0. The specs added here pass on H5 and H7 (DetachedCriteriaCloneSpec, JpaQueryBuilderFunctionSpec, the H7 DetachCriteriaSubquerySpec, and WhereLazyScenarioSpec in both example apps). grails-datastore-core, grails-datamapping-core and grails-datamapping-core-test are green in full, as are the H5 core where/criteria/subquery specs and codeStyle on the three touched modules.

In the H7 core where/criteria/subquery specs the only failure is SubqueryAliasSpec > Test subquery with root alias. It fails the same way on the base branch (the fail-fast added in #16818) and looks like it is repaired by #16821, so it is not from this change, but it does mean the stack cannot land one PR at a time.

Materializing on a copy fixes the race. Five comments inline: the first is a regression against the base branch, the second is a gap on Hibernate 5.

Not in the diff: grails.gorm.rx.DetachedCriteria.prepareQuery still calls applyLazyCriteria() on the shared instance, so RxGORM keeps the race described in #16814.

List<Query.Criterion> clonedCriteria = new ArrayList<>(this.criteria.size())
for (Query.Criterion criterion : this.criteria) {
clonedCriteria.add(criterion instanceof DetachedAssociationCriteria ?
copyAssociationCriteria((DetachedAssociationCriteria) criterion, associationCopies) : criterion)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This copies the association nodes that sit directly in criteria, but a node added inside or {} / and {} / not {} lives in a Query.Junction, and the junction is still shared by reference. After clone() the junction holds the original node while associationCriteriaMap holds a copy, so the tree and the map no longer refer to the same node (the invariant the copyAssociationCriteria Javadoc states), and createAlias() on a derived criteria sets the alias on a node that is not in the tree.

This passes on the base branch and fails on this head with PathElementException: Could not resolve attribute 'o' of '...User' (H7 core, with the User / Organisation classes from DetachCriteriaSubquerySpec):

DetachedCriteria<User> base = new DetachedCriteria<>(User).build {
    or {
        organisation { eq('name', 'A') }
        eq('email', 'user1@b')
    }
}
List<User> result = base.build {
    createAlias('organisation', 'o')
    like('o.name', 'A%')
}.list()

The same query with organisation { ... } at the top level instead of inside the or works. Copying junctions recursively through the same associationCopies map would keep the tree and the map in step. Please add a clone test with the association node inside a junction.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 97949fc: clone() now copies junctions recursively (copyCriterion), creating a new Conjunction/Disjunction/Negation and copying children through the same association map, so an alias added inside or {}/and {}/not {} reaches the copy and not the shared node. Covered per junction type and nested in DetachedCriteriaCloneSpec.


void "a lazy criteria keeps its restriction as a subquery"() {
given:
def annIds = ScenarioOwner.whereLazy { name == 'ann' }.property('id')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Hibernate 5 this spec never has a deferred criteria. GormStaticApi.whereLazy is where(callable); only H7's HibernateGormStaticApi overrides it with buildLazy. The criteria list of ScenarioOwner.whereLazy { ... } is already populated on H5 before anything runs, so the five features here pass on H5 with or without this PR.

With a criteria that really is deferred, H5 queries still drop the restriction, because HibernateCriteriaBuilder.getHibernateDetachedCriteria reads queryableCriteria.getCriteria() without materializing. On this head, in grails-data-hibernate5-core with Organisation rows A and B:

def sub = new DetachedCriteria<>(Organisation).buildLazy { eq('name', 'A') }.property('name')
Organisation.where { inList('name', sub) }.list()*.name   // [A, B], expected [A]
Organisation.where { notIn('name', sub) }.list()*.name    // [],     expected [B]
Organisation.where { inList('name', sub) }.deleteAll()     // 1, because JpaQueryBuilder now materializes

exists with a deferred subquery that matches nothing returns every row as well. So on H5 the same criteria selects different rows for list() and for deleteAll().

Every H5 subquery path goes through getHibernateDetachedCriteria, so materializing at the top of it covers them all. If H5 queries are meant to be out of scope here, please say so in the description and on #16814, and drop the H5 copy of this spec, since it reads as coverage it does not give. Either way, a buildLazy case in the H5 DetachCriteriaSubquerySpec is what would exercise this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and fixed in 97949fc: the H5 WhereLazyScenarioSpec is removed, since H5 whereLazy is plain where. H5 now has real buildLazy coverage in DetachCriteriaSubquerySpec (core), and H5 materializes at getHibernateDetachedCriteria.

* Applies a deferred criteria definition, if any, so that {@link #getCriteria()} and
* {@link #getProjections()} can be read. Implementations without deferred criteria do nothing.
*/
default void applyLazyCriteria() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing calls this through the interface. The H7 handlers and JpaQueryBuilder use materializeLazyCriteria(), and the only callers of applyLazyCriteria() left are AbstractDetachedCriteria.add() and the RxGORM criteria, each on itself.

Declaring it here, and widening AbstractDetachedCriteria.applyLazyCriteria() from protected to public, publishes the in-place mutation this PR removes as new API in a patch release. The Javadoc on the AbstractDetachedCriteria method ("Query implementations call this before they translate a criteria that only appears as a subquery value") also describes a design the PR no longer uses.

Please drop it from the interface and leave the method protected. The tests that pin it (QueryableCriteria default methods return self and do nothing in part, and test applyLazyCriteria mutates in place idempotently when called directly) go with it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 97949fc: applyLazyCriteria() is gone from QueryableCriteria; only materializeLazyCriteria() stays as the default. applyLazyCriteria() is protected again in AbstractDetachedCriteria.

}
AbstractDetachedCriteria<T> materialized = clone()
materialized.lazyQuery = null
materialized.with(pending)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A behavior change that should be stated: the deferred closure now runs on every execution. On the base branch it ran once, at first use, and the result stuck.

Map wanted = [name: 'A']
Closure deferred = { eq('name', wanted.name) }
def criteria = new DetachedCriteria<>(Organisation).buildLazy(deferred)
criteria.list()*.name   // [A]
wanted.name = 'B'
criteria.list()*.name   // base branch: [A]   this PR: [B]

Either meaning is defensible, but the difference is observable and whereLazy is documented as returning a criteria "that is lazily initialized". Please decide which one is intended, say so in the PR description and in the whereLazy / buildLazy Javadoc, and add a test that pins it. If evaluate-once is the intent, the materialized copy can be cached instead of rebuilt.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stated and decided in 97949fc: the closure runs once, on first use. The result is kept in a private template and each materialization returns a copy of it, so use never changes the criteria and the closure is not re-run. Documented in the buildLazy Javadoc; DetachedCriteriaCloneSpec pins run-once and independent copies.

thrown(InvalidDataAccessResourceUsageException)
}

void "JpaQueryBuilder(QueryableCriteria) materializes deferred criteria on construction and in subqueries"() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The title says "and in subqueries", but this feature only covers the root criteria. In this spec the materializeLazyCriteria() call in buildSubQuery is only reached with the stub above that returns itself (line 77), so the spec does not notice if that call is missing. A case where the In subquery stub materializes to a different criteria would cover it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 97949fc: the spec now has a deferred In-subquery case with an exact expected query string and parameters, so the materializeLazyCriteria() call in buildSubQuery is exercised for real. RxGORM's prepareQuery race is not in this diff and grails-datamapping-rx is not built, so I left it alone.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferred criteria are copied before they are translated. Not approving. An association inside and, or, or not is copied into the lookup map and left as it was inside the junction, so a later restriction is added to the copy the query never runs.

…ate template, cover Hibernate 5

clone() copies junctions with their association nodes through the same map,
so the criteria tree and the association map never diverge. A deferred
criteria runs its closure once on a private template and each read gets a
copy. Hibernate 5 subqueries materialize a deferred criteria. The
applyLazyCriteria method leaves the QueryableCriteria interface and is
protected again. The H5 scenario spec that never had a deferred criteria is
replaced by H5 core subquery specs.

Claude-Session: https://claude.ai/code/session_01Dmki4SQGwRSj8pRJ5KCWyX

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved. After the first execution, a change to an existing association is ignored, and extending the query runs the deferred closure again. Cloning an association inside a junction now keeps a later restriction.

…, at every way in

A review of the deferred-criteria template showed two bugs and one cause. After the
first query a restriction, alias, order or projection added to the criteria was ignored,
because queries read a cached template while most mutators changed the criteria; and
extending the criteria after a query ran the closure again, so a value the closure read
could differ between the original and the derived query.

The builder had two sets of ways in: the in-place methods (the criterion DSL, order,
sort, join, select, setAlias, createAlias, the projection methods) and the clone-and-modify
methods (max, offset, withConnection, buildLazy, where), plus getters and translators that
read the lists. Only add, methodMissing and handleJunction applied the pending closure.

The invariant is now one rule with one chokepoint: applyLazyCriteria() runs the closure
once, under a lock and with a re-entrancy guard, and every method that reads or writes
state, and clone(), calls it first. A query reads a copy through materializeLazyCriteria().
The template is gone, and buildLazy on a deferred criteria keeps the earlier closure
instead of replacing it.

A clone built its projection list as an inner class of the criteria it was cloned from, so
its methods belonged to the wrong criteria; a clone now builds its own.

DetachedCriteriaDeferredQuerySpec lists every way in and fails for one that does not apply
the closure (32 of its 33 rows fail on the previous implementation). The reviewer scenarios
are also pinned against Hibernate 5 and 7.

Claude-Session: https://claude.ai/code/session_01Dmki4Tgxp7dLBhzpe2xT93
…ading it shows nothing

Eleven specs proved that a criteria was not changed by use by reading getCriteria() and
expecting it to be empty. A read now runs the deferred query, as every way into the
criteria does, so the criteria holds what its closure added. They now assert that the
closure's restrictions are present exactly once and that translating a subquery added no
projection and no further restriction to the shared criteria.

Claude-Session: https://claude.ai/code/session_01Dmki4Tgxp7dLBhzpe2xT93

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved. A clone made while the deferred closure is running copies that closure and runs it again, until the query overflows the stack. A restriction added after the first execution is kept. Extending an executed query no longer reruns the closure. Threads PRRT_kwDOAAfRJ860EXVo and PRRT_kwDOAAfRJ860EXVu resolved.

criteria.@joinTypes = new HashMap<>(this.joinTypes)
criteria.@junctions = new ArrayList(this.junctions)
criteria.@connectionName = this.connectionName
criteria.@lazyQuery = this.lazyQuery

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildLazy { sort('name') } clones the criteria while the deferred closure is still running. The clone copies that closure and has no guard, so sort reads the orders and runs the closure again, until the query throws StackOverflowError. A clone made during that run must not carry the running closure.

…carry the query

buildLazy { sort('name') } derives a criteria while its own closure runs. The copy
held the running closure, ran it again on first read, and recursed until the query
threw StackOverflowError. clone() now leaves the pending query off a copy made by the
thread that is running it.

Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
@borinquenkid

Copy link
Copy Markdown
Member Author

@jamesfredley Fixed in 348ca6c, on top of 6d2431c.

You were right: buildLazy { sort('name') } derives a criteria while its own closure runs. The copy took the running closure, ran it again on first read, and recursed until StackOverflowError. clone() now leaves the pending query off a copy made by the thread that is running it, so the closure runs once and the copy holds only what the closure added so far.

DetachedCriteriaDeferredQuerySpec has a row for each way the closure can derive from itself: sort, max, where, build and a second buildLazy. The query runs once, the copy holds what it added, and neither criteria is left with a pending query. sort and where fail with StackOverflowError without the change, and the other three pin that they stay correct. The datamapping-core suite passes.

One thing outside this change: Hibernate 7 SubqueryAliasSpec > Test subquery with root alias fails at the head of this PR with and without this commit. #16821 carries the fix and rewrites that spec. Tracked in #16884.

…s-comparisons-8.0.x

Takes the review changes of #16816: the merge of 8.0.x with #16573, the subquery criteria joined with AND, the function list, the
rejection of a function over an rlike in a bulk statement, the package-private criterion handlers and the restored SPI documentation.

Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
…-projection-immutability-8.0.x

Takes the merge of fix/h7-where-query-functions-8.0.x, the review changes of #16816.

Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
…e-wherelazy-thread-safe-8.0.x

Takes the merge of fix/h7-scalar-subqueries-comparisons-8.0.x, the review changes of #16816. buildSubQuery keeps the deferred
criteria materialization of this branch and joins the criteria of the subquery with AND.

Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
jamesfredley
jamesfredley previously approved these changes Nov 8, 2026

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A clone made while the deferred closure is running no longer runs that closure again.

@matrei matrei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 8f1df45 (base fix/h7-criteria-projection-immutability-8.0.x, #16818's head branch, at 0164592). CI has never run on this PR: it carries no ci.

What I ran locally, on this head:

  • The suites of the modules this changes or depends on: grails-datastore-core (371), grails-datamapping-core (1873), grails-data-simple (15), grails-data-hibernate5-core (1182) and grails-data-hibernate7-core (4543). The only failure is H7 SubqueryAliasSpec > Test subquery with root alias (item 1).
  • On #16816's head with #16818 (10531d1) applied on top: H7 SubqueryAliasSpec > Test subquery with root alias fails (item 1).

jdaugherty's five points from 11-04 are answered in 97949fc and after: junctions are copied with their association nodes through one identity map, Hibernate 5 materializes in getHibernateDetachedCriteria and has real buildLazy cases, applyLazyCriteria() is off the interface and protected again, the run-once semantics are decided and documented, and the subquery case in JpaQueryBuilderFunctionSpec materializes to a different criteria. James approved this head.

Needs a fix

  1. This PR cannot land on its own, and it carries #16818. Its base is #16818's head branch, and #16818 is not in #16816: it was squashed into #16817's branch after #16817 had been merged into #16816 (details on #16816). Once this PR is retargeted to #16816's branch, #16818's changes show up in its diff. They also bring the failure that keeps this PR from landing alone: with #16818 on #16816, H7 SubqueryAliasSpec > Test subquery with root alias fails with ConfigurationException: Cannot use property comparison criteria on non-existent property [t.club], as it does at this head (jdaugherty found it on 11-04; you track it in #16884), and #16821 repairs it. Please say in the description that this PR carries #16818 and lands together with #16821, or bring #16818 and the repair into one PR, so that no single merge leaves the stack red.

  2. CI. The same as on #16816: the stack has never had a CI run. Before the first of these merges into 8.0.x, the combined branch needs one.

  3. Description. The two "Update" sections describe two designs, and the second replaces the first (the private template is gone). Please replace them with what this head does:

    • The deferred closure runs once, on first use, under a lock, on the criteria itself, and what it adds stays part of it. A query reads a copy (materializeLazyCriteria()), and a criteria derived later (where, max, offset, withConnection, buildLazy) carries the result instead of running the closure again.
    • Reading getCriteria(), getProjections(), getOrders(), getAlias() and the other getters runs the closure. Before, a getter on a deferred criteria returned what was there without it.
    • clone() copies junctions and association nodes, and now the root alias as well, so a derived criteria keeps the alias.
    • Deferred subqueries are materialized before they are translated on Hibernate 7, Hibernate 5 and in bulk statements, so a restriction in a buildLazy subquery is no longer dropped.

Note

  1. The same race and the dropped subquery restriction exist on 7.0.x: applyLazyCriteria() there clears lazyQuery before it runs the closure, and nothing materializes a deferred subquery. On 7.0.x only an explicit buildLazy builds a deferred criteria, since whereLazy there is where. That is rare enough that this fix on 8.0.x, where Hibernate 7's whereLazy makes every such criteria deferred, is the right place; a 7.0.x port can wait for someone who hits it.

  2. QueryableCriteria.materializeLazyCriteria() is a new default method on a public interface in grails-datastore-core, on a patch line. It is needed across modules and a default method keeps implementations compatible, so I'm fine with it; a line in its Javadoc saying it is for query implementations would keep applications from calling it.

Verified as correct

  • applyLazyCriteria() takes the lock only while a closure is pending (a volatile read otherwise), and a thread that finds it running waits for the result, so no thread reads a half-built criteria. "threads that use a deferred criteria for the first time at once see the closure run once" pins that with eight threads.
  • A copy made by the thread that is running the closure (buildLazy { sort('name') }) does not carry the closure, so it cannot recurse.
  • RxGORM's prepareQuery calls the same applyLazyCriteria(), so it gets the lock as well, and grails-datamapping-rx is not built on 8.0.x.
  • On 7.0.x and here, the closure runs once and the result sticks, so existing behaviour is kept apart from the race.

@borinquenkid

Copy link
Copy Markdown
Member Author

Thank you, all five points are handled in ef5bdda and the description.

  1. The description now says this PR carries fix(hibernate7): preserve subquery projections, fail fast on unresolved paths, and filter nulls in inList #16818 and lands together with Fix root alias registration and nested alias path resolution in subqueries #16821, which repairs the SubqueryAliasSpec root-alias failure. I have not moved the base, so the diff stays what the review ran against; if you want the retarget to fix(hibernate7): support where-query functions and refactor PredicateGenerator #16816 first, say so and I will do it with the stack.
  2. Agreed on CI: I will remove no ci and let the combined branch run once before the first of these merges.
  3. The description is rewritten to what this head does (run once under a lock, getters run the closure, clone() and the alias, deferred subqueries materialized on both Hibernate versions).
  4. Noted, no 7.0.x port for now.
  5. materializeLazyCriteria() has a line in its Javadoc saying it is for query implementations.

@github-actions
github-actions Bot dismissed jamesfredley’s stale review November 19, 2026 02:18

Dismissed automatically: the pull request changed after this approval, from 8f1df45 to ef5bdda.

@borinquenkid
borinquenkid changed the base branch from fix/h7-criteria-projection-immutability-8.0.x to fix/h7-where-query-functions-8.0.x November 19, 2026 02:19
@borinquenkid

Copy link
Copy Markdown
Member Author

Retargeted to #16816's branch as you asked, so the diff now carries #16818. The one CI run of the combined stack is on #16821 (head 6a8e77d, which merges this branch's Javadoc commit); #16820 stays no ci so it does not run twice.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested changes. The note says the returned criteria can be changed without changing the original. The predicates are still the same objects.

/**
* Returns the criteria to read the query from, with its deferred criteria applied: the deferred query runs once,
* the first time the criteria is used, and what it adds stays part of the criteria. A criteria with a deferred
* query returns a copy, so that a query implementation can extend or translate it without changing the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A query implementation calls this and then lowercases the returned equals value in place. The original criteria changes too, so a later case-sensitive run misses Alice. Either copy the predicates, or say that changing them changes the original.

@matrei matrei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 2, reviewed at ef5bdda (base: #16816's branch fix/h7-where-query-functions-8.0.x). The code is the same as in round 1, plus the Javadoc line of ef5bdda.

Round 1

  1. #16818 and #16821: done. The description says this PR carries #16818 and lands with #16821, and the base is #16816's branch now.

  2. CI: the stack's run is on #16821 at 595ed4b, which merges this head. It passes apart from two jobs that failed on the infrastructure:

    • Windows JDK 25, shard 1: the Gradle worker crashed (0xC00000FD) while compiling the test fixtures of grails-geb.
    • Neo4j Functional Tests (Java 21, indy=true): the JDK download got an HTTP 500.

    Both need a rerun. That run predates #16816's latest commit, though (item A).

  3. Description: done. Two lines are out of date since the retarget: it still says "the base of this PR is #16818's branch", and that "the stack has not had a CI run".

  4. 7.0.x: noted.

  5. materializeLazyCriteria() Javadoc: done, but see item B.

Needs a fix

A. This PR conflicts with #16816's branch now. It moved to 85c8df3 ("keep a null in an in list ...") after your retarget. A trial merge conflicts in:

  • FunctionCallingCriterionHandler.java
  • PropertyCriterionHandler.java
  • JpaQueryBuilder.java
  • CriteriaInListScenarioSpec.groovy in both example applications (added on both sides)

#16821 doesn't have 85c8df3 either, so the stack's CI run hasn't seen it. Please merge #16816's head here and into #16821, and let #16821's CI run again.

B. James is right that the copy shares the criteria it holds. clone() copies the lists, the junctions and the association nodes. Every other criterion is shared: copyCriterion returns it as it is, with the comment "Any other criterion is immutable". It isn't. A PropertyCriterion has setValue, and query code on 8.0.x sets values in place while it translates:

  • Query.addToJunction sets the resolved value of every property criterion it adds.
  • Hibernate 5's applySubCriteriaToJunction replaces a subquery value with a Hibernate DetachedCriteria, and AbstractHibernateQuery converts values to the property's type.
  • HibernateQuery sets the converted subquery.
  • MongoDB coerces id values (MongoQuery, around line 805).

All of these are on 8.0.x already, and before this PR they changed the criteria directly. So nothing gets worse, but the documentation promises more than the copy gives. The promise is in the materializeLazyCriteria() Javadoc ("can extend or translate it without changing the criteria it was given") and in the class Javadoc of AbstractDetachedCriteria ("a translator that extends the result never changes the criteria it was given"). Copying every criterion would be a larger change than this fix needs on a patch line. I'd rather correct the text, for example:

  • materializeLazyCriteria(): "A criteria with a deferred query returns a copy of its lists, junctions and association nodes, so that a query implementation can add to it without changing the criteria it was given. The other criteria are shared: a query implementation that changes one in place changes it in the original too."
  • copyCriterion: "Any other criterion is shared. A query implementation that sets its value, as Hibernate 5 does when it converts a value or translates a subquery, sets it in the original as well."

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

5 participants