Skip to content

Subqueries take the connection of the query that holds them (#16886) - #16895

Open
borinquenkid wants to merge 6 commits into
fix/where-subquery-mongo-neo4j-standalone-8.0.xfrom
fix/subquery-connection-8.0.x
Open

borinquenkid wants to merge 6 commits into
fix/where-subquery-mongo-neo4j-standalone-8.0.xfrom
fix/subquery-connection-8.0.x

Conversation

@borinquenkid

Copy link
Copy Markdown
Member

Fixes #16886

Stacked on #16875: the base is its branch, so the diff is only this change. When #16875 merges, this one is retargeted to 8.0.x.

Problem

MongoDB and Neo4j run a subquery before the outer query. avg(age).of { } is built by the where transform as a criteria with the default connection, so on Person.where { age > avg(age).of { } }.withConnection('archive') the average was taken on the default connection and compared with the rows of archive.

Change

AbstractDetachedCriteria gives a subquery that has the default connection the connection of the criteria that holds it, both when withConnection is applied (after the criterion was added) and when a criterion is added to a criteria that already has a connection. It covers ==, !=, >, >=, <, <=, inList, notIn and the same inside or, and and not. A subquery built with its own withConnection keeps it. The criteria withConnection was called on is not changed: criteria that move are copied.

Not covered: a subquery inside an association block runs on the connection of its own criteria; the upgrade guide says so.

Tests

  • DetachedCriteriaSubqueryConnectionSpec (datamapping-core): 13 cases; 10 fail without the change.
  • SubqueryConnectionSpec for MongoDB (real Mongo) and Neo4j (two embedded servers), with different ages on each connection so the wrong average selects different rows. Without the change 4 of 5 (Mongo) and 2 of 5 (Neo4j) fail. The Neo4j inList case passes either way and guards against a regression.
  • Full grails-datamapping-core suite, Checkstyle and CodeNarc pass.

Docs: where reference and the 8.0.x upgrade guide.

https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL

…a that holds it (#16886)

A subquery written as avg(age).of { } is built with the default connection, so MongoDB
and Neo4j, which run it before the outer query, read it from the wrong connection when
the outer query uses withConnection. AbstractDetachedCriteria now gives a subquery that
has the default connection the connection of the criteria that holds it, when
withConnection is applied and when a criterion is added to a criteria that has one.
A subquery built with its own connection keeps it.

Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
Copilot AI balanced review requested due to automatic review settings November 8, 2026 16:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approving. A subquery that inherited its connection is left behind when the outer query is switched back, and a subquery that chose the default connection is moved when the outer query uses another one.

}

private static QueryableCriteria onConnection(QueryableCriteria subquery, String name) {
if (subquery instanceof AbstractDetachedCriteria && ((AbstractDetachedCriteria) subquery).connectionName == ConnectionSource.DEFAULT) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Input: an archive query whose age is compared with avg(age), then switched back to the default connection. The subquery stays on the archive connection, so the default rows are compared with the archive average and a match is dropped. Keep an explicit subquery connection. Move an inherited one with the outer query, including back to the default.

…ria in both directions, and keeps one it chose (#16886)

A subquery that took the connection of the criteria holding it stayed there when that criteria went back to the default connection, and a subquery that had chosen the default connection was moved when the criteria used another one. Whether the connection was chosen with withConnection is now recorded; only a connection that was not chosen follows the criteria, to any connection including the default.

Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
@borinquenkid

Copy link
Copy Markdown
Member Author

@jamesfredley Fixed in af8e07b. Whether a subquery's connection was chosen with withConnection is now recorded. Only one that was not chosen follows the outer criteria, to any connection including the default; one that was chosen, the default included, keeps it. Three features in DetachedCriteriaSubqueryConnectionSpec cover both directions and a third connection.

…subquery matches no row) into fix/subquery-connection-8.0.x

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approving. An ordinary where subquery is treated as if the caller chose its connection, so it does not follow the outer query.


AbstractDetachedCriteria<T> withConnection(String name) {
AbstractDetachedCriteria newCriteria = switchConnection(name)
newCriteria.connectionExplicit = true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the archive connection, inList('age', Person.where { age > 50 }.property('age')) returns no rows. The archive has Jasper and Mona above 50. Person.where already sets the default connection, and this line marks that as an explicit choice, so the subquery does not inherit archive. Keep a connection the caller asked for. Do not mark the automatic default as an explicit choice.

…tandalone-8.0.x' into fix/subquery-connection-8.0.x
…ws the query that holds it

Entity.where marked the criteria it returns as having a chosen connection, the default one too, so
inList('age', Person.where { age > 50 }.property('age')) inside a query on the archive connection kept
the default connection: MongoDB and Neo4j, which run the subquery by themselves, read the default
rows and found none above 50. A connection the caller names, as in Entity.archive.where, is chosen;
the default is the one nobody chose, so a criteria that has it is left alone and follows the query
that holds it. A subquery given a connection with withConnection keeps it, the default one included.

The Neo4j rows read a count outside a transaction: inside one, the subquery borrows the session that
is bound, and the connection it asked for does not show.

Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
@borinquenkid

Copy link
Copy Markdown
Member Author

@jamesfredley The second case reproduced and is fixed in the new commit; the first one already behaves as you describe on the branch, and I pinned it.

  • inList('age', Person.where { age > 50 }.property('age')) on the archive connection. Entity.where called withConnection(qualifier), so the criteria it returns counted as having a chosen connection even when the qualifier was only the default, and the subquery kept the default connection while the outer query read the archive: MongoDB and Neo4j found no age above 50. where now leaves the criteria alone when the qualifier is the default, which it has anyway, so it is not marked as chosen and follows the query that holds it. A connection the caller names, as in Entity.archive.where, is still chosen, and so is one given with withConnection, the default one included: the spec has a subquery given the default connection that keeps it inside an archive query, on both stores.
  • An archive query switched back to the default connection. I ran Person.where { age > avg(age) }.withConnection('archive').withConnection(ConnectionSource.DEFAULT) on Neo4j, and it compares with the default average (15) and finds Lisa, with and without this change. Your comment was written against an earlier commit than the explicit-connection one (6117ef06b0), which moves an inherited connection with the outer query and keeps a chosen one. The row is in the spec so that it stays that way. If you have an input that still keeps the archive average, please send it.

Why your second case was not found by the rows already in the spec: the Neo4j rows ran inside withTransaction of the connection they read, and inside a transaction a subquery borrows the session that is bound, so the connection it asked for did not show. The two new Neo4j rows read a count outside any transaction, and both fail without the change. The MongoDB rows have no transaction, and pass with it.

The merge of the base branch (#16875) is in this push: its last commit repaired the Neo4j test sources, which did not compile on the old head this branch was built on.

The Neo4j module (643 tests), the MongoDB module (1,053), grails-datamapping-core (1,832), and the Hibernate 7 (4,463) and Hibernate 5 (1,165) modules pass, and so does codeStyle.

Branch policy: the throws that are listed in Query.java come with the base branch (#16875) and were already in this PR; this push adds none. Where the PR belongs is still open; I have not moved it.

If you find another shape that does not reproduce first time, a failing spec on a throwaway branch, or the few lines you ran, would help me a lot.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants