Skip to content

Make the joins of a where query when the query is executed - #17066

Open
jdaugherty wants to merge 13 commits into
apache:7.0.xfrom
jdaugherty:fix/16853-06-joins-at-execution
Open

jdaugherty wants to merge 13 commits into
apache:7.0.xfrom
jdaugherty:fix/16853-06-joins-at-execution

Conversation

@jdaugherty

Copy link
Copy Markdown
Contributor

Description

Part 6 of the split of #16853, as agreed there and in the dev@ thread Fixes that change results on patch and minor branches. Relates to #15589.

Stacked on #17065. Until that merges, this pull request also shows the commits below it. The change to review is the last commit, 0309713.

A where query, a detached criteria and a dynamic finder joined an
association as soon as criteria, a projection or a sort needed it. The
rest of the query can still change after that: criteria passed to list
are added later, and the fetch argument of list, which can request a
join type, is read last. The result depended on the order in which the
query was put together:

  • a join type in the fetch argument of list was not used for the join
    that the criteria or a projection had already made;
  • a sort stored on the query with sort() was joined before a join type
    requested in list was known, and criteria passed to list could not
    join the same association through an alias;
  • a join type was looked up by the name of the association, so a type
    requested for club was used for team.club, and one requested for
    team.club was not;
  • criteria on a nested association used twice failed with
    "duplicate association path".

Criteria, projections and orders only need the alias of a join. A join
is now registered under its path from the root entity and its alias,
and made when the query is executed, or when a subquery is handed to
the query that embeds it, with the join type that the query requests
for that path. A listener of the query events that adds criteria on an
association to find and findAll with an example, findWhere or
findAllWhere gets its joins made after the event.

Results that change

  • What changes: A join type that the fetch argument of list requests is used for a join that the criteria or a projection of the where query already made; a join type is looked up by the path of the association, not by its name; criteria on a nested association used twice no longer fail.
  • How to get the old result: Remove the join type that the query requests, or request it for the path that the query joins.
  • Setting: No setting. The query requests the join type and it was ignored, which contradicts the call.

The upgrade guide has a note under Grails 7.0.18, and the upgrade skill has the same entry.

Verification

The association sort and join specs of grails-data-hibernate5-core pass on this branch, with grails.gorm.leftJoinAssociationSort off and on, and codeStyle passes for the touched modules. The full build was not run locally.

Contributor Checklist

Please review the following checklist before submitting your pull request. Pull requests that do not meet these requirements may be closed without review.

Issue and Scope

  • This PR is linked to an existing issue that has been acknowledged or approved by the project team. If no approved issue exists, please give background on why this change is necessary. Tickets are preferred for release change log history.
  • This PR addresses the complete scope of the linked issue. Partial implementations or unfinished work should not be submitted for review.
  • This PR contains a single, focused change. Unrelated changes should be submitted as separate pull requests.
  • This PR targets the correct branch for the type of change:
    • Patch release branches (e.g., 7.0.x): Bug fixes only. No new features or API changes.
    • Minor release branches (e.g., 7.1.x): New features are welcome, but breaking existing APIs must be avoided.
    • Major release branches (e.g., 8.0.x): Reserved for major changes. Breaking API changes are permitted.

Code Quality

  • I have added or updated tests that cover the changes introduced in this PR. All code contributions are expected to include appropriate test coverage.
  • I have verified that all existing tests pass by running ./gradlew build --rerun-tasks.
  • My code follows the project's code style guidelines. I have run ./gradlew codeStyle and resolved any violations. See Code Style for details.
  • This PR does not include mass reformatting, style-only changes, or large-scale refactoring unless it was explicitly approved in the linked issue. Unsolicited reformatting will not be accepted.
  • If generative AI tooling was used in preparing this contribution, a quality model was used to ensure contributions are consistent with the project's quality standards.

Licensing and Attribution

Documentation

  • If this PR introduces user-facing changes, I have included or updated the relevant documentation.
  • If this PR adds a new feature, I have updated the What's New section of the Grails Guide.
  • If this PR introduces breaking changes or changes that require user action during an upgrade, I have updated the Upgrade Notes for the corresponding version in the Grails Guide.
  • The PR description clearly explains what was changed and why.

First-time contributors: Please read our Contributing Guide before submitting.
Pull requests that appear to be auto-generated, incomplete, or unrelated to an approved issue may be
closed to help maintainers focus on reviewed and planned work. We appreciate your understanding.

matrei and others added 10 commits November 19, 2026 09:30
Sorting a where query by a property of an association, such as
sort: 'club.name', failed with a QueryException when the query also
had criteria on that association. The criteria join the association
under a generated alias, and the order was added with the association
path, which Hibernate cannot resolve. The order now uses the alias of
the join.
A sort by a path through more than one association, such as
owner.employer.name, is still not supported.
A sort by a path that crosses more than one association, such as
team.club.name, failed with a QueryException whether or not the where
query joined the nested association, because order() split the path at
its first dot only. The sort property is now qualified with the alias of
the last association on its path: an association the query has already
joined is reused, and one it has not joined yet is joined for the sort
and registered like any other join. A sort by two properties of the same
association no longer fails with "duplicate association path" either,
since the sort join is registered instead of created anew for every
order.
Since order() walks the whole sort path, a sort by the identifier of an
association that follows another one on the path, such as team.club.id,
joined that last association too. The join is an inner join unless fetch
requests another type, so the sort left out the rows where the
association is null. Before, only the first association was joined and
Hibernate read club.id from the foreign key of team, which kept them.

The identifier is sorted by the foreign key again when the entity before
the association holds it. An association without such a foreign key, a
hasOne or the inverse side of a one-to-one, is still joined, and so is
the first association of a path, as it always has been.

The spec also covers the join type of a sort join: an inner join by
default, and a left join when fetch requests one. Team.club is nullable
in the fixture for these cases. The docs describe the inner join, how to
request a left join, and the identifier of a nested association.
list with a map of sort properties, such as
Team.list(sort: ['club.name': 'asc', name: 'desc']), sorted by the last
entry only, because each entry replaced the orders of the query. The
same held for a default sort that the mapping declares for more than
one property. Every entry is now applied, in the order of the map.

An association is joined once for all the sort properties that use it.
list joined it once for each, and find and findAll with an example,
findWhere and findAllWhere failed with "duplicate association path".

Part 2 of the split of apache#16853.
A paged query of a detached criteria that has an alias for the queried
class, and criteria that use it, failed when totalCount was read:

    new DetachedCriteria(Team, 't').eq('t.name', 'Arsenal').list(max: 2).totalCount

The count runs on a copy of the criteria, and the copy was created
without the alias, so Hibernate could not resolve t.name. The copy now
has the alias of the original.

Part 3 of the split of apache#16853.
count on a where query or a detached criteria that has a sort, such as
Team.where { ... }.sort('name').count(), kept the order by in the query
that counts the rows, which H2 rejects with "must be in the GROUP BY
list". A query that only counts its rows now has no orders.

The join that a sort by a property of an association makes is kept, so
the count is that of the rows which list returns for the same query.

Part 4 of the split of apache#16853.
… by it

A sort by a property of an association joins the association with an
inner join, so it leaves out the instances whose association is null:
Team.list(sort: 'club.name') does not return the teams without a club,
and count and the totalCount of a paged list do not count them.

The new grails.gorm.leftJoinAssociationSort setting makes an association
that is joined only for a sort a left join, in where queries, detached
criteria, dynamic finders, list, find and findAll with an example,
findWhere, findAllWhere and a criteria query that is passed pagination
arguments. It is disabled by default, so the results of existing
applications do not change in a patch release. Grails 8.1 enables it by
default.

With the setting, a sort by a nested identifier, such as team.club.id,
goes through the left join. Without it, it is still read from the
foreign key.

A sort by a property of a collection keeps its inner join: Hibernate
fetches a collection through a left join, which is handled separately.

Part 5 of the split of apache#16853.
A where query, a detached criteria and a dynamic finder joined an
association as soon as criteria, a projection or a sort needed it. The
rest of the query can still change after that: criteria passed to list
are added later, and the fetch argument of list, which can request a
join type, is read last. The result depended on the order in which the
query was put together:

- a join type in the fetch argument of list was not used for the join
  that the criteria or a projection had already made;
- a sort stored on the query with sort() was joined before a join type
  requested in list was known, and criteria passed to list could not
  join the same association through an alias;
- a join type was looked up by the name of the association, so a type
  requested for club was used for team.club, and one requested for
  team.club was not;
- criteria on a nested association used twice failed with
  "duplicate association path".

Criteria, projections and orders only need the alias of a join. A join
is now registered under its path from the root entity and its alias,
and made when the query is executed, or when a subquery is handed to
the query that embeds it, with the join type that the query requests
for that path. A listener of the query events that adds criteria on an
association to find and findAll with an example, findWhere or
findAllWhere gets its joins made after the event.

Part 6 of the split of apache#16853.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested changes. An inner join the query asked for is missing when the sort uses the foreign key.

associationPath.append(token);
String path = associationPath.toString();
Association association = (Association) persistentProperty;
boolean leftJoin = GrailsHibernateUtil.isLeftJoinAssociationSort(entity);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A player whose team has no club is queried with an inner join of team.club and a sort by that path. The player should not come back. It does, because the sort uses the foreign key and never makes the inner join. Make the join the query asked for.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a sort ending in team.club.id, reading the foreign key means that the sort does not need a join of club. Applying a requested join that neither criteria nor the sort needs is part 7, #17067. That is why this case still returns the player at part 6.

Please target this finding to #17067, where the requested join is materialized independently of the sort. This part is limited to when the joins a query needs are created, and to looking up their requested type by the complete association path. Keeping those changes separate is the purpose of the split.

@matrei matrei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 7786cbc (base 7.0.x): 0309713 and the merge of #17064 (5dc4d34) into it. I probed where queries, detached criteria and finders that request a join type, sort, or have criteria and projections on an association:

  • at this head and at 0309713, at #17065 (310a227), on 7.0.x (aee4c04) and at the stack top (e359719), with the setting off and on;
  • with Hibernate 7 on 8.0.x (0518f48);
  • with a team that has no club and a player whose team has no club.

Every query returns the same at this head as at 0309713, with the setting off and on. With the setting off, only queries that request a join type return other rows than at #17065. Every other query returns what it did there, except for calls that threw.

Setting off 7.0.x #17065 This head Hibernate 7
Team.where { club.name == 'Chelsea' || name == 'Unattached First Team' }.list(fetch: [club: JoinType.LEFT]), also count(fetch: …), findAll(fetch: …) { … } and a detached criteria 1 team 1 2 2
the same with max: 1, totalCount 1 1 2 2
Team.where { name ==~ '%First%' }.property('club.name').list(fetch: [club: JoinType.INNER]) 4 values, with null 4 3 3
Player.where { team.club.name == 'Arsenal' || name == 'Unattached Player' }.join('team.club', JoinType.LEFT).list(), also fetch: ['team.club': JoinType.LEFT] 1 player 1 2 2
the same with join('club', JoinType.LEFT) 2 players 2 1 1
Team.where { name == 'Unattached First Team' }.sort('club.name').list(fetch: [club: JoinType.LEFT]) nothing nothing the team the team
Player.where { name ==~ '%Player%' }.sort('team.club.name').list(fetch: ['team.club': JoinType.LEFT]) throws 3 players 4 4

Each new result is what Hibernate 7 returns on 8.0.x. The old result ignores a join type that the call requests. So I agree that this needs no setting, by #17061: the old result contradicts the call, and another API of the framework already returns the new one. After the merge-up, Hibernate 5 and Hibernate 7 agree on these calls.

These calls threw and work here, so they need no note:

  • Player.where { team.club.name ==~ '%e%' && team.club.name != 'Chelsea' }.list(), and two where blocks on team.club: duplicate association path on 7.0.x and at #17065;
  • Team.where { … }.sort('club.name').list { createAlias('club', 'c'); like('c.name', '%e%') };
  • the projection of ground.address.city in the new feature of AssociationSortSpec: could not resolve property at #17065.

With the setting on, the where query cases of my first point on #17065 return here what they return with the setting off. Those cases are a requested inner join with sort(), also with find, a nested sort, and criteria that list adds. Team.list(sort: 'club.name', fetch: [club: JoinType.INNER]) still returns the team without a club with the setting on. That's for #17069 and #17071, as said there.

Needs a fix

1. A sort stored with sort() is missing from "Results that change", the note and the skill entry. The commit message names it, and so do three of the new features. But the description and the note only list criteria and projections. In the last two rows of the table, a sort stored with sort() now uses the join type that the fetch argument requests, and the query returns the instances without the association.

The note's sentence "it no longer matters in which order the query was put together" doesn't tell a reader that rows change. A third bullet would, for example: "A sort stored on the query with sort(), as in Book.where { … }.sort('author.name').list(fetch: [author: JoinType.LEFT]), now joins the association with the join type that the fetch argument requests, and also returns the books without an author."

2. The note's list doesn't render as a list. The bullets follow the paragraph without an empty line, so Asciidoctor adds them to the paragraph. In the guide that I built at this head, upgrading.html shows them as * A join type that … within the paragraph. An empty line before the first bullet fixes it, as in the note of #17065.

3. The pull request conflicts with 7.0.x, so GitHub ran no workflow for 7786cbc. The conflicts are in upgrading60x.adoc and the skill, with notes that were merged into 7.0.x after the split. They start at #17062, so every part of the stack conflicts, while #16853 merges cleanly. A merge of 7.0.x into #17062, then up the stack, resolves them.

4. Add run ci, as on the other parts: the test jobs were skipped. Locally, the tests of grails-data-hibernate5-core pass at this head (951 tests, 55 skipped). They include the two features that #17065 skipped with the setting, which run here with it on.

The merge of #17064 keeps the RowCountProjection check of 5dc4d34. applyDeferredJoins() applies the orders that a PreQueryEvent listener adds, and then removes the orders of a count, so the count keeps their joins, as on #17064. The new feature of PreQueryListenerAssociationCriteriaSpec passes here. The description still names 0309713 as the change to review; it could add the merge.

Notes

  1. On James's comment, I agree with your reply. At this head, Player.where { name ==~ '%Player%' }.join('team.club', JoinType.INNER).list() returns the player whose team has no club with or without sort('team.club.id'): 4 players, as on 7.0.x. At the stack top, both return 3.

  2. The commit message doesn't mention two fixes that the commit makes:

    • the projection of a property of an embedded component of an association, which threw;
    • a sort by an alias that criteria passed to list define, as in sort('c.name', 'desc'), which was ignored: the rows came back in the order of their ids.
  3. Some changes have nothing to do with the fix:

    • The imports of AssociationJoinTypeSpec, AssociationSortSpec and WhereQueryWithAssociationSortSpec are reordered: the spock.lang imports move above grails.gorm, Requires before IgnoreIf, and Issue stays where it was.
    • HibernateGormStaticApi gets a blank line among its imports.
    • getAssociation(String) is split off getAssociatedEntity(String), its only caller.

    I'd revert them.

  4. Not for this pull request: with Hibernate 7 on 8.0.x, a projection of a property of an association leaves out the rows without the association. Team.where { name ==~ '%First%' }.property('club.name').list() returns 3 values there, and 4, with null, with Hibernate 5, here as on 7.0.x. The same goes for groupProperty('club.name'). By #17061 that's a Hibernate 7 result to bring in line with Hibernate 5, in an issue of its own.

  5. As a fix that changes a result, this needs a second reviewer (#17061).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants