Repository navigation
Apply a requested join type without criteria or a sort - #17067
jdaugherty wants to merge 15 commits into
Conversation
Sorting a where query by a property of an association, such as sort: 'club.name', failed with a QueryException when the query also had criteria on that association. The criteria join the association under a generated alias, and the order was added with the association path, which Hibernate cannot resolve. The order now uses the alias of the join.
A sort by a path through more than one association, such as owner.employer.name, is still not supported.
This reverts commit b494caa.
A sort by a path that crosses more than one association, such as team.club.name, failed with a QueryException whether or not the where query joined the nested association, because order() split the path at its first dot only. The sort property is now qualified with the alias of the last association on its path: an association the query has already joined is reused, and one it has not joined yet is joined for the sort and registered like any other join. A sort by two properties of the same association no longer fails with "duplicate association path" either, since the sort join is registered instead of created anew for every order.
Since order() walks the whole sort path, a sort by the identifier of an association that follows another one on the path, such as team.club.id, joined that last association too. The join is an inner join unless fetch requests another type, so the sort left out the rows where the association is null. Before, only the first association was joined and Hibernate read club.id from the foreign key of team, which kept them. The identifier is sorted by the foreign key again when the entity before the association holds it. An association without such a foreign key, a hasOne or the inverse side of a one-to-one, is still joined, and so is the first association of a path, as it always has been. The spec also covers the join type of a sort join: an inner join by default, and a left join when fetch requests one. Team.club is nullable in the fixture for these cases. The docs describe the inner join, how to request a left join, and the identifier of a nested association.
list with a map of sort properties, such as Team.list(sort: ['club.name': 'asc', name: 'desc']), sorted by the last entry only, because each entry replaced the orders of the query. The same held for a default sort that the mapping declares for more than one property. Every entry is now applied, in the order of the map. An association is joined once for all the sort properties that use it. list joined it once for each, and find and findAll with an example, findWhere and findAllWhere failed with "duplicate association path". Part 2 of the split of apache#16853.
A paged query of a detached criteria that has an alias for the queried
class, and criteria that use it, failed when totalCount was read:
new DetachedCriteria(Team, 't').eq('t.name', 'Arsenal').list(max: 2).totalCount
The count runs on a copy of the criteria, and the copy was created
without the alias, so Hibernate could not resolve t.name. The copy now
has the alias of the original.
Part 3 of the split of apache#16853.
count on a where query or a detached criteria that has a sort, such as
Team.where { ... }.sort('name').count(), kept the order by in the query
that counts the rows, which H2 rejects with "must be in the GROUP BY
list". A query that only counts its rows now has no orders.
The join that a sort by a property of an association makes is kept, so
the count is that of the rows which list returns for the same query.
Part 4 of the split of apache#16853.
… by it A sort by a property of an association joins the association with an inner join, so it leaves out the instances whose association is null: Team.list(sort: 'club.name') does not return the teams without a club, and count and the totalCount of a paged list do not count them. The new grails.gorm.leftJoinAssociationSort setting makes an association that is joined only for a sort a left join, in where queries, detached criteria, dynamic finders, list, find and findAll with an example, findWhere, findAllWhere and a criteria query that is passed pagination arguments. It is disabled by default, so the results of existing applications do not change in a patch release. Grails 8.1 enables it by default. With the setting, a sort by a nested identifier, such as team.club.id, goes through the left join. Without it, it is still read from the foreign key. A sort by a property of a collection keeps its inner join: Hibernate fetches a collection through a left join, which is handled separately. Part 5 of the split of apache#16853.
A where query, a detached criteria and a dynamic finder joined an association as soon as criteria, a projection or a sort needed it. The rest of the query can still change after that: criteria passed to list are added later, and the fetch argument of list, which can request a join type, is read last. The result depended on the order in which the query was put together: - a join type in the fetch argument of list was not used for the join that the criteria or a projection had already made; - a sort stored on the query with sort() was joined before a join type requested in list was known, and criteria passed to list could not join the same association through an alias; - a join type was looked up by the name of the association, so a type requested for club was used for team.club, and one requested for team.club was not; - criteria on a nested association used twice failed with "duplicate association path". Criteria, projections and orders only need the alias of a join. A join is now registered under its path from the root entity and its alias, and made when the query is executed, or when a subquery is handed to the query that embeds it, with the join type that the query requests for that path. A listener of the query events that adds criteria on an association to find and findAll with an example, findWhere or findAllWhere gets its joins made after the event. Part 6 of the split of apache#16853.
A where query, a detached criteria or a dynamic finder can request a
join type for an association, with fetch: [club: JoinType.INNER] or
join('club', JoinType.INNER). The type was only used when criteria or
a sort joined the association. Otherwise the association was fetched
with the outer join of its fetch mode, so
Team.where { ... }.list(fetch: [club: JoinType.INNER]) returned the
teams without a club, and the same query sorted by club.name did not.
A requested join type other than LEFT is now applied when the query is
executed, to list, get, find, count, the totalCount of a paged list, a
subquery and a query with a projection:
- INNER and RIGHT on an association to a single instance join and
fetch it; a right join is a right join, not an outer fetch;
- a path that ends in a collection or passes through one restricts the
query with a correlated subquery, so that the collection stays
fetched with all of its elements, each instance is counted once, and
the filters of the session, such as the tenant filter, apply;
- every association on a requested path is joined and fetched;
- a collection of a basic type is handled like any other collection;
- a fetch mode requested afterwards replaces the join type, on the
query and on the detached criteria, while the join that criteria on
the association make keeps it.
Part 7 of the split of apache#16853.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. An inner join with no criteria and no sort still keeps a row that has no child.
| for (Object o : fetch.keySet()) { | ||
| String associationName = (String) o; | ||
|
|
||
| final FetchMode fetchMode = getFetchMode(fetch.get(associationName)); |
There was a problem hiding this comment.
Team.list(fetch: [club: JoinType.INNER]) should leave out a team with no club. The team stays, because the inner join is read as a fetch mode and no join is made. Make the join the call asked for.
There was a problem hiding this comment.
Team.list(fetch: [club: JoinType.INNER]) is the static-list path addressed by part 11, #17071. This part covers where queries, detached criteria and dynamic finders; it does not change static list or query-by-example.
Please target this finding to #17071 rather than adding the static-list implementation here. Both pieces remain subject to the same merge ordering agreed on dev@: the Hibernate 7 rule and spec in #17079 must land on 8.0.x before the requested-join parts merge into 7.0.x.
matrei
left a comment
There was a problem hiding this comment.
Reviewed at 1814db6 (base 7.0.x): 7fc46a7 and the merge of #17064, which leaves the diff of 7fc46a7 as it was. I probed where queries, detached criteria, finders, data services, subqueries and bulk updates that request a join type:
- at this head, at #17066 (7786cbc), on 7.0.x (aee4c04) and at the stack top (16bd465), with the setting off and on;
- with Hibernate 7 on 8.0.x (0518f48) and at #17079 (6de9334);
- with a team without a club, a team without players, a team with three players of which one has no contract, and clubs that no team refers to.
The setting changes none of these results, and #17066 returns what 7.0.x returns for all of them. This part changes these:
| 7.0.x and #17066 | This head | Hibernate 7 on 8.0.x | #17079 | |
|---|---|---|---|---|
Team.where { name ==~ '%First%' }.list(fetch: [club: JoinType.INNER]), also join('club', JoinType.INNER), findAllByNameLike('%First%', [fetch: …]) and findAll(fetch: …) { … } |
5 teams | 4 | 4 | 4 |
the same with count(), the totalCount of max: 2, projections { count() } and property('name') |
5 | 4 | 5 | 4 |
Team.where { name == 'Unattached First Team' }.join('club', JoinType.INNER).get(), also find(fetch: …) |
the team | null |
null |
null |
fetch: [players: JoinType.INNER], also JoinType.RIGHT |
5 | 4 | 4 | 4 |
the same with count() and totalCount |
5 | 4 | 5 | 4 |
Player.where { … }.join('team.club', JoinType.INNER).list(), also count() and totalCount |
7 players | 6 | 7 | 6 |
Team.where { name ==~ '%Team%' }.list(fetch: ['players.contract': JoinType.INNER], max: 3).totalCount |
7 | 5 | 7 | 5 |
a subquery, as in new DetachedCriteria(Team).inList('id', Team.where { … }.join('club', JoinType.INNER).id()) |
5 | 4 | 5 | 4 |
So I agree that this needs no setting, by #17061: the old result ignores a join type that the call requests. For the list of a where query, Hibernate 7 on 8.0.x already returns the new rows, while its count and totalCount don't, which #17079 fixes. Apart from points 1, 2 and 6, and the projections that #17079 lists as different, this head returns what #17079 returns.
Needs a fix
1. A page of a query that requests a join type for a path through a collection holds collections without all of their elements. This head fetches the collection on such a path, such as players for fetch: ['players.contract': JoinType.INNER]. With max, Hibernate 5 limits the rows of the join, so a collection on the page holds only the elements in those rows:
Team.where { name ==~ '%Team%' }.list(fetch: ['players.contract': JoinType.INNER], max: 2, sort: 'name', order: 'desc') |
7.0.x | This head | #17079 |
|---|---|---|---|
players of the team with three players |
not fetched, 3 players when loaded | fetched, with 1 of its 3 players | fetched, with its 3 players |
The same happens with:
JoinType.LEFT;maxwithoffset;- a detached criteria with
join('players.contract', JoinType.INNER); Player.where { … }.list(fetch: ['team.players': JoinType.LEFT], max: 2, sort: 'name', order: 'desc'): the team with three players holds 1 of them.
The note says that a join type "never leaves elements out of a fetched collection". A fetch of the collection itself, as in fetch: [players: JoinType.LEFT] with max, already does this on 7.0.x, where Hibernate 7 on 8.0.x fetches all 3 players. These paths didn't fetch the collection on 7.0.x, though. Not fetching a collection on the path of a paged query keeps the result of 7.0.x.
2. A right join returns one null for all the clubs that no team refers to, and counts each of them. With two such clubs, new DetachedCriteria(Team).join('club', JoinType.RIGHT) returns:
| This head | Hibernate 7 on 8.0.x | #17079 | |
|---|---|---|---|
list() |
3 teams and one null |
3 teams and two null |
3 teams and two null |
count(), and the totalCount of max: 10 |
5 | 4 | 5 |
So count doesn't return the number of rows that list returns, which the note of #17064 says it does, and Hibernate 5 and Hibernate 7 return different lists after the merge-up. The note describes the single null. A null for each row, as Hibernate 7 returns it, would make list, count and both modules agree.
3. "Results that change" lists only the inner join for list, get, find, count and totalCount. The note has more, and the description should list it too:
- a right join;
- a subquery and a property projection;
- a path through more than one association, and a path through a collection;
- a left join for a path, which now fetches the associations on it;
- a collection of a basic type;
- a data service method with
@Join(point 4).
Two changes are in neither:
Team.where { club.name == 'Chelsea' || name == 'Unattached First Team' }.join('club', JoinType.LEFT).select('club').list()returned 1 team on 7.0.x and returns 2 here, as at #17079. The left join is now kept for the criteria afterselect. The persistence guide describes it, but no note says that it changed.Team.where { name ==~ '%First%' }.property('club.name').join('club', JoinType.INNER).join('club').list()returned 3 values on 7.0.x and returns 4 here, withnull. The fetch mode now replaces the inner join for the projection too. Hibernate 7 leaves thenullout, as it does for every projection of a property of an association (note 8 on #17066).
4. @Join on a data service. The note says that a data service method that queries by its parameters, such as Book find(String title), applies the join type of @Join(value = 'author', type = JoinType.INNER). The skill entry says to search for @Join, and that each of these queries now applies the join type. It holds for such a method. With @Join(value = 'club', type = JoinType.INNER), Team find(String name) returns null here for the team without a club, List<Team> listTeams(String name) returns nothing and Number count(String name) returns 0. On 7.0.x they returned the team, and 1.
A method named like a dynamic finder doesn't apply it. findByName, findAllByNameLike and countByNameLike with @Join(value = 'club', type = JoinType.INNER) still return the team without a club, at this head, at the stack top and at #17079. FindAllByImplementer.buildArgs passes fetch: [club: 'join'] and drops the type. Either the note and the skill entry say that such a method doesn't apply the type, or the implementer passes it on, here and in #17079.
5. New public API on 7.0.x. This part adds:
AbstractDetachedCriteria.getSupersededJoinTypes(), with a protected field, on every detached criteria of every datastore;- the public static
GrailsHibernateUtil.fetchAssociationsBeforeandjoinedInSubquery.
The Hibernate skill asks for no new public or protected methods on 7.0.x and 8.0.x. Only AbstractHibernateQuery calls the two helpers, so they can be private there. DynamicFinder and HibernateCriteriaBuilder read the getter. If it has to stay public, the description should say why. DetachedCriteriaJoinTypeSpec tests that map, not what a query returns (rule 9 of AGENTS.md, "Test via public APIs").
6. This part should merge together with #17068, #17069 and #17071. #17079 makes all four changes for Hibernate 7.
Without #17068, updateAll and deleteAll change rows that list of the same query leaves out:
| This head | Stack top and #17079 | |
|---|---|---|
Team.where { name ==~ '%First%' }.join('club', JoinType.INNER).updateAll(name: 'Renamed'), whose list has 4 teams |
5 updated | 4 |
the same with join('players', JoinType.INNER) |
5 | 4 |
Team.where { name == 'Empty Team' || name == 'Playerless First Team' }.join('club', JoinType.INNER).deleteAll(), one of them without a club |
2 deleted | 1 |
Without #17071, these keep the rows of 7.0.x here, while the stack top and #17079 apply the join type:
Team.list(fetch: [club: JoinType.INNER]): 7 teams, against 5;Player.list(fetch: ['team.club': JoinType.INNER]): 7 players, against 6;findAllWhere([name: 'Unattached First Team'], [fetch: [club: JoinType.INNER]]): the team, against nothing.
The description says that this merges once #17079 is on 8.0.x. It should also say that the three parts merge with it, into the same release. Otherwise the Hibernate 5 module of 8.0.x differs from Hibernate 7 after the merge-up.
7. The pull request conflicts with 7.0.x, and needs run ci, as on #17066. Locally, the tests pass at this head: grails-data-hibernate5-core has 1052 tests, 55 skipped, and grails-datamapping-core has 257 tests, 1 skipped. The guide builds, and the note's list renders as a list.
Notes
-
On James's comment, I agree with your reply.
Team.list(fetch: [club: JoinType.INNER])returns 7 teams here. At the stack top it returns 5, as at #17079. -
The imports of
AssociationSortSpecandWhereQueryWithAssociationSortSpecare reordered again, which has nothing to do with the fix. -
DetachedCriteriaJoinTypeSpecuses double-quoted feature names anddeffor locals. New features take single quotes andfinal. -
As a fix that changes a result, this needs a second reviewer (#17061).
Description
Part 7 of the split of #16853, as agreed there and in the dev@ thread Fixes that change results on patch and minor branches. Relates to #15589.
Stacked on #17066. Until that merges, this pull request also shows the commits below it. The change to review is the last commit, 7fc46a7.
A where query, a detached criteria or a dynamic finder can request a
join type for an association, with fetch: [club: JoinType.INNER] or
join('club', JoinType.INNER). The type was only used when criteria or
a sort joined the association. Otherwise the association was fetched
with the outer join of its fetch mode, so
Team.where { ... }.list(fetch: [club: JoinType.INNER]) returned the
teams without a club, and the same query sorted by club.name did not.
A requested join type other than LEFT is now applied when the query is
executed, to list, get, find, count, the totalCount of a paged list, a
subquery and a query with a projection:
fetch it; a right join is a right join, not an outer fetch;
query with a correlated subquery, so that the collection stays
fetched with all of its elements, each instance is counted once, and
the filters of the session, such as the tenant filter, apply;
query and on the detached criteria, while the join that criteria on
the association make keeps it.
Results that change
JoinType.LEFTthat a where query, a detached criteria or a dynamic finder requests is applied also when nothing else in the query uses the association:JoinType.INNERleaves out the instances without the association fromlist,get,find,countandtotalCount.JoinType.LEFT.The upgrade guide has a note under Grails 7.0.18, and the upgrade skill has the same entry.
Verification
The association sort and join specs of
grails-data-hibernate5-corepass on this branch, withgrails.gorm.leftJoinAssociationSortoff and on, andcodeStylepasses for the touched modules. The full build was not run locally.Contributor Checklist
Please review the following checklist before submitting your pull request. Pull requests that do not meet these requirements may be closed without review.
Issue and Scope
7.0.x): Bug fixes only. No new features or API changes.7.1.x): New features are welcome, but breaking existing APIs must be avoided.8.0.x): Reserved for major changes. Breaking API changes are permitted.Code Quality
./gradlew build --rerun-tasks../gradlew codeStyleand resolved any violations. See Code Style for details.Licensing and Attribution
Documentation