Repository navigation
Add a setting that keeps the rows without an association when sorting by it - #17065
jdaugherty wants to merge 11 commits into
Conversation
Sorting a where query by a property of an association, such as sort: 'club.name', failed with a QueryException when the query also had criteria on that association. The criteria join the association under a generated alias, and the order was added with the association path, which Hibernate cannot resolve. The order now uses the alias of the join.
A sort by a path through more than one association, such as owner.employer.name, is still not supported.
This reverts commit b494caa.
A sort by a path that crosses more than one association, such as team.club.name, failed with a QueryException whether or not the where query joined the nested association, because order() split the path at its first dot only. The sort property is now qualified with the alias of the last association on its path: an association the query has already joined is reused, and one it has not joined yet is joined for the sort and registered like any other join. A sort by two properties of the same association no longer fails with "duplicate association path" either, since the sort join is registered instead of created anew for every order.
Since order() walks the whole sort path, a sort by the identifier of an association that follows another one on the path, such as team.club.id, joined that last association too. The join is an inner join unless fetch requests another type, so the sort left out the rows where the association is null. Before, only the first association was joined and Hibernate read club.id from the foreign key of team, which kept them. The identifier is sorted by the foreign key again when the entity before the association holds it. An association without such a foreign key, a hasOne or the inverse side of a one-to-one, is still joined, and so is the first association of a path, as it always has been. The spec also covers the join type of a sort join: an inner join by default, and a left join when fetch requests one. Team.club is nullable in the fixture for these cases. The docs describe the inner join, how to request a left join, and the identifier of a nested association.
list with a map of sort properties, such as Team.list(sort: ['club.name': 'asc', name: 'desc']), sorted by the last entry only, because each entry replaced the orders of the query. The same held for a default sort that the mapping declares for more than one property. Every entry is now applied, in the order of the map. An association is joined once for all the sort properties that use it. list joined it once for each, and find and findAll with an example, findWhere and findAllWhere failed with "duplicate association path". Part 2 of the split of apache#16853.
A paged query of a detached criteria that has an alias for the queried
class, and criteria that use it, failed when totalCount was read:
new DetachedCriteria(Team, 't').eq('t.name', 'Arsenal').list(max: 2).totalCount
The count runs on a copy of the criteria, and the copy was created
without the alias, so Hibernate could not resolve t.name. The copy now
has the alias of the original.
Part 3 of the split of apache#16853.
count on a where query or a detached criteria that has a sort, such as
Team.where { ... }.sort('name').count(), kept the order by in the query
that counts the rows, which H2 rejects with "must be in the GROUP BY
list". A query that only counts its rows now has no orders.
The join that a sort by a property of an association makes is kept, so
the count is that of the rows which list returns for the same query.
Part 4 of the split of apache#16853.
… by it A sort by a property of an association joins the association with an inner join, so it leaves out the instances whose association is null: Team.list(sort: 'club.name') does not return the teams without a club, and count and the totalCount of a paged list do not count them. The new grails.gorm.leftJoinAssociationSort setting makes an association that is joined only for a sort a left join, in where queries, detached criteria, dynamic finders, list, find and findAll with an example, findWhere, findAllWhere and a criteria query that is passed pagination arguments. It is disabled by default, so the results of existing applications do not change in a patch release. Grails 8.1 enables it by default. With the setting, a sort by a nested identifier, such as team.club.id, goes through the left join. Without it, it is still read from the foreign key. A sort by a property of a collection keeps its inner join: Hibernate fetches a collection through a left join, which is handled separately. Part 5 of the split of apache#16853.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A sort that should keep a row with no association still drops it when the call also fetches that association eagerly.
| for (Object existing : from.getJoins()) { | ||
| Join join = (Join) existing; | ||
| if (associationName.equals(join.getAttribute().getName())) { | ||
| return join; |
There was a problem hiding this comment.
The setting that keeps a row with no association is on. The list sorts by club.name and fetches club eagerly. A team with no club should stay. It is dropped, because the eager fetch joins club first and the sort reuses that join. Make the sort's join the one the setting asks for.
There was a problem hiding this comment.
If this is Team.list(sort: "club.name", fetch: [club: "eager"]), the eager-fetch join of static list is still the old implementation at this point in the stack. Changing that fetch join is part 11, #17071, including the Hibernate 5 counterpart of #16898. It should be reviewed there rather than pulling that change into the sort-setting PR.
The setting controls a join made only for sorting; it does not replace a join that the query already has. If you meant an eager fetch declared in the mapping rather than the fetch argument, could you post that mapping and the public GORM call so we can identify the owning part?
matrei
left a comment
There was a problem hiding this comment.
Reviewed at 310a227 (base 7.0.x), the last commit. I probed sorts with the setting off and on:
- at this head, on 7.0.x (aee4c04) and at the stack top (e359719);
- with a team that has no club and a player whose team has no club.
With the setting off, every result is the one of 7.0.x, except for the calls that throw on 7.0.x. With the setting on, a sort keeps the instances without the association:
| Setting off, and 7.0.x | Setting on | |
|---|---|---|
Team.list(sort: 'club.name'), also sort: 'club.id' |
2 teams | 3 |
Team.list(sort: 'club.name', max: 2).totalCount |
2 | 3 |
Player.list(sort: 'team.club.name'), also team.club.id |
1 player | 2 |
a where query, a dynamic finder and a paged criteria query, sorted by club.name with fetch: [club: 'join'] |
2 teams | 3 |
a mapping with sort 'club.name', and a mapping with club fetch: 'join' and a sort |
1 team | 2 |
Team.list(sort: 'players.name') |
2 teams | 2: the collection keeps its inner join |
Needs a fix
1. With the setting on, the sort's left join wins over a join that the query asks for. The description says that an association is joined with a left join "unless the query requests another join type for it". The upgrade note says that criteria on the association keep their inner join. With the setting on, at this head, neither holds:
| Setting off | Setting on | Stack top, off and on | |
|---|---|---|---|
Team.where { name == 'Unattached First Team' }.sort('club.name').list(fetch: [club: JoinType.INNER]), also findAllWhere |
nothing | the team | nothing |
Team.list(sort: 'club.name', fetch: [club: JoinType.INNER]), also a paged criteria query |
2 teams | 3 | 2 |
Team.where { name ==~ '%First%' }.sort('club.name').list { club { isNull('name') } }, also count { … } |
nothing | the team | nothing |
the same with or { club { eq('name', 'Arsenal') }; eq('name', 'Unattached First Team') } |
1 team | 2 | 1 |
The features that cover these cases are skipped with the setting, through @IgnoreIf({ instance.leftJoinSort }):
- two in
WhereQueryWithAssociationSortSpec, until #17066; - three in
AssociationJoinTypeSpec, until #17069 and #17071.
So with the setting on, this part on its own returns rows that the later parts take away again. Either it merges together with those three, into the same release, or it moves after them in the stack. The description should say which.
2. The settings page. #17061 names the page Compatibility Settings (grails-doc/src/en/guide/upgrading/compatibilitySettings.adoc). This page is resultSettings.adoc, titled "Settings for Fixes That Change a Result". This is the first setting, so this part creates the page, and the name should match.
The enum row doesn't fit the column "Default in Grails 7 and 8.0": 8.0 has no such setting, since 8.0.0 removed the legacy marshaller. A default for each release line, as #17061 lists it, would fit both rows.
3. The new links don't resolve. The guide has no guide/upgrading60x.html and no guide/resultSettings.html. A section is on the page of its chapter and on its own page under guide/pages/.
I built the guide at this head. Three links point to pages that don't exist, on upgrading.html, on single.html and on the section pages:
link:resultSettings.htmlin the note;link:upgrading60x.html#grails-7-0-18and#grails-7-0-2on the page.
link:upgrading.html#grails-7-0-18[…] resolves on every page, as link:upgrading.html#_12_5_hibernate_ehcache already does in the same file.
4. The warning, the default in 8.1 and the removal in 9.0. The dev@ thread and #17061 agree on this: while an application keeps the old result through the setting, it logs a warning at startup that names the major release that removes the setting. This part has no such warning.
From 8.1.0, the warning concerns every application that sets the setting to false. Before that, it concerns one that sets it explicitly. The new default and the removal don't come with a merge-up either. Please say in the description which change does each:
- the warning;
- the default in 8.1.x, with its notes and the row of the page;
- the removal in 9.0.x, for Hibernate 5 and for #16863.
An issue for each would keep them from being forgotten.
5. Add run ci, as on the other parts: the test jobs were skipped. Locally, the tests of grails-data-hibernate5-core pass at this head (907 tests, 57 skipped).
Notes
-
On James's comment: with the setting on, the team is dropped here, but the sort isn't the cause.
Team.list(fetch: [club: 'join'])joins the club with an inner join with no sort at all. It returns 2 teams on 7.0.x and here, with the setting on or off.- The sort reuses that join, so it leaves out only what the fetch already leaves out.
- #17071 fetches the club through a left join instead. At the stack top, the list keeps the team with or without the sort.
- A mapping with
club fetch: 'join'keeps the team here already.
So there's nothing to change in this part for it. At the stack top,
Team.list(sort: 'club.name', fetch: [club: 'join'])also returns 3 teams with the setting off, which I'll look at on #17071. -
As a fix that changes a result, this needs a second reviewer (#17061).
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A plain join the call asked for is reused by a sort, so a row with no association is left out.
| for (Object existing : from.getJoins()) { | ||
| Join join = (Join) existing; | ||
| if (associationName.equals(join.getAttribute().getName())) { | ||
| return join; |
There was a problem hiding this comment.
The setting that keeps a row with no association is on. Team.list(sort: 'club.name', fetch: [club: 'join']) should include Unattached First Team. It is left out, because the plain join is an inner join and the sort reuses it. Keep the row unless the call asked for an inner join.
Description
Part 5 of the split of #16853, as agreed there and in the dev@ thread Fixes that change results on patch and minor branches. Relates to #15589.
Stacked on #17064. Until that merges, this pull request also shows the commits below it. The change to review is the last commit, 310a227.
A sort by a property of an association joins the association with an
inner join, so it leaves out the instances whose association is null:
Team.list(sort: 'club.name') does not return the teams without a club,
and count and the totalCount of a paged list do not count them.
The new grails.gorm.leftJoinAssociationSort setting makes an association
that is joined only for a sort a left join, in where queries, detached
criteria, dynamic finders, list, find and findAll with an example,
findWhere, findAllWhere and a criteria query that is passed pagination
arguments. It is disabled by default, so the results of existing
applications do not change in a patch release. Grails 8.1 enables it by
default.
With the setting, a sort by a nested identifier, such as team.club.id,
goes through the left join. Without it, it is still read from the
foreign key.
A sort by a property of a collection keeps its inner join: Hibernate
fetches a collection through a left join, which is handled separately.
Results that change
grails.gorm.leftJoinAssociationSort: true, a sort by a property of an association joins it with a left join, solist,countandtotalCountkeep the instances whose association isnull.grails.gorm.leftJoinAssociationSort, off by default, on by default from 8.1.0, removed in 9.0.0. Every GORM API returns the old result today and many applications sort a list by an association, so this follows the setting rule from the dev@ thread. The Hibernate 7 fix (Keep the rows without an association when sorting by it with Hibernate 7 #16863) reads the same key.The upgrade guide has a note under Grails 7.0.18, and the upgrade skill has the same entry.
Verification
The association sort and join specs of
grails-data-hibernate5-corepass on this branch, withgrails.gorm.leftJoinAssociationSortoff and on, andcodeStylepasses for the touched modules. The full build was not run locally.Contributor Checklist
Please review the following checklist before submitting your pull request. Pull requests that do not meet these requirements may be closed without review.
Issue and Scope
7.0.x): Bug fixes only. No new features or API changes.7.1.x): New features are welcome, but breaking existing APIs must be avoided.8.0.x): Reserved for major changes. Breaking API changes are permitted.Code Quality
./gradlew build --rerun-tasks../gradlew codeStyleand resolved any violations. See Code Style for details.Licensing and Attribution
Documentation