Skip to content

Add a setting that keeps the rows without an association when sorting by it - #17065

Open
jdaugherty wants to merge 11 commits into
apache:7.0.xfrom
jdaugherty:fix/16853-05-sort-keeps-rows-setting
Open

jdaugherty wants to merge 11 commits into
apache:7.0.xfrom
jdaugherty:fix/16853-05-sort-keeps-rows-setting

Conversation

@jdaugherty

Copy link
Copy Markdown
Contributor

Description

Part 5 of the split of #16853, as agreed there and in the dev@ thread Fixes that change results on patch and minor branches. Relates to #15589.

Stacked on #17064. Until that merges, this pull request also shows the commits below it. The change to review is the last commit, 310a227.

A sort by a property of an association joins the association with an
inner join, so it leaves out the instances whose association is null:
Team.list(sort: 'club.name') does not return the teams without a club,
and count and the totalCount of a paged list do not count them.

The new grails.gorm.leftJoinAssociationSort setting makes an association
that is joined only for a sort a left join, in where queries, detached
criteria, dynamic finders, list, find and findAll with an example,
findWhere, findAllWhere and a criteria query that is passed pagination
arguments. It is disabled by default, so the results of existing
applications do not change in a patch release. Grails 8.1 enables it by
default.

With the setting, a sort by a nested identifier, such as team.club.id,
goes through the left join. Without it, it is still read from the
foreign key.

A sort by a property of a collection keeps its inner join: Hibernate
fetches a collection through a left join, which is handled separately.

Results that change

  • What changes: Nothing by default. With grails.gorm.leftJoinAssociationSort: true, a sort by a property of an association joins it with a left join, so list, count and totalCount keep the instances whose association is null.
  • How to get the old result: Leave the setting off, which is the default on 7.0.x, 7.1.x, 7.2.x and 8.0.x.
  • Setting: Yes: grails.gorm.leftJoinAssociationSort, off by default, on by default from 8.1.0, removed in 9.0.0. Every GORM API returns the old result today and many applications sort a list by an association, so this follows the setting rule from the dev@ thread. The Hibernate 7 fix (Keep the rows without an association when sorting by it with Hibernate 7 #16863) reads the same key.

The upgrade guide has a note under Grails 7.0.18, and the upgrade skill has the same entry.

Verification

The association sort and join specs of grails-data-hibernate5-core pass on this branch, with grails.gorm.leftJoinAssociationSort off and on, and codeStyle passes for the touched modules. The full build was not run locally.

Contributor Checklist

Please review the following checklist before submitting your pull request. Pull requests that do not meet these requirements may be closed without review.

Issue and Scope

  • This PR is linked to an existing issue that has been acknowledged or approved by the project team. If no approved issue exists, please give background on why this change is necessary. Tickets are preferred for release change log history.
  • This PR addresses the complete scope of the linked issue. Partial implementations or unfinished work should not be submitted for review.
  • This PR contains a single, focused change. Unrelated changes should be submitted as separate pull requests.
  • This PR targets the correct branch for the type of change:
    • Patch release branches (e.g., 7.0.x): Bug fixes only. No new features or API changes.
    • Minor release branches (e.g., 7.1.x): New features are welcome, but breaking existing APIs must be avoided.
    • Major release branches (e.g., 8.0.x): Reserved for major changes. Breaking API changes are permitted.

Code Quality

  • I have added or updated tests that cover the changes introduced in this PR. All code contributions are expected to include appropriate test coverage.
  • I have verified that all existing tests pass by running ./gradlew build --rerun-tasks.
  • My code follows the project's code style guidelines. I have run ./gradlew codeStyle and resolved any violations. See Code Style for details.
  • This PR does not include mass reformatting, style-only changes, or large-scale refactoring unless it was explicitly approved in the linked issue. Unsolicited reformatting will not be accepted.
  • If generative AI tooling was used in preparing this contribution, a quality model was used to ensure contributions are consistent with the project's quality standards.

Licensing and Attribution

Documentation

  • If this PR introduces user-facing changes, I have included or updated the relevant documentation.
  • If this PR adds a new feature, I have updated the What's New section of the Grails Guide.
  • If this PR introduces breaking changes or changes that require user action during an upgrade, I have updated the Upgrade Notes for the corresponding version in the Grails Guide.
  • The PR description clearly explains what was changed and why.

First-time contributors: Please read our Contributing Guide before submitting.
Pull requests that appear to be auto-generated, incomplete, or unrelated to an approved issue may be
closed to help maintainers focus on reviewed and planned work. We appreciate your understanding.

matrei and others added 9 commits November 19, 2026 09:30
Sorting a where query by a property of an association, such as
sort: 'club.name', failed with a QueryException when the query also
had criteria on that association. The criteria join the association
under a generated alias, and the order was added with the association
path, which Hibernate cannot resolve. The order now uses the alias of
the join.
A sort by a path through more than one association, such as
owner.employer.name, is still not supported.
A sort by a path that crosses more than one association, such as
team.club.name, failed with a QueryException whether or not the where
query joined the nested association, because order() split the path at
its first dot only. The sort property is now qualified with the alias of
the last association on its path: an association the query has already
joined is reused, and one it has not joined yet is joined for the sort
and registered like any other join. A sort by two properties of the same
association no longer fails with "duplicate association path" either,
since the sort join is registered instead of created anew for every
order.
Since order() walks the whole sort path, a sort by the identifier of an
association that follows another one on the path, such as team.club.id,
joined that last association too. The join is an inner join unless fetch
requests another type, so the sort left out the rows where the
association is null. Before, only the first association was joined and
Hibernate read club.id from the foreign key of team, which kept them.

The identifier is sorted by the foreign key again when the entity before
the association holds it. An association without such a foreign key, a
hasOne or the inverse side of a one-to-one, is still joined, and so is
the first association of a path, as it always has been.

The spec also covers the join type of a sort join: an inner join by
default, and a left join when fetch requests one. Team.club is nullable
in the fixture for these cases. The docs describe the inner join, how to
request a left join, and the identifier of a nested association.
list with a map of sort properties, such as
Team.list(sort: ['club.name': 'asc', name: 'desc']), sorted by the last
entry only, because each entry replaced the orders of the query. The
same held for a default sort that the mapping declares for more than
one property. Every entry is now applied, in the order of the map.

An association is joined once for all the sort properties that use it.
list joined it once for each, and find and findAll with an example,
findWhere and findAllWhere failed with "duplicate association path".

Part 2 of the split of apache#16853.
A paged query of a detached criteria that has an alias for the queried
class, and criteria that use it, failed when totalCount was read:

    new DetachedCriteria(Team, 't').eq('t.name', 'Arsenal').list(max: 2).totalCount

The count runs on a copy of the criteria, and the copy was created
without the alias, so Hibernate could not resolve t.name. The copy now
has the alias of the original.

Part 3 of the split of apache#16853.
count on a where query or a detached criteria that has a sort, such as
Team.where { ... }.sort('name').count(), kept the order by in the query
that counts the rows, which H2 rejects with "must be in the GROUP BY
list". A query that only counts its rows now has no orders.

The join that a sort by a property of an association makes is kept, so
the count is that of the rows which list returns for the same query.

Part 4 of the split of apache#16853.
… by it

A sort by a property of an association joins the association with an
inner join, so it leaves out the instances whose association is null:
Team.list(sort: 'club.name') does not return the teams without a club,
and count and the totalCount of a paged list do not count them.

The new grails.gorm.leftJoinAssociationSort setting makes an association
that is joined only for a sort a left join, in where queries, detached
criteria, dynamic finders, list, find and findAll with an example,
findWhere, findAllWhere and a criteria query that is passed pagination
arguments. It is disabled by default, so the results of existing
applications do not change in a patch release. Grails 8.1 enables it by
default.

With the setting, a sort by a nested identifier, such as team.club.id,
goes through the left join. Without it, it is still read from the
foreign key.

A sort by a property of a collection keeps its inner join: Hibernate
fetches a collection through a left join, which is handled separately.

Part 5 of the split of apache#16853.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested changes. A sort that should keep a row with no association still drops it when the call also fetches that association eagerly.

for (Object existing : from.getJoins()) {
Join join = (Join) existing;
if (associationName.equals(join.getAttribute().getName())) {
return join;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setting that keeps a row with no association is on. The list sorts by club.name and fetches club eagerly. A team with no club should stay. It is dropped, because the eager fetch joins club first and the sort reuses that join. Make the sort's join the one the setting asks for.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is Team.list(sort: "club.name", fetch: [club: "eager"]), the eager-fetch join of static list is still the old implementation at this point in the stack. Changing that fetch join is part 11, #17071, including the Hibernate 5 counterpart of #16898. It should be reviewed there rather than pulling that change into the sort-setting PR.

The setting controls a join made only for sorting; it does not replace a join that the query already has. If you meant an eager fetch declared in the mapping rather than the fetch argument, could you post that mapping and the public GORM call so we can identify the owning part?

@matrei matrei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 310a227 (base 7.0.x), the last commit. I probed sorts with the setting off and on:

  • at this head, on 7.0.x (aee4c04) and at the stack top (e359719);
  • with a team that has no club and a player whose team has no club.

With the setting off, every result is the one of 7.0.x, except for the calls that throw on 7.0.x. With the setting on, a sort keeps the instances without the association:

Setting off, and 7.0.x Setting on
Team.list(sort: 'club.name'), also sort: 'club.id' 2 teams 3
Team.list(sort: 'club.name', max: 2).totalCount 2 3
Player.list(sort: 'team.club.name'), also team.club.id 1 player 2
a where query, a dynamic finder and a paged criteria query, sorted by club.name with fetch: [club: 'join'] 2 teams 3
a mapping with sort 'club.name', and a mapping with club fetch: 'join' and a sort 1 team 2
Team.list(sort: 'players.name') 2 teams 2: the collection keeps its inner join

Needs a fix

1. With the setting on, the sort's left join wins over a join that the query asks for. The description says that an association is joined with a left join "unless the query requests another join type for it". The upgrade note says that criteria on the association keep their inner join. With the setting on, at this head, neither holds:

Setting off Setting on Stack top, off and on
Team.where { name == 'Unattached First Team' }.sort('club.name').list(fetch: [club: JoinType.INNER]), also findAllWhere nothing the team nothing
Team.list(sort: 'club.name', fetch: [club: JoinType.INNER]), also a paged criteria query 2 teams 3 2
Team.where { name ==~ '%First%' }.sort('club.name').list { club { isNull('name') } }, also count { … } nothing the team nothing
the same with or { club { eq('name', 'Arsenal') }; eq('name', 'Unattached First Team') } 1 team 2 1

The features that cover these cases are skipped with the setting, through @IgnoreIf({ instance.leftJoinSort }):

  • two in WhereQueryWithAssociationSortSpec, until #17066;
  • three in AssociationJoinTypeSpec, until #17069 and #17071.

So with the setting on, this part on its own returns rows that the later parts take away again. Either it merges together with those three, into the same release, or it moves after them in the stack. The description should say which.

2. The settings page. #17061 names the page Compatibility Settings (grails-doc/src/en/guide/upgrading/compatibilitySettings.adoc). This page is resultSettings.adoc, titled "Settings for Fixes That Change a Result". This is the first setting, so this part creates the page, and the name should match.

The enum row doesn't fit the column "Default in Grails 7 and 8.0": 8.0 has no such setting, since 8.0.0 removed the legacy marshaller. A default for each release line, as #17061 lists it, would fit both rows.

3. The new links don't resolve. The guide has no guide/upgrading60x.html and no guide/resultSettings.html. A section is on the page of its chapter and on its own page under guide/pages/.

I built the guide at this head. Three links point to pages that don't exist, on upgrading.html, on single.html and on the section pages:

  • link:resultSettings.html in the note;
  • link:upgrading60x.html#grails-7-0-18 and #grails-7-0-2 on the page.

link:upgrading.html#grails-7-0-18[…] resolves on every page, as link:upgrading.html#_12_5_hibernate_ehcache already does in the same file.

4. The warning, the default in 8.1 and the removal in 9.0. The dev@ thread and #17061 agree on this: while an application keeps the old result through the setting, it logs a warning at startup that names the major release that removes the setting. This part has no such warning.

From 8.1.0, the warning concerns every application that sets the setting to false. Before that, it concerns one that sets it explicitly. The new default and the removal don't come with a merge-up either. Please say in the description which change does each:

  • the warning;
  • the default in 8.1.x, with its notes and the row of the page;
  • the removal in 9.0.x, for Hibernate 5 and for #16863.

An issue for each would keep them from being forgotten.

5. Add run ci, as on the other parts: the test jobs were skipped. Locally, the tests of grails-data-hibernate5-core pass at this head (907 tests, 57 skipped).

Notes

  1. On James's comment: with the setting on, the team is dropped here, but the sort isn't the cause.

    • Team.list(fetch: [club: 'join']) joins the club with an inner join with no sort at all. It returns 2 teams on 7.0.x and here, with the setting on or off.
    • The sort reuses that join, so it leaves out only what the fetch already leaves out.
    • #17071 fetches the club through a left join instead. At the stack top, the list keeps the team with or without the sort.
    • A mapping with club fetch: 'join' keeps the team here already.

    So there's nothing to change in this part for it. At the stack top, Team.list(sort: 'club.name', fetch: [club: 'join']) also returns 3 teams with the setting off, which I'll look at on #17071.

  2. As a fix that changes a result, this needs a second reviewer (#17061).

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested changes. A plain join the call asked for is reused by a sort, so a row with no association is left out.

for (Object existing : from.getJoins()) {
Join join = (Join) existing;
if (associationName.equals(join.getAttribute().getName())) {
return join;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setting that keeps a row with no association is on. Team.list(sort: 'club.name', fetch: [club: 'join']) should include Unattached First Team. It is left out, because the plain join is an inner join and the sort reuses it. Keep the row unless the call asked for an inner join.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants