Skip to content

Count a where query that has a sort - #17064

Open
jdaugherty wants to merge 9 commits into
apache:7.0.xfrom
jdaugherty:fix/16853-04-count-with-sort
Open

jdaugherty wants to merge 9 commits into
apache:7.0.xfrom
jdaugherty:fix/16853-04-count-with-sort

Conversation

@jdaugherty

Copy link
Copy Markdown
Contributor

Description

Part 4 of the split of #16853, as agreed there and in the dev@ thread Fixes that change results on patch and minor branches. Relates to #15589.

Stacked on #17063. Until that merges, this pull request also shows the commits below it. The change to review is the last commit, eb84a32.

count on a where query or a detached criteria that has a sort, such as
Team.where { ... }.sort('name').count(), kept the order by in the query
that counts the rows, which H2 rejects with "must be in the GROUP BY
list". A query that only counts its rows now has no orders.

The join that a sort by a property of an association makes is kept, so
the count is that of the rows which list returns for the same query.

Results that change

  • What changes: count() on a where query or a detached criteria that has a sort returns the count instead of throwing. It counts the rows that list returns for the same query.
  • How to get the old result: Not applicable: the call failed.
  • Setting: No setting. A failing call now works.

The upgrade guide has a note under Grails 7.0.18, and the upgrade skill has the same entry.

Verification

The association sort and join specs of grails-data-hibernate5-core pass on this branch, with grails.gorm.leftJoinAssociationSort off and on, and codeStyle passes for the touched modules. The full build was not run locally.

Contributor Checklist

Please review the following checklist before submitting your pull request. Pull requests that do not meet these requirements may be closed without review.

Issue and Scope

  • This PR is linked to an existing issue that has been acknowledged or approved by the project team. If no approved issue exists, please give background on why this change is necessary. Tickets are preferred for release change log history.
  • This PR addresses the complete scope of the linked issue. Partial implementations or unfinished work should not be submitted for review.
  • This PR contains a single, focused change. Unrelated changes should be submitted as separate pull requests.
  • This PR targets the correct branch for the type of change:
    • Patch release branches (e.g., 7.0.x): Bug fixes only. No new features or API changes.
    • Minor release branches (e.g., 7.1.x): New features are welcome, but breaking existing APIs must be avoided.
    • Major release branches (e.g., 8.0.x): Reserved for major changes. Breaking API changes are permitted.

Code Quality

  • I have added or updated tests that cover the changes introduced in this PR. All code contributions are expected to include appropriate test coverage.
  • I have verified that all existing tests pass by running ./gradlew build --rerun-tasks.
  • My code follows the project's code style guidelines. I have run ./gradlew codeStyle and resolved any violations. See Code Style for details.
  • This PR does not include mass reformatting, style-only changes, or large-scale refactoring unless it was explicitly approved in the linked issue. Unsolicited reformatting will not be accepted.
  • If generative AI tooling was used in preparing this contribution, a quality model was used to ensure contributions are consistent with the project's quality standards.

Licensing and Attribution

Documentation

  • If this PR introduces user-facing changes, I have included or updated the relevant documentation.
  • If this PR adds a new feature, I have updated the What's New section of the Grails Guide.
  • If this PR introduces breaking changes or changes that require user action during an upgrade, I have updated the Upgrade Notes for the corresponding version in the Grails Guide.
  • The PR description clearly explains what was changed and why.

First-time contributors: Please read our Contributing Guide before submitting.
Pull requests that appear to be auto-generated, incomplete, or unrelated to an approved issue may be
closed to help maintainers focus on reviewed and planned work. We appreciate your understanding.

matrei and others added 8 commits November 19, 2026 09:30
Sorting a where query by a property of an association, such as
sort: 'club.name', failed with a QueryException when the query also
had criteria on that association. The criteria join the association
under a generated alias, and the order was added with the association
path, which Hibernate cannot resolve. The order now uses the alias of
the join.
A sort by a path through more than one association, such as
owner.employer.name, is still not supported.
A sort by a path that crosses more than one association, such as
team.club.name, failed with a QueryException whether or not the where
query joined the nested association, because order() split the path at
its first dot only. The sort property is now qualified with the alias of
the last association on its path: an association the query has already
joined is reused, and one it has not joined yet is joined for the sort
and registered like any other join. A sort by two properties of the same
association no longer fails with "duplicate association path" either,
since the sort join is registered instead of created anew for every
order.
Since order() walks the whole sort path, a sort by the identifier of an
association that follows another one on the path, such as team.club.id,
joined that last association too. The join is an inner join unless fetch
requests another type, so the sort left out the rows where the
association is null. Before, only the first association was joined and
Hibernate read club.id from the foreign key of team, which kept them.

The identifier is sorted by the foreign key again when the entity before
the association holds it. An association without such a foreign key, a
hasOne or the inverse side of a one-to-one, is still joined, and so is
the first association of a path, as it always has been.

The spec also covers the join type of a sort join: an inner join by
default, and a left join when fetch requests one. Team.club is nullable
in the fixture for these cases. The docs describe the inner join, how to
request a left join, and the identifier of a nested association.
list with a map of sort properties, such as
Team.list(sort: ['club.name': 'asc', name: 'desc']), sorted by the last
entry only, because each entry replaced the orders of the query. The
same held for a default sort that the mapping declares for more than
one property. Every entry is now applied, in the order of the map.

An association is joined once for all the sort properties that use it.
list joined it once for each, and find and findAll with an example,
findWhere and findAllWhere failed with "duplicate association path".

Part 2 of the split of apache#16853.
A paged query of a detached criteria that has an alias for the queried
class, and criteria that use it, failed when totalCount was read:

    new DetachedCriteria(Team, 't').eq('t.name', 'Arsenal').list(max: 2).totalCount

The count runs on a copy of the criteria, and the copy was created
without the alias, so Hibernate could not resolve t.name. The copy now
has the alias of the original.

Part 3 of the split of apache#16853.
count on a where query or a detached criteria that has a sort, such as
Team.where { ... }.sort('name').count(), kept the order by in the query
that counts the rows, which H2 rejects with "must be in the GROUP BY
list". A query that only counts its rows now has no orders.

The join that a sort by a property of an association makes is kept, so
the count is that of the rows which list returns for the same query.

Part 4 of the split of apache#16853.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested changes. A count of a sorted query throws when a listener adds an order.

criteria.setProjection(hibernateProjectionList.getHibernateProjectionList());
}
criteria.setResultTransformer(CriteriaSpecification.DISTINCT_ROOT_ENTITY);
removeOrdersOfRowCount();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A listener adds an order by name before a count of a sorted query. The count should return the number of matching rows. It throws, because the original sort is cleared and the listener's order is left on the count. Do not leave an order on the count.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5dc4d34. Orders are removed after PreQueryEvent listeners have run, immediately before the count executes. Their joins remain, so a listener that orders through an association still restricts the count in the same way as the list. A query selecting only a count() projection is covered too.

The four new public-API cases failed before the fix. All 34 cases of the listener and association-sort specs pass with the fix, as does the module codeStyle check. The upgrade note and upgrade skill have the same clarification.

@matrei matrei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 5dc4d34 (base 7.0.x), the last two commits. I probed counts with a sort on 7.0.x (aee4c04), at part 3 (62bf8ed), at eb84a32 and here. The data is three matching teams: two with a club and players, one with neither.

The fix is right. Every count with a sort throws on 7.0.x and at part 3 with Column "…" must be in the GROUP BY list. Here it returns what the list of the same query returns:

Team.where { name ==~ '%First%' } with 7.0.x and part 3 This head list()
.sort('name').count(), countByNameLike('%First%', [sort: 'name']), and if (query.sort('name')) throws 3 3
.sort('club.name').count(), .count(sort: 'club.name'), a detached criteria, and countByNameLike('%First%', [sort: 'club.name']) throws 2 2
.join('club', JoinType.LEFT).sort('club.name').count() throws 3 3
.projections { count() }.sort('club.name').list() throws, also at eb84a32 [2] 2

The count runs the 7.0.x statement without its order by, with the same joins, so a database that accepted the order counts as before. By #17061 that makes this a fix like any other. A grouped projection keeps its orders: projections { groupProperty('name'); count() }.sort('name') returns the same three rows as on 7.0.x.

Needs a fix

1. Parts 5 to 13 are stacked on eb84a32.

  • A merge of their head with 5dc4d34 conflicts in AbstractHibernateQuery.java and upgrading60x.adoc.
  • Part 6 (0309713) moves the removal into applyDeferredJoins, after the listeners, but keeps the isRowCount() condition.
  • So at the stack top (e359719), the two count projection features of 5dc4d34 fail, and projections { count() }.sort('name').list() still throws.

The rebase has to keep the RowCountProjection check. Squashing 5dc4d34 into eb84a32 would also keep one commit per part, and its message has no body yet.

2. The description and the note. The description names eb84a32 as the change to review.

By #17061, a fix that makes a failing call work needs no "Results that change" section, no note and no skill entry. I'd drop all three, as on #17063. The skill entry says that nothing needs to change, so it gives an upgrade nothing to do.

If you keep the note, its last sentence needs a change, and so does the commit message. With a sort by a property of a collection, count counts the rows of the join, not the instances that list returns:

Team.where { name ==~ '%First%' }.sort('players.name') 7.0.x This head
list() 2 teams 2 teams
count() throws 5
list(max: 10).totalCount 5 5

The number itself is fine:

  • The count runs the same statement as that totalCount.
  • A criterion on the collection, where { players.name ==~ '%' }.count(), gives 5 on 7.0.x too.

3. Add run ci, as on #17062 and #17063: the test jobs were skipped. Locally, the tests of grails-data-hibernate5-core pass at the head (778 tests, 23 skipped).

Notes

  1. With Hibernate 7 on 8.0.x (0518f48), every count above works, but a count with a sort by an association leaves out the sort's join. Team.where { … }.sort('club.name').count() is 3 for a list of 2, as is the totalCount of list(max: 2, sort: 'club.name'). After the merge-up, Hibernate 5 gives 2 and Hibernate 7 gives 3 for the same call. That's for #16863, which keeps those rows in the list.

  2. Two counts with a sort still throw with Hibernate 5 and work with Hibernate 7:

    • the criteria builder's Team.createCriteria().count { like('name', '%First%'); order('name') }, which can share its closure with list;
    • projections { countDistinct('name') } with a sort.

    That's not for this pull request. A later fix on 7.0.x would make a failing call work there too.

  3. Two style points:

    • import RowCountProjection instead of naming it in full;
    • def criteria in the new listener feature can be final.

Verified as correct

  • The new feature of WhereQueryWithAssociationSortSpec fails at part 3 with a SQLGrammarException and passes here.
  • James's listener case is fixed by 5dc4d34: the four new features of PreQueryListenerAssociationCriteriaSpec fail at eb84a32 and pass here.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested changes. A sort by a nested identifier drops rows whose association is empty.

*/
private boolean isForeignKeyIdentifier(PersistentEntity owner, Association association, String property) {
PersistentEntity associatedEntity = association.getAssociatedEntity();
PersistentProperty identity = associatedEntity != null ? associatedEntity.getIdentity() : null;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A player whose team has no club should still be listed when the sort is team.club.id. That player is left out, because this joins club instead of sorting by the foreign key. Keep the player.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants