Skip to content

Apply a requested join type with Hibernate 7 - #17079

Open
jdaugherty wants to merge 14 commits into
apache:8.0.xfrom
jdaugherty:fix/h7-requested-join-type
Open

jdaugherty wants to merge 14 commits into
apache:8.0.xfrom
jdaugherty:fix/h7-requested-join-type

Conversation

@jdaugherty

Copy link
Copy Markdown
Contributor

Description

The Hibernate 7 counterpart of the requested join type parts of the split of #16853: #17067, #17068, #17069 and #17071 on 7.0.x. As agreed on dev@, Hibernate 7 gets the same rule and spec on 8.0.x before those merge into 7.0.x, so the merge-up does not leave the two modules returning different rows for the same query. Relates to #15589.

Stacked on #16863. Until that merges, this pull request also shows its commits. The change to review is the last commit, e7fbf6e.

A where query, a detached criteria, a dynamic finder or a criteria query can request the type of the join of an association, with the fetch argument, with join(association, JoinType) or with @Join. Hibernate 7 applied it to the list of a query that requests it for an association of the queried class, and otherwise ignored it. A join type other than a left join is now applied whether or not anything else in the query uses the association.

AssociationJoinTypeSpec is the specification of the Hibernate 5 change, ported. On 8.0.x 79 of its 140 cases failed; all pass now. Three things differ from the Hibernate 5 spec on purpose:

  • A criteria query returns an instance once, where Hibernate 5 returns it once for each element of a collection that it fetches.
  • A projection of one property returns a value for each row, where Hibernate 5 makes it distinct.
  • The cases for a detached criteria whose criteria name the alias of the queried class (new DetachedCriteria(Team, 't').eq('t.name', ...)) are left out. Hibernate 7 does not resolve such a path at all, which is a separate fix.

Results that change

  • What changes:
    • count and the totalCount of a paged list count the instances that the list returns, also for a collection, a collection of a basic type and a right join.
    • A join type for a path through more than one association is applied; a path that ends in or passes through a collection restricts the query with a correlated subquery, so each instance is returned and counted once and a fetched collection keeps all of its elements.
    • A subquery and a property projection apply the join type.
    • updateAll and deleteAll apply it, so they change the instances that list returns. On MySQL and on MariaDB before 10.3 they reject a path that selects from the modified table.
    • list, findWhere, findAllWhere and the example queries apply a join type in their fetch argument.
    • The fetch argument of the list of a criteria query is applied; it was ignored, for a fetch mode as well.
    • In a criteria query, join('author') and select('author') are fetch modes, as in a where query. join applied an inner join, and select returned the authors instead of the instances.
    • A fetch mode requested after a join type replaces it; the join that criteria on the association make keeps the join type.
    • A join type requested for a property that is not an association is ignored; it failed with a SemanticException.
  • How to get the old result: remove the join type that the query requests, or request JoinType.LEFT. For a criteria query that relied on join('author') leaving out instances, request join('author', JoinType.INNER).
  • Setting: No setting. A requested join type that is ignored contradicts the call, which the rule from the dev@ thread sends to the oldest branch that has the bug. Two reviewers.

The change to AbstractDetachedCriteria and DynamicFinder in grails-datamapping-core is the same one that #17067 makes on 7.0.x.

The upgrade guide has a note under Grails 8.0.1, and the upgrade skill has the same entry.

Verification

grails-data-hibernate7-core (5074 tests) and grails-datamapping-core (1836 tests) pass, and codeStyle passes for both. Only H2 was used; the MySQL and MariaDB guard for updateAll and deleteAll is ported from the Hibernate 5 change and was not run against those databases. The full build was not run locally.

Contributor Checklist

Please review the following checklist before submitting your pull request. Pull requests that do not meet these requirements may be closed without review.

Issue and Scope

  • This PR is linked to an existing issue that has been acknowledged or approved by the project team. If no approved issue exists, please give background on why this change is necessary. Tickets are preferred for release change log history.
  • This PR addresses the complete scope of the linked issue. Partial implementations or unfinished work should not be submitted for review.
  • This PR contains a single, focused change. Unrelated changes should be submitted as separate pull requests.
  • This PR targets the correct branch for the type of change:
    • Patch release branches (e.g., 7.0.x): Bug fixes only. No new features or API changes.
    • Minor release branches (e.g., 7.1.x): New features are welcome, but breaking existing APIs must be avoided.
    • Major release branches (e.g., 8.0.x): Reserved for major changes. Breaking API changes are permitted.

Code Quality

  • I have added or updated tests that cover the changes introduced in this PR. All code contributions are expected to include appropriate test coverage.
  • I have verified that all existing tests pass by running ./gradlew build --rerun-tasks.
  • My code follows the project's code style guidelines. I have run ./gradlew codeStyle and resolved any violations. See Code Style for details.
  • This PR does not include mass reformatting, style-only changes, or large-scale refactoring unless it was explicitly approved in the linked issue. Unsolicited reformatting will not be accepted.
  • If generative AI tooling was used in preparing this contribution, a quality model was used to ensure contributions are consistent with the project's quality standards.

Licensing and Attribution

Documentation

  • If this PR introduces user-facing changes, I have included or updated the relevant documentation.
  • If this PR adds a new feature, I have updated the What's New section of the Grails Guide.
  • If this PR introduces breaking changes or changes that require user action during an upgrade, I have updated the Upgrade Notes for the corresponding version in the Grails Guide.
  • The PR description clearly explains what was changed and why.

First-time contributors: Please read our Contributing Guide before submitting.
Pull requests that appear to be auto-generated, incomplete, or unrelated to an approved issue may be
closed to help maintainers focus on reviewed and planned work. We appreciate your understanding.

With GORM for Hibernate 7, a sort by a property of an association read
the property through the association path, which Hibernate joins with
an implicit inner join. The sort left out the instances whose
association is null, while the total count of a paged list still counted
them. A sort must not change which rows a query returns, so each
association on a sort path is now joined explicitly, with a left join
unless the query already joins it or requests another join type for it.

Where queries, detached criteria, dynamic finders and criteria queries
resolve an order through ExpressionResolver.resolveSort, after the
criteria, so that an order reuses the join the criteria have for an
association. list builds its HQL with a left join for each association
it sorts through.

An order inside an association block of a criteria query was ignored,
because only the orders of the root criteria reach the query. It now
moves to the enclosing criteria, qualified with the association.

This is the Hibernate 7 counterpart of apache#16853 for Hibernate 5.
A query that selects a property of an association with distinct,
property or groupProperty and sorts by the same property was ordered
through a second join of the association. A database rejects that for
a distinct query, because the order is not in the select list. Such an
order now sorts by the expression that the projection selects.

An order in an association block by an alias that the block declares
was moved to the enclosing criteria as <association>.<alias>.<property>,
which does not resolve. It is now moved as an order by the path that
the alias stands for. An order that names neither a property of the
association nor an alias of the block stays in the block.

A sort through an embedded component, such as details.author.name,
still read the association inside the component through an implicit
inner join, and left out the rows where it is null. The association is
now joined with a left join, in criteria queries and in list.
An order in an association block by an alias that the block declares
was moved to the enclosing criteria as an order by the path that the
alias stands for. When the block declares two aliases of one
collection, each alias has a join of its own, and the sort could use
the join of the other alias. Such an order is now an
AssociationAliasOrder, which keeps its block and the property as the
block names it. PredicateGenerator records the context that the
criteria of a block are built against, and the order is resolved
against that context, so it sorts by the join of the alias it names.

An order in the block of an embedded component threw an
IllegalArgumentException: the property was looked up on the entity of
the criteria of the block, and the class of a component is not a
domain class. It is now looked up on the entity of the association,
and the order sorts by the property of the component.

A sort by a path reused the join that From.getJoins() returned first
for an association, and that set has no order. With two aliases of a
collection, an order by the path of the collection sorted by one alias
at the root and by the other in an association block. A sort now
reuses the first join that the query made for the association, which
is the join that the path shares with its first alias.
The closure of and, or and not in a criteria query is evaluated by the
detached criteria and not by the builder, so an order in an association
block inside a junction stayed in its block and was ignored. The
builder now moves the orders of the blocks inside a junction, at any
depth, to the enclosing criteria once the junction is added. In such a
block createAlias declares an alias with association criteria that
have the alias, and not with a HibernateAlias, and an order by it is
resolved against its block as well.

The upgrade notes now describe a sort by a property of a collection,
and say that the instances without the association can be on the first
page of a paged list and be what first or a findBy finder returns.
An order of an association block had no effect while it stayed in its
block. Moved to the root criteria, it reached queries that can only be
ordered by what they select: listDistinct and queries with a distinct
or aggregate projection ran before and were now rejected by the
database. An order moved out of a block is an AssociationOrder, and in
a query that selects distinct or grouped values or aggregates it is
left out as before, unless the query selects its property.

An order by an alias of a block was sorted by the expression of a
projection that selects the path of the alias, which is the join of
the path and not the join of the alias. Only a distinct or grouped
query has to be ordered by what it selects, so in any other query the
order sorts by the alias that it names.

A sort by the identifier of an association, such as club.id, read the
foreign key column without a join, and now joined the association.
When the association is mapped as a many-to-one, which has its key in
the table of the entity, the path is read as it is again, in criteria
queries and in list. The JPA metamodel reports a hasOne association,
whose key is in the other table, as a many-to-one too, so the mapping
of the property decides.

The join that a sort reuses is read from the joins that Hibernate
keeps in order. The fallback to From.getJoins() was only reachable
from unit specs that mocked From, which now mock Hibernate's types.
The contexts of association blocks are recorded in a map that only the
context of the query creates.
The rules that turn an order of an association block into an order of
the enclosing criteria move to AssociationOrders. The criteria builder
still applies them when a block ends. Criteria that are built in any
other way, such as a detached criteria, keep the orders in their
blocks, and the query now reads them from there, after the orders of
the criteria themselves. An order in a block by an alias of the
criteria around the block is resolved against the block, which sees
that alias, and no longer ignored.

A distinct list ordered by a property of an association was rejected
by the database, since SQL orders a distinct query only by what it
selects. Such a query is built without the keyword and its entities
are made distinct once they are read, as with Hibernate 5. An order of
an association block therefore sorts a distinct list too. An order by
a property that does not resolve is rejected as an invalid sort
property and no longer fails an assertion in Hibernate.

Inside the block of an embedded component, an association block threw
"Not an entity" and createAlias threw "is not a domain class". The
builder takes the managed type and the entity of the component there.

An inner join requested for a to-one association was only applied by
the fetch join of a full-entity select. A count, the total count of a
paged list and a projection query now join the association as well, so
a paged list counts the rows that it returns.

A sort reuses the fetch join of an association when the query has no
other join of it, in criteria queries and in list. HqlListQueryBuilder
validates a sort path and builds its joins in one walk. The methods
that the unit specs call are public, or protected on the method
invoker.
An order of an association block was resolved by its path from the
root of the query, which does not tell the joins of one block from
those of another. Two blocks of the same association share the join of
that association and each join what is inside it, so an order in the
second block sorted by the join of the first. Every order that comes
out of a block now keeps its block and the property as the block names
it, and is resolved against the context that the criteria of the block
were built against. A sort takes the join that this context tracks for
a path before any other join of the association. AssociationAliasOrder
is removed, since an order by an alias is no longer the only order
that needs its block.

The closure of and, or and not was evaluated by the detached criteria,
so an association block inside a junction was not a block of the
builder, and its orders were added after the orders that were declared
after it. The builder now evaluates the closure of a junction itself,
as it evaluates every other closure of a criteria query. A block
inside a junction is therefore the same as a block outside one: its
orders keep the place they were declared in, a join type given to it
is applied, and its association is fetched with the query.

A distinct projection of an association that is ordered by a property
of the association ran on 8.0.x and was rejected by the database,
because the order had a left join of its own next to the join of the
projection. An order now sorts by the expression of the projection
when the query selects the property or the association that the
property belongs to.
8.0.x now keeps the joins of a query and of its association blocks in
one map by their path, so two blocks of one association share a join.
The four features that expected a join for each block are removed: such
a query now applies the criteria of both blocks to the same join.

The fetch mode join of list is a left join on 8.0.x, so the fetch joins
that a sort reuses are written as left joins.
…nSort setting

A sort by a property of an association changes which instances a query
returns when it joins the association with a left join instead of an
inner join. Following the dev@ discussion of fixes that change results,
that is now behind grails.gorm.leftJoinAssociationSort, the setting that
GORM for Hibernate 5 reads for the same change on 7.0.x. It is disabled
by default, and 8.1 enables it by default.

Without the setting, list, dynamic finders, criteria queries, where
queries and detached criteria join an association that only a sort uses
with an inner join. With it, they join it with a left join.

The specifications of the sorts run with the setting off and on.
A where query, a detached criteria, a dynamic finder or a criteria query
can request the type of the join of an association, with the fetch
argument, with join(association, JoinType) or with @Join. Hibernate 7
applied it to the list of a query that requests it for an association of
the queried class and otherwise ignored it. A join type other than a
left join is now applied whether or not anything else in the query uses
the association, as GORM for Hibernate 5 applies it on 7.0.x:

- count and the totalCount of a paged list count the instances that the
  list returns, also for a collection, a collection of a basic type and
  a right join;
- a path through more than one association joins and fetches every
  association on it, and a path that ends in or passes through a
  collection restricts the query with a correlated subquery, so each
  instance is returned and counted once and a fetched collection keeps
  all of its elements;
- a subquery and a property projection apply it;
- updateAll and deleteAll apply it as a restriction, and reject a path
  that selects from the modified table on MySQL and on MariaDB before
  10.3;
- list, findWhere, findAllWhere and the example queries apply a join
  type in their fetch argument, by running as a criteria query;
- the fetch argument of the list of a criteria query is applied; it was
  ignored;
- in a criteria query, join(association) and select(association) are
  fetch modes, as they are in a where query: join applied an inner join
  and select returned the association instead of the instances;
- a fetch mode requested after a join type replaces it, while the join
  that criteria on the association make keeps the join type. The change
  to AbstractDetachedCriteria and DynamicFinder for this is the one that
  the Hibernate 5 pull request makes on 7.0.x;
- a join type requested for a property that is not an association is
  ignored instead of failing.

AssociationJoinTypeSpec is the specification of the Hibernate 5 change,
with the expectations that describe the Hibernate 5 engine adjusted: a
criteria query returns an instance once, and a projection of one
property is not made distinct.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested changes. A join of a collection of basic values deletes the collection rows and leaves the owners.

if (!(criteria.getPersistentEntity().getPropertyByName(association) instanceof ToMany)) {
continue;
}
if (persister.getPropertyType(association) instanceof CollectionType collectionType &&

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An entity has a collection of strings. join('tags').deleteAll() should delete the owners. It deletes the collection rows, then matches nothing, and the owners stay. Delete the owners by their identifiers when the join is a collection of basic values.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b5fecc8. A requested non-LEFT join of a basic collection now takes the existing identifier-selection path before Hibernate removes the collection rows.

The new regression checks both INNER and RIGHT, verifies the deleted owner count, and confirms that an unmatched owner and its collection stay intact. Both cases failed before the fix. All 139 cases of AssociationJoinTypeSpec pass; the full Hibernate 7 core suite and required clean aggregate checks also pass, with no failures or errors in the 26,618-test combined report.

@testlens-app

testlens-app Bot commented Nov 19, 2026

Copy link
Copy Markdown

✅ All tests passed ✅

🏷️ Commit: e7fbf6e
▶️ Tests: 106386 executed
⚪️ Checks: 94/94 completed


Learn more about TestLens at testlens.app/docs.

@jdaugherty

Copy link
Copy Markdown
Contributor Author

6de9334 brings in the reviewed fixes from the parent, #16863. The combined branch now passes the complete Hibernate 7 core suite and the required aggregate checks: the final combined report contains 26,624 tests, zero failures/errors and 472 skipped. Checkstyle, CodeNarc, repository conventions, enabled PMD checks and RAT pass; SpotBugs is disabled.

The Hibernate 5 review fixes have also been carried through the dependent split branches, with the combined Hibernate 5 core suite passing. The agreed merge order remains: this Hibernate 7 rule and spec must be on 8.0.x before the requested-join parts merge into 7.0.x.

@jamesfredley jamesfredley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested changes. A block overwrites a left join the call asked for.

@@ -221,19 +251,25 @@ protected Object tryAssociationOrJunction(String name, CriteriaMethods method, O
hibernateQuery.join(name, joinType);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A criteria query joins club as a left join, then orders by club.name inside a club block. Unattached First Team is left out. The block replaces the left join with an inner join. Keep the left join the call asked for.

@matrei matrei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 6de9334 (base 8.0.x): e7fbf6e, b5fecc8 and the merge of #16863's head (87594ea), which merges without a conflict. I ran with Hibernate 7 the probes of my reviews of #17067 to #17074, about 300 queries, with grails.gorm.leftJoinAssociationSort off and on:

Against #16863, every result that changes moves to the result of the Hibernate 5 stack, except in points 1, 2 and 4. For example:

8.0.x #16863 This head Hibernate 5 stack
Player.where { … }.join('team.club', JoinType.INNER).list(), also a criteria query 7 players 7 6 6
Team.list(fetch: [club: JoinType.INNER]) 7 teams 7 5 5
Team.createCriteria().list(fetch: [club: JoinType.INNER]) { … } 5 5 4 4
Team.where { … }.join('club', JoinType.INNER).updateAll(…) 5 updated 5 4 4
Team.where { … }.join('players.contract', JoinType.INNER).updateAll(…) 7 7 4 4
deleteAll with an inner join of a unidirectional collection 2 deleted 2 1 1
Team.where { … }.join('club', JoinType.INNER).select('club').list() 3 clubs instead of teams 3 clubs 5 teams 5
Team.createCriteria().list { join('club', JoinType.INNER); join('club') } 4 4 5 5

A paged list that fetches a collection through a path, as in fetch: ['players.contract': JoinType.INNER] with max: 2, fetches the team with all three of its players here. That's the result that I asked Hibernate 5 to have, in point 1 on #17067. I agree that this needs no setting, by #17061, and that the Hibernate 5 parts wait for it.

The tests pass at this head: grails-data-hibernate7-core has 5076 tests, 44 skipped, and grails-datamapping-core 1836 tests, 2 skipped.

Needs a fix

1. With discriminator multi-tenancy, deleteAll through a join table deletes an instance that list leaves out. Cases that aren't multi-tenant, with items that are, in a unidirectional collection. Under one tenant:

Case.where { name ==~ '%case' }.join('items', JoinType.INNER) 8.0.x and #16863 This head Hibernate 5 stack
list() 1 case 1 1
updateAll(name: 'X') 3 1 1
deleteAll() 3 2 1

The second case that deleteAll deletes has only an item of the other tenant. updateAll matches the one case, so the restriction with the tenant filter is right. The identifiers that deleteByIdentifiers selects first aren't, so the select seems to run without the filter of the subquery. Hibernate 5 deletes the one case.

2. A subquery's join type isn't applied to updateAll and deleteAll, as in point 3 on #17068. Since this head, list applies the join type of a subquery, and the bulk operations still don't:

#16863 This head
new DetachedCriteria(Team).inList('id', Team.where { name ==~ '%First%' }.join('club', JoinType.INNER).id()): list / updateAll 5 / 5 4 / 5
the same for deleteAll with two teams, one without a club: list / deleteAll 2 / 2 1 / 2
Player.where { team in Team.where { … }.join('club', JoinType.INNER) }: list / updateAll 4 / 4 3 / 4

At #16863 these agreed. Now updateAll changes rows that list leaves out.

3. A join type requested inside the block of an association is ignored. Player.createCriteria().list { team { join('club', JoinType.INNER) } } returns all 7 players here, while join('team.club', JoinType.INNER) returns 6. The Hibernate 5 stack returns 6 for both.

4. James's latest comment holds: in a criteria query, the order of join and the block of the association decides the join type.

Team.createCriteria().list { like('name', '%First%'); … } This head Hibernate 5 stack
join('club', JoinType.LEFT); club { order('name') } 4 teams 4
club { order('name') }; join('club', JoinType.LEFT) 5 4
club { order('name') } with list(fetch: [club: JoinType.LEFT]) 5 4
or { club { eq('name', 'Chelsea') }; eq('name', 'Unattached First Team') }; join('club', JoinType.LEFT) 3 2

Hibernate 5 keeps the inner join of the block in every order, and I asked it to use the requested type instead, in point 1 on #17069. Here the requested type wins when it comes after the block, and loses when it comes before it. Whichever of the two is kept, both modules need it in any order.

5. A bag holds repeated elements. Members are a bag, trophies a set, and the team has two members:

8.0.x This head Hibernate 5 stack
Team.where { members { name == 'a' } }.join('members', JoinType.LEFT).join('trophies').list() the bag holds 4 members 4 2, loaded when read
Team.list(sort: 'trophies.title', fetch: [members: 'join']), with the setting off and on fails 4 4 with the setting off; throws with it on

The first comes from 8.0.x, and the Hibernate 5 stack fixes it in #17072. The second comes from #16863, where a sort by a collection starts to work. In both, the session holds a collection with elements that aren't there.

6. Player.list(fetch: ['team.players': 'join'], max: 2) fails with Query specified join fetching, but the owner of the fetched association was not present, on 8.0.x and here. The Hibernate 5 stack returns the 2 players, with the team's players fetched. This is the fetch argument of list, which this pull request ports from #17071.

7. Resolve the conflicts with 8.0.x. upgrading80x.adoc conflicts, as #16863 does. No check has run for b5fecc8 or for this head, so the green run is the one of e7fbf6e.

Notes

  1. The description lists three results that differ from Hibernate 5 on purpose. The first two are a criteria query that returns an instance once, and a property projection that isn't distinct. #17061 counts a Hibernate 7 result that differs from Hibernate 5 as a regression, so the migration from Hibernate 5 to Hibernate 7 in the guide should name them. The same holds for these, which my probes found:

    • order('players.name') and Team.list(sort: 'players.name') return the team once here, and once for each player with Hibernate 5;
    • a right join: here list has a null for each club that no team refers to, and Team.list(fetch: [club: JoinType.RIGHT]) has the null as well, where Hibernate 5 has one null (point 2 on #17067) and applies an inner join (point 2 on #17071);
    • a projection of a property of an association leaves out the rows without the association (note 8 on #17066).
  2. The count and the totalCount of a sort through an association leave out the join of the sort, so they count the teams without a club that the list leaves out. For example, Team.createCriteria().list(sort: 'club.name') { like('name', '%First%') } returns 4 teams with a totalCount of 5. That's #16863's, which I noted on #17064.

  3. This pull request ports #17067, #17068, #17069 and #17071. The other parts of the stack change Hibernate 5 only, and Hibernate 7 returns other results for them:

    • #17070: a map of sort properties in a criteria query fails with a ClassCastException, here and on 8.0.x;
    • #17072: no exception for a sort by a bag next to the fetch of another collection;
    • #17073: a sort by the alias of the queried class fails with Invalid sort property;
    • #17074 on 9.0.x: other exceptions for an undefined alias.

    The merge-up then leaves the two modules apart there. They need a counterpart here or in a pull request of its own, before those parts merge.

  4. Team.findAll(new Team(), [sort: 'x.name']) and [sort: 'this.name'] return null with Hibernate 7, here and on 8.0.x, where list fails with Invalid sort property for the same sorts. That's a bug of its own.

  5. AbstractDetachedCriteria.getSupersededJoinTypes() is the same new public API on 8.0.x as on 7.0.x (point 5 on #17067).

  6. The description names e7fbf6e as the change to review. b5fecc8 is on top of it, for James's first comment.

  7. As a fix that changes a result, this needs a second reviewer (#17061).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants