Repository navigation
Apply a requested join type with Hibernate 7 - #17079
jdaugherty wants to merge 14 commits into
Conversation
With GORM for Hibernate 7, a sort by a property of an association read the property through the association path, which Hibernate joins with an implicit inner join. The sort left out the instances whose association is null, while the total count of a paged list still counted them. A sort must not change which rows a query returns, so each association on a sort path is now joined explicitly, with a left join unless the query already joins it or requests another join type for it. Where queries, detached criteria, dynamic finders and criteria queries resolve an order through ExpressionResolver.resolveSort, after the criteria, so that an order reuses the join the criteria have for an association. list builds its HQL with a left join for each association it sorts through. An order inside an association block of a criteria query was ignored, because only the orders of the root criteria reach the query. It now moves to the enclosing criteria, qualified with the association. This is the Hibernate 7 counterpart of apache#16853 for Hibernate 5.
A query that selects a property of an association with distinct, property or groupProperty and sorts by the same property was ordered through a second join of the association. A database rejects that for a distinct query, because the order is not in the select list. Such an order now sorts by the expression that the projection selects. An order in an association block by an alias that the block declares was moved to the enclosing criteria as <association>.<alias>.<property>, which does not resolve. It is now moved as an order by the path that the alias stands for. An order that names neither a property of the association nor an alias of the block stays in the block. A sort through an embedded component, such as details.author.name, still read the association inside the component through an implicit inner join, and left out the rows where it is null. The association is now joined with a left join, in criteria queries and in list.
An order in an association block by an alias that the block declares was moved to the enclosing criteria as an order by the path that the alias stands for. When the block declares two aliases of one collection, each alias has a join of its own, and the sort could use the join of the other alias. Such an order is now an AssociationAliasOrder, which keeps its block and the property as the block names it. PredicateGenerator records the context that the criteria of a block are built against, and the order is resolved against that context, so it sorts by the join of the alias it names. An order in the block of an embedded component threw an IllegalArgumentException: the property was looked up on the entity of the criteria of the block, and the class of a component is not a domain class. It is now looked up on the entity of the association, and the order sorts by the property of the component. A sort by a path reused the join that From.getJoins() returned first for an association, and that set has no order. With two aliases of a collection, an order by the path of the collection sorted by one alias at the root and by the other in an association block. A sort now reuses the first join that the query made for the association, which is the join that the path shares with its first alias.
The closure of and, or and not in a criteria query is evaluated by the detached criteria and not by the builder, so an order in an association block inside a junction stayed in its block and was ignored. The builder now moves the orders of the blocks inside a junction, at any depth, to the enclosing criteria once the junction is added. In such a block createAlias declares an alias with association criteria that have the alias, and not with a HibernateAlias, and an order by it is resolved against its block as well. The upgrade notes now describe a sort by a property of a collection, and say that the instances without the association can be on the first page of a paged list and be what first or a findBy finder returns.
An order of an association block had no effect while it stayed in its block. Moved to the root criteria, it reached queries that can only be ordered by what they select: listDistinct and queries with a distinct or aggregate projection ran before and were now rejected by the database. An order moved out of a block is an AssociationOrder, and in a query that selects distinct or grouped values or aggregates it is left out as before, unless the query selects its property. An order by an alias of a block was sorted by the expression of a projection that selects the path of the alias, which is the join of the path and not the join of the alias. Only a distinct or grouped query has to be ordered by what it selects, so in any other query the order sorts by the alias that it names. A sort by the identifier of an association, such as club.id, read the foreign key column without a join, and now joined the association. When the association is mapped as a many-to-one, which has its key in the table of the entity, the path is read as it is again, in criteria queries and in list. The JPA metamodel reports a hasOne association, whose key is in the other table, as a many-to-one too, so the mapping of the property decides. The join that a sort reuses is read from the joins that Hibernate keeps in order. The fallback to From.getJoins() was only reachable from unit specs that mocked From, which now mock Hibernate's types. The contexts of association blocks are recorded in a map that only the context of the query creates.
The rules that turn an order of an association block into an order of the enclosing criteria move to AssociationOrders. The criteria builder still applies them when a block ends. Criteria that are built in any other way, such as a detached criteria, keep the orders in their blocks, and the query now reads them from there, after the orders of the criteria themselves. An order in a block by an alias of the criteria around the block is resolved against the block, which sees that alias, and no longer ignored. A distinct list ordered by a property of an association was rejected by the database, since SQL orders a distinct query only by what it selects. Such a query is built without the keyword and its entities are made distinct once they are read, as with Hibernate 5. An order of an association block therefore sorts a distinct list too. An order by a property that does not resolve is rejected as an invalid sort property and no longer fails an assertion in Hibernate. Inside the block of an embedded component, an association block threw "Not an entity" and createAlias threw "is not a domain class". The builder takes the managed type and the entity of the component there. An inner join requested for a to-one association was only applied by the fetch join of a full-entity select. A count, the total count of a paged list and a projection query now join the association as well, so a paged list counts the rows that it returns. A sort reuses the fetch join of an association when the query has no other join of it, in criteria queries and in list. HqlListQueryBuilder validates a sort path and builds its joins in one walk. The methods that the unit specs call are public, or protected on the method invoker.
An order of an association block was resolved by its path from the root of the query, which does not tell the joins of one block from those of another. Two blocks of the same association share the join of that association and each join what is inside it, so an order in the second block sorted by the join of the first. Every order that comes out of a block now keeps its block and the property as the block names it, and is resolved against the context that the criteria of the block were built against. A sort takes the join that this context tracks for a path before any other join of the association. AssociationAliasOrder is removed, since an order by an alias is no longer the only order that needs its block. The closure of and, or and not was evaluated by the detached criteria, so an association block inside a junction was not a block of the builder, and its orders were added after the orders that were declared after it. The builder now evaluates the closure of a junction itself, as it evaluates every other closure of a criteria query. A block inside a junction is therefore the same as a block outside one: its orders keep the place they were declared in, a join type given to it is applied, and its association is fetched with the query. A distinct projection of an association that is ordered by a property of the association ran on 8.0.x and was rejected by the database, because the order had a left join of its own next to the join of the projection. An order now sorts by the expression of the projection when the query selects the property or the association that the property belongs to.
8.0.x now keeps the joins of a query and of its association blocks in one map by their path, so two blocks of one association share a join. The four features that expected a join for each block are removed: such a query now applies the criteria of both blocks to the same join. The fetch mode join of list is a left join on 8.0.x, so the fetch joins that a sort reuses are written as left joins.
…nSort setting A sort by a property of an association changes which instances a query returns when it joins the association with a left join instead of an inner join. Following the dev@ discussion of fixes that change results, that is now behind grails.gorm.leftJoinAssociationSort, the setting that GORM for Hibernate 5 reads for the same change on 7.0.x. It is disabled by default, and 8.1 enables it by default. Without the setting, list, dynamic finders, criteria queries, where queries and detached criteria join an association that only a sort uses with an inner join. With it, they join it with a left join. The specifications of the sorts run with the setting off and on.
A where query, a detached criteria, a dynamic finder or a criteria query can request the type of the join of an association, with the fetch argument, with join(association, JoinType) or with @Join. Hibernate 7 applied it to the list of a query that requests it for an association of the queried class and otherwise ignored it. A join type other than a left join is now applied whether or not anything else in the query uses the association, as GORM for Hibernate 5 applies it on 7.0.x: - count and the totalCount of a paged list count the instances that the list returns, also for a collection, a collection of a basic type and a right join; - a path through more than one association joins and fetches every association on it, and a path that ends in or passes through a collection restricts the query with a correlated subquery, so each instance is returned and counted once and a fetched collection keeps all of its elements; - a subquery and a property projection apply it; - updateAll and deleteAll apply it as a restriction, and reject a path that selects from the modified table on MySQL and on MariaDB before 10.3; - list, findWhere, findAllWhere and the example queries apply a join type in their fetch argument, by running as a criteria query; - the fetch argument of the list of a criteria query is applied; it was ignored; - in a criteria query, join(association) and select(association) are fetch modes, as they are in a where query: join applied an inner join and select returned the association instead of the instances; - a fetch mode requested after a join type replaces it, while the join that criteria on the association make keeps the join type. The change to AbstractDetachedCriteria and DynamicFinder for this is the one that the Hibernate 5 pull request makes on 7.0.x; - a join type requested for a property that is not an association is ignored instead of failing. AssociationJoinTypeSpec is the specification of the Hibernate 5 change, with the expectations that describe the Hibernate 5 engine adjusted: a criteria query returns an instance once, and a projection of one property is not made distinct.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A join of a collection of basic values deletes the collection rows and leaves the owners.
| if (!(criteria.getPersistentEntity().getPropertyByName(association) instanceof ToMany)) { | ||
| continue; | ||
| } | ||
| if (persister.getPropertyType(association) instanceof CollectionType collectionType && |
There was a problem hiding this comment.
An entity has a collection of strings. join('tags').deleteAll() should delete the owners. It deletes the collection rows, then matches nothing, and the owners stay. Delete the owners by their identifiers when the join is a collection of basic values.
There was a problem hiding this comment.
Fixed in b5fecc8. A requested non-LEFT join of a basic collection now takes the existing identifier-selection path before Hibernate removes the collection rows.
The new regression checks both INNER and RIGHT, verifies the deleted owner count, and confirms that an unmatched owner and its collection stay intact. Both cases failed before the fix. All 139 cases of AssociationJoinTypeSpec pass; the full Hibernate 7 core suite and required clean aggregate checks also pass, with no failures or errors in the 26,618-test combined report.
✅ All tests passed ✅🏷️ Commit: e7fbf6e Learn more about TestLens at testlens.app/docs. |
|
6de9334 brings in the reviewed fixes from the parent, #16863. The combined branch now passes the complete Hibernate 7 core suite and the required aggregate checks: the final combined report contains 26,624 tests, zero failures/errors and 472 skipped. Checkstyle, CodeNarc, repository conventions, enabled PMD checks and RAT pass; SpotBugs is disabled. The Hibernate 5 review fixes have also been carried through the dependent split branches, with the combined Hibernate 5 core suite passing. The agreed merge order remains: this Hibernate 7 rule and spec must be on 8.0.x before the requested-join parts merge into 7.0.x. |
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A block overwrites a left join the call asked for.
| @@ -221,19 +251,25 @@ protected Object tryAssociationOrJunction(String name, CriteriaMethods method, O | |||
| hibernateQuery.join(name, joinType); | |||
There was a problem hiding this comment.
A criteria query joins club as a left join, then orders by club.name inside a club block. Unattached First Team is left out. The block replaces the left join with an inner join. Keep the left join the call asked for.
matrei
left a comment
There was a problem hiding this comment.
Reviewed at 6de9334 (base 8.0.x): e7fbf6e, b5fecc8 and the merge of #16863's head (87594ea), which merges without a conflict. I ran with Hibernate 7 the probes of my reviews of #17067 to #17074, about 300 queries, with grails.gorm.leftJoinAssociationSort off and on:
- at this head and at #16863;
- compared with 8.0.x (0518f48) and with Hibernate 5 at the top of the 7.0.x stack (#17073, 16bd465).
Against #16863, every result that changes moves to the result of the Hibernate 5 stack, except in points 1, 2 and 4. For example:
| 8.0.x | #16863 | This head | Hibernate 5 stack | |
|---|---|---|---|---|
Player.where { … }.join('team.club', JoinType.INNER).list(), also a criteria query |
7 players | 7 | 6 | 6 |
Team.list(fetch: [club: JoinType.INNER]) |
7 teams | 7 | 5 | 5 |
Team.createCriteria().list(fetch: [club: JoinType.INNER]) { … } |
5 | 5 | 4 | 4 |
Team.where { … }.join('club', JoinType.INNER).updateAll(…) |
5 updated | 5 | 4 | 4 |
Team.where { … }.join('players.contract', JoinType.INNER).updateAll(…) |
7 | 7 | 4 | 4 |
deleteAll with an inner join of a unidirectional collection |
2 deleted | 2 | 1 | 1 |
Team.where { … }.join('club', JoinType.INNER).select('club').list() |
3 clubs instead of teams | 3 clubs | 5 teams | 5 |
Team.createCriteria().list { join('club', JoinType.INNER); join('club') } |
4 | 4 | 5 | 5 |
A paged list that fetches a collection through a path, as in fetch: ['players.contract': JoinType.INNER] with max: 2, fetches the team with all three of its players here. That's the result that I asked Hibernate 5 to have, in point 1 on #17067. I agree that this needs no setting, by #17061, and that the Hibernate 5 parts wait for it.
The tests pass at this head: grails-data-hibernate7-core has 5076 tests, 44 skipped, and grails-datamapping-core 1836 tests, 2 skipped.
Needs a fix
1. With discriminator multi-tenancy, deleteAll through a join table deletes an instance that list leaves out. Cases that aren't multi-tenant, with items that are, in a unidirectional collection. Under one tenant:
Case.where { name ==~ '%case' }.join('items', JoinType.INNER) |
8.0.x and #16863 | This head | Hibernate 5 stack |
|---|---|---|---|
list() |
1 case | 1 | 1 |
updateAll(name: 'X') |
3 | 1 | 1 |
deleteAll() |
3 | 2 | 1 |
The second case that deleteAll deletes has only an item of the other tenant. updateAll matches the one case, so the restriction with the tenant filter is right. The identifiers that deleteByIdentifiers selects first aren't, so the select seems to run without the filter of the subquery. Hibernate 5 deletes the one case.
2. A subquery's join type isn't applied to updateAll and deleteAll, as in point 3 on #17068. Since this head, list applies the join type of a subquery, and the bulk operations still don't:
| #16863 | This head | |
|---|---|---|
new DetachedCriteria(Team).inList('id', Team.where { name ==~ '%First%' }.join('club', JoinType.INNER).id()): list / updateAll |
5 / 5 | 4 / 5 |
the same for deleteAll with two teams, one without a club: list / deleteAll |
2 / 2 | 1 / 2 |
Player.where { team in Team.where { … }.join('club', JoinType.INNER) }: list / updateAll |
4 / 4 | 3 / 4 |
At #16863 these agreed. Now updateAll changes rows that list leaves out.
3. A join type requested inside the block of an association is ignored. Player.createCriteria().list { team { join('club', JoinType.INNER) } } returns all 7 players here, while join('team.club', JoinType.INNER) returns 6. The Hibernate 5 stack returns 6 for both.
4. James's latest comment holds: in a criteria query, the order of join and the block of the association decides the join type.
Team.createCriteria().list { like('name', '%First%'); … } |
This head | Hibernate 5 stack |
|---|---|---|
join('club', JoinType.LEFT); club { order('name') } |
4 teams | 4 |
club { order('name') }; join('club', JoinType.LEFT) |
5 | 4 |
club { order('name') } with list(fetch: [club: JoinType.LEFT]) |
5 | 4 |
or { club { eq('name', 'Chelsea') }; eq('name', 'Unattached First Team') }; join('club', JoinType.LEFT) |
3 | 2 |
Hibernate 5 keeps the inner join of the block in every order, and I asked it to use the requested type instead, in point 1 on #17069. Here the requested type wins when it comes after the block, and loses when it comes before it. Whichever of the two is kept, both modules need it in any order.
5. A bag holds repeated elements. Members are a bag, trophies a set, and the team has two members:
| 8.0.x | This head | Hibernate 5 stack | |
|---|---|---|---|
Team.where { members { name == 'a' } }.join('members', JoinType.LEFT).join('trophies').list() |
the bag holds 4 members | 4 | 2, loaded when read |
Team.list(sort: 'trophies.title', fetch: [members: 'join']), with the setting off and on |
fails | 4 | 4 with the setting off; throws with it on |
The first comes from 8.0.x, and the Hibernate 5 stack fixes it in #17072. The second comes from #16863, where a sort by a collection starts to work. In both, the session holds a collection with elements that aren't there.
6. Player.list(fetch: ['team.players': 'join'], max: 2) fails with Query specified join fetching, but the owner of the fetched association was not present, on 8.0.x and here. The Hibernate 5 stack returns the 2 players, with the team's players fetched. This is the fetch argument of list, which this pull request ports from #17071.
7. Resolve the conflicts with 8.0.x. upgrading80x.adoc conflicts, as #16863 does. No check has run for b5fecc8 or for this head, so the green run is the one of e7fbf6e.
Notes
-
The description lists three results that differ from Hibernate 5 on purpose. The first two are a criteria query that returns an instance once, and a property projection that isn't distinct. #17061 counts a Hibernate 7 result that differs from Hibernate 5 as a regression, so the migration from Hibernate 5 to Hibernate 7 in the guide should name them. The same holds for these, which my probes found:
order('players.name')andTeam.list(sort: 'players.name')return the team once here, and once for each player with Hibernate 5;- a right join: here
listhas anullfor each club that no team refers to, andTeam.list(fetch: [club: JoinType.RIGHT])has thenullas well, where Hibernate 5 has onenull(point 2 on #17067) and applies an inner join (point 2 on #17071); - a projection of a property of an association leaves out the rows without the association (note 8 on #17066).
-
The
countand thetotalCountof a sort through an association leave out the join of the sort, so they count the teams without a club that the list leaves out. For example,Team.createCriteria().list(sort: 'club.name') { like('name', '%First%') }returns 4 teams with atotalCountof 5. That's #16863's, which I noted on #17064. -
This pull request ports #17067, #17068, #17069 and #17071. The other parts of the stack change Hibernate 5 only, and Hibernate 7 returns other results for them:
- #17070: a map of sort properties in a criteria query fails with a
ClassCastException, here and on 8.0.x; - #17072: no exception for a sort by a bag next to the fetch of another collection;
- #17073: a sort by the alias of the queried class fails with
Invalid sort property; - #17074 on 9.0.x: other exceptions for an undefined alias.
The merge-up then leaves the two modules apart there. They need a counterpart here or in a pull request of its own, before those parts merge.
- #17070: a map of sort properties in a criteria query fails with a
-
Team.findAll(new Team(), [sort: 'x.name'])and[sort: 'this.name']returnnullwith Hibernate 7, here and on 8.0.x, wherelistfails withInvalid sort propertyfor the same sorts. That's a bug of its own. -
AbstractDetachedCriteria.getSupersededJoinTypes()is the same new public API on 8.0.x as on 7.0.x (point 5 on #17067). -
The description names e7fbf6e as the change to review. b5fecc8 is on top of it, for James's first comment.
-
As a fix that changes a result, this needs a second reviewer (#17061).
Description
The Hibernate 7 counterpart of the requested join type parts of the split of #16853: #17067, #17068, #17069 and #17071 on 7.0.x. As agreed on dev@, Hibernate 7 gets the same rule and spec on 8.0.x before those merge into 7.0.x, so the merge-up does not leave the two modules returning different rows for the same query. Relates to #15589.
Stacked on #16863. Until that merges, this pull request also shows its commits. The change to review is the last commit, e7fbf6e.
A where query, a detached criteria, a dynamic finder or a criteria query can request the type of the join of an association, with the
fetchargument, withjoin(association, JoinType)or with@Join. Hibernate 7 applied it to the list of a query that requests it for an association of the queried class, and otherwise ignored it. A join type other than a left join is now applied whether or not anything else in the query uses the association.AssociationJoinTypeSpecis the specification of the Hibernate 5 change, ported. On 8.0.x 79 of its 140 cases failed; all pass now. Three things differ from the Hibernate 5 spec on purpose:new DetachedCriteria(Team, 't').eq('t.name', ...)) are left out. Hibernate 7 does not resolve such a path at all, which is a separate fix.Results that change
countand thetotalCountof a paged list count the instances that the list returns, also for a collection, a collection of a basic type and a right join.updateAllanddeleteAllapply it, so they change the instances thatlistreturns. On MySQL and on MariaDB before 10.3 they reject a path that selects from the modified table.list,findWhere,findAllWhereand the example queries apply a join type in theirfetchargument.fetchargument of thelistof a criteria query is applied; it was ignored, for a fetch mode as well.join('author')andselect('author')are fetch modes, as in a where query.joinapplied an inner join, andselectreturned the authors instead of the instances.SemanticException.JoinType.LEFT. For a criteria query that relied onjoin('author')leaving out instances, requestjoin('author', JoinType.INNER).The change to
AbstractDetachedCriteriaandDynamicFinderingrails-datamapping-coreis the same one that #17067 makes on 7.0.x.The upgrade guide has a note under Grails 8.0.1, and the upgrade skill has the same entry.
Verification
grails-data-hibernate7-core(5074 tests) andgrails-datamapping-core(1836 tests) pass, andcodeStylepasses for both. Only H2 was used; the MySQL and MariaDB guard forupdateAllanddeleteAllis ported from the Hibernate 5 change and was not run against those databases. The full build was not run locally.Contributor Checklist
Please review the following checklist before submitting your pull request. Pull requests that do not meet these requirements may be closed without review.
Issue and Scope
7.0.x): Bug fixes only. No new features or API changes.7.1.x): New features are welcome, but breaking existing APIs must be avoided.8.0.x): Reserved for major changes. Breaking API changes are permitted.Code Quality
./gradlew build --rerun-tasks../gradlew codeStyleand resolved any violations. See Code Style for details.Licensing and Attribution
Documentation