Repository navigation
Apply a requested join type to updateAll and deleteAll - #17068
jdaugherty wants to merge 18 commits into
Conversation
Sorting a where query by a property of an association, such as sort: 'club.name', failed with a QueryException when the query also had criteria on that association. The criteria join the association under a generated alias, and the order was added with the association path, which Hibernate cannot resolve. The order now uses the alias of the join.
A sort by a path through more than one association, such as owner.employer.name, is still not supported.
This reverts commit b494caa.
A sort by a path that crosses more than one association, such as team.club.name, failed with a QueryException whether or not the where query joined the nested association, because order() split the path at its first dot only. The sort property is now qualified with the alias of the last association on its path: an association the query has already joined is reused, and one it has not joined yet is joined for the sort and registered like any other join. A sort by two properties of the same association no longer fails with "duplicate association path" either, since the sort join is registered instead of created anew for every order.
Since order() walks the whole sort path, a sort by the identifier of an association that follows another one on the path, such as team.club.id, joined that last association too. The join is an inner join unless fetch requests another type, so the sort left out the rows where the association is null. Before, only the first association was joined and Hibernate read club.id from the foreign key of team, which kept them. The identifier is sorted by the foreign key again when the entity before the association holds it. An association without such a foreign key, a hasOne or the inverse side of a one-to-one, is still joined, and so is the first association of a path, as it always has been. The spec also covers the join type of a sort join: an inner join by default, and a left join when fetch requests one. Team.club is nullable in the fixture for these cases. The docs describe the inner join, how to request a left join, and the identifier of a nested association.
list with a map of sort properties, such as Team.list(sort: ['club.name': 'asc', name: 'desc']), sorted by the last entry only, because each entry replaced the orders of the query. The same held for a default sort that the mapping declares for more than one property. Every entry is now applied, in the order of the map. An association is joined once for all the sort properties that use it. list joined it once for each, and find and findAll with an example, findWhere and findAllWhere failed with "duplicate association path". Part 2 of the split of apache#16853.
A paged query of a detached criteria that has an alias for the queried
class, and criteria that use it, failed when totalCount was read:
new DetachedCriteria(Team, 't').eq('t.name', 'Arsenal').list(max: 2).totalCount
The count runs on a copy of the criteria, and the copy was created
without the alias, so Hibernate could not resolve t.name. The copy now
has the alias of the original.
Part 3 of the split of apache#16853.
count on a where query or a detached criteria that has a sort, such as
Team.where { ... }.sort('name').count(), kept the order by in the query
that counts the rows, which H2 rejects with "must be in the GROUP BY
list". A query that only counts its rows now has no orders.
The join that a sort by a property of an association makes is kept, so
the count is that of the rows which list returns for the same query.
Part 4 of the split of apache#16853.
… by it A sort by a property of an association joins the association with an inner join, so it leaves out the instances whose association is null: Team.list(sort: 'club.name') does not return the teams without a club, and count and the totalCount of a paged list do not count them. The new grails.gorm.leftJoinAssociationSort setting makes an association that is joined only for a sort a left join, in where queries, detached criteria, dynamic finders, list, find and findAll with an example, findWhere, findAllWhere and a criteria query that is passed pagination arguments. It is disabled by default, so the results of existing applications do not change in a patch release. Grails 8.1 enables it by default. With the setting, a sort by a nested identifier, such as team.club.id, goes through the left join. Without it, it is still read from the foreign key. A sort by a property of a collection keeps its inner join: Hibernate fetches a collection through a left join, which is handled separately. Part 5 of the split of apache#16853.
Team.where { ... }.join('club', JoinType.INNER) lists and counts the
teams with a club, but deleteAll and updateAll on it ignored the join
type and deleted or updated the teams without a club as well. They now
modify the instances that list returns.
An update or a delete cannot join, so each requested join becomes a
restriction: a subquery that selects from the entity of the first
association on the path and joins the rest of it. deleteAll selects the
identifiers first when the path starts with a collection that has a
join table, because Hibernate deletes the join rows before the
instances with the same restriction. A one-to-one whose foreign key the
other class has is matched through the property that refers back.
MySQL, and MariaDB before 10.3, do not allow an update or a delete to
select from the table that it modifies. For a join that leads back to
the queried class, these dialects get an UnsupportedOperationException
that says to restrict the query with criteria instead.
Part 8 of the split of apache#16853.
A where query, a detached criteria and a dynamic finder joined an association as soon as criteria, a projection or a sort needed it. The rest of the query can still change after that: criteria passed to list are added later, and the fetch argument of list, which can request a join type, is read last. The result depended on the order in which the query was put together: - a join type in the fetch argument of list was not used for the join that the criteria or a projection had already made; - a sort stored on the query with sort() was joined before a join type requested in list was known, and criteria passed to list could not join the same association through an alias; - a join type was looked up by the name of the association, so a type requested for club was used for team.club, and one requested for team.club was not; - criteria on a nested association used twice failed with "duplicate association path". Criteria, projections and orders only need the alias of a join. A join is now registered under its path from the root entity and its alias, and made when the query is executed, or when a subquery is handed to the query that embeds it, with the join type that the query requests for that path. A listener of the query events that adds criteria on an association to find and findAll with an example, findWhere or findAllWhere gets its joins made after the event. Part 6 of the split of apache#16853.
A where query, a detached criteria or a dynamic finder can request a
join type for an association, with fetch: [club: JoinType.INNER] or
join('club', JoinType.INNER). The type was only used when criteria or
a sort joined the association. Otherwise the association was fetched
with the outer join of its fetch mode, so
Team.where { ... }.list(fetch: [club: JoinType.INNER]) returned the
teams without a club, and the same query sorted by club.name did not.
A requested join type other than LEFT is now applied when the query is
executed, to list, get, find, count, the totalCount of a paged list, a
subquery and a query with a projection:
- INNER and RIGHT on an association to a single instance join and
fetch it; a right join is a right join, not an outer fetch;
- a path that ends in a collection or passes through one restricts the
query with a correlated subquery, so that the collection stays
fetched with all of its elements, each instance is counted once, and
the filters of the session, such as the tenant filter, apply;
- every association on a requested path is joined and fetched;
- a collection of a basic type is handled like any other collection;
- a fetch mode requested afterwards replaces the join type, on the
query and on the detached criteria, while the join that criteria on
the association make keeps it.
Part 7 of the split of apache#16853.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A bulk write with an inner join updates every row, including one with no association.
| if (associatedEntity == null && !(association instanceof Basic)) { | ||
| return null; | ||
| } | ||
| String joinAlias = "join" + index + '_' + i; |
There was a problem hiding this comment.
One row has a category. Another has none. An inner join of category and updateAll should change only the row that has a category. Both rows change, because the join's alias is the same as the outer alias. Give the join its own alias.
There was a problem hiding this comment.
Fixed in b3ae39f. The generated aliases for a requested bulk join and its owner subquery are now derived from the outer alias, so neither can shadow it.
The new public-API regression uses an entity whose decapitalized name equals the old generated alias, and checks both updateAll and deleteAll. Both failed before the fix; the requested-join specs in both sort-setting modes and the dialect spec now pass: 215 cases, 208 passed and 7 skipped. The module codeStyle check passes too.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A bulk write ignores a join that a subquery asked for.
| if (!(criteria instanceof AbstractDetachedCriteria)) { | ||
| return query; | ||
| } | ||
| Map<String, JoinType> joinTypes = ((AbstractDetachedCriteria<?>) criteria).getJoinTypes(); |
There was a problem hiding this comment.
A subquery joins author as an inner join, so a book with no author is left out of the list. The same subquery in a bulk update still changes that book, because the join is ignored. Apply the join the subquery asked for.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A bulk write ignores a join that a subquery asked for.
| if (!(criteria instanceof AbstractDetachedCriteria)) { | ||
| return query; | ||
| } | ||
| Map<String, JoinType> joinTypes = ((AbstractDetachedCriteria<?>) criteria).getJoinTypes(); |
There was a problem hiding this comment.
A subquery joins author as an inner join, so a book with no author is left out of the list. The same subquery in a bulk update still changes that book, because this only looks at the outer criteria. Apply the join the subquery asked for.
matrei
left a comment
There was a problem hiding this comment.
Reviewed at 07b71ee (base 7.0.x): ae6dbd1, b3ae39f and the merge of #17067's head, which leaves the diff of the two commits as it was. I compared list and count with updateAll and deleteAll on queries that request a join type:
- at this head, at #17067 (1814db6), on 7.0.x (aee4c04) and at the stack top (16bd465);
- with Hibernate 7 on 8.0.x (0518f48) and at #17079 (6de9334);
- for a to-one association, a right join, a nested path, a path through a collection, a
hasOne, a unidirectional collection with a join table, a many-to-many from both sides, and a collection of a basic type; - for a query without criteria, with a root alias, with two requested joins, and with a join type that a later fetch mode replaced;
- with discriminator multi-tenancy, for collections whose elements belong to tenants.
Without multi-tenancy, updateAll and deleteAll change here the rows that list returns in each of these cases, as at #17079. A subquery is the exception (point 3).
| 7.0.x | #17067 | This head and #17079 | |
|---|---|---|---|
Team.where { name ==~ '%First%' }.join('club', JoinType.INNER).updateAll(name: 'X'), also JoinType.RIGHT and join('players', JoinType.INNER) (list: 4) |
5 updated | 5 | 4 |
Player.where { … }.join('team.club', JoinType.INNER).updateAll(…) (list: 6) |
7 | 7 | 6 |
Team.where { … }.join('players.contract', JoinType.INNER).updateAll(…) (list: 4) |
7 | 7 | 4 |
deleteAll with an inner join of a unidirectional collection, or of a many-to-many (list: 1) |
2 deleted | 2 | 1 |
At #17067, list and count already leave the rows out while updateAll and deleteAll don't, so the two parts belong in the same release, as the description says. No setting, agreed.
Needs a fix
1. With discriminator multi-tenancy, updateAll fails for an inner join of a collection whose multi-tenant elements refer back to their owner. Shelves that aren't multi-tenant, with books that are. Under one tenant, list returns only the shelf that has a book of that tenant. updateAll with the same join fails with a syntax error:
Shelf.where { name ==~ '%shelf' }.join('books', JoinType.INNER) |
7.0.x | #17067 | This head | #17079 |
|---|---|---|---|---|
list() |
2 shelves | 1 | 1 | 1 |
updateAll(name: 'X') |
2 updated | 2 | throws | 1 |
The SQL that H2 rejects:
update mt_shelf set name=? where (name like ?) and (exists (select 1 from mt_book mtbook1_ where ? = mtbook1_.tenant_id and (mtbook1_.id in (select books2_.id from mt_book books2_ where mt_shelf.id=books2_.shelf_id and :tenantId.tenantId = books2_.tenant_id))))
Hibernate 5 renders MEMBER OF on a one-to-many as a subquery and leaves the parameter of the tenant filter in it unbound. A call that worked on 7.0.x now throws.
A unidirectional collection, which has a join table, works. updateAll and deleteAll change the one case that list returns there. For a collection whose elements refer back to the owner, matching them through that property, as in book.shelf = shelf, avoids MEMBER OF, as the commit already does for a hasOne. It would also keep the tenant filter in the one subquery.
2. The MySQL exception should be in "Results that change". On MySQL, and on MariaDB before 10.3, updateAll and deleteAll with a join type for a path back to the queried class, such as join('parent', JoinType.INNER), now throw an UnsupportedOperationException. The same holds for a path that starts with a collection of a basic type. On 7.0.x these calls updated or deleted rows. The note says so, but the description doesn't. It's a call that worked and now fails, which by #17061 the description has to name.
deleteAll through a join table already selects the identifiers first, in deleteByIdentifiers. In a statement of its own, that select doesn't run into the restriction of MySQL, which applies to a subquery of the update or delete itself. Doing the same for these paths, and for updateAll, would make them work there instead of throwing. Then the dialect check and the exception wouldn't be needed.
3. James's second comment holds: a subquery's requested join isn't applied to updateAll and deleteAll. Since #17067, list applies the join type that a subquery requests. applyRequestedJoins reads only the join types of the outer criteria:
list |
updateAll / deleteAll here, at the stack top and at #17079 |
|
|---|---|---|
new DetachedCriteria(Team).inList('id', Team.where { name ==~ '%First%' }.join('club', JoinType.INNER).id()), also Team.where { id in … } |
4 teams | 5 updated |
the same for Team.where { name == 'Empty Team' || name == 'Playerless First Team' }, one of them without a club |
1 | 2 deleted |
So the description's "They now modify the instances that list returns" doesn't hold for these. #17079 has the same gap.
4. Add run ci, and resolve the conflicts with 7.0.x, as on #17066 and #17067. Locally, the tests of grails-data-hibernate5-core pass at this head (1096 tests, 55 skipped).
Notes
-
The description names ae6dbd1 as the change to review. b3ae39f is on top of it now.
-
updateAllanddeleteAllwith criteria on an association, such asTeam.where { club.name == 'Chelsea' || name == 'Unattached First Team' }, throwJoins cannot be used in a DELETE or UPDATE operationon 7.0.x and on 8.0.x, with or without a join type. That isn't for this pull request. The description's "They now modify the instances that list returns" holds for queries without such criteria. -
With Hibernate 7 at #17079,
deleteAllof the cases with an inner join of their multi-tenant items deletes 2 whilelistreturns 1. I'll raise that on #17079. -
The imports of
AssociationSortSpecandWhereQueryWithAssociationSortSpecmove again, as in #17066 and #17067. -
As a fix that changes a result, this needs a second reviewer (#17061).
Description
Part 8 of the split of #16853, as agreed there and in the dev@ thread Fixes that change results on patch and minor branches. Relates to #15589.
Stacked on #17067. Until that merges, this pull request also shows the commits below it. The change to review is the last commit, ae6dbd1.
Team.where { ... }.join('club', JoinType.INNER) lists and counts the
teams with a club, but deleteAll and updateAll on it ignored the join
type and deleted or updated the teams without a club as well. They now
modify the instances that list returns.
An update or a delete cannot join, so each requested join becomes a
restriction: a subquery that selects from the entity of the first
association on the path and joins the rest of it. deleteAll selects the
identifiers first when the path starts with a collection that has a
join table, because Hibernate deletes the join rows before the
instances with the same restriction. A one-to-one whose foreign key the
other class has is matched through the property that refers back.
MySQL, and MariaDB before 10.3, do not allow an update or a delete to
select from the table that it modifies. For a join that leads back to
the queried class, these dialects get an UnsupportedOperationException
that says to restrict the query with criteria instead.
Results that change
updateAllanddeleteAllapply the join type that the query requests, so they change the rows thatlistreturns. This is the other half of the previous branch: without itlistwould return 4 rows anddeleteAlldelete 5.JoinType.LEFT.The upgrade guide has a note under Grails 7.0.18, and the upgrade skill has the same entry.
Verification
The association sort and join specs of
grails-data-hibernate5-corepass on this branch, withgrails.gorm.leftJoinAssociationSortoff and on, andcodeStylepasses for the touched modules. The full build was not run locally.Contributor Checklist
Please review the following checklist before submitting your pull request. Pull requests that do not meet these requirements may be closed without review.
Issue and Scope
7.0.x): Bug fixes only. No new features or API changes.7.1.x): New features are welcome, but breaking existing APIs must be avoided.8.0.x): Reserved for major changes. Breaking API changes are permitted.Code Quality
./gradlew build --rerun-tasks../gradlew codeStyleand resolved any violations. See Code Style for details.Licensing and Attribution
Documentation