Repository navigation
Hibernate 7: write the rows of unidirectional hasMany collections that share a join table - #16701
Conversation
… that share a join table CollectionKeyColumnUpdater made the key of a unidirectional to-many not updatable when the owner had several of them sharing one join table, the rule of issue 10811. Hibernate 7's collection persister writes no rows for a collection whose key is not updatable, so every addTo* on such an owner was dropped silently, even with failOnError. The key is now always updatable. Hibernate creates the implicit unique key of the shared table for the first collection only, over its key column and the element column, so a row of a later collection leaves that key column null and two such rows for one element are duplicates on a database that treats nulls as equal in a unique constraint. The new SharedJoinTableUniqueKeyBinder, run by GrailsSecondPass after the keys are created, adds the key columns of every sharing collection to that unique key. SharedJoinTableUniqueKeyBinderSpec pins the unique key of a shared table, of tables of their own and of a bidirectional one-to-many; CollectionKeyColumnUpdaterSpec expects the shared-table key to be updatable; MultipleUnidirectionalHasManySpec writes both collections of a shared table and one role denied by two owners. Fixes #16699
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## fix/h7-multiple-hasmany-8.0.x #16701 +/- ##
=======================================================================
- Coverage 62.1164% 62.1155% -0.0009%
+ Complexity 26777 26774 -3
=======================================================================
Files 2237 2237
Lines 110433 110433
Branches 20152 20152
=======================================================================
- Hits 68597 68596 -1
Misses 32616 32616
- Partials 9220 9221 +1 🚀 New features to boost your workflow:
|
jamesfredley
left a comment
There was a problem hiding this comment.
Not approving. Making the shared-table keys updatable is the right direction, and the separate-table behavior from 16686 is left alone. The unique-key change is not safe on the databases Grails actually runs.
Widening the implicit unique key with the sibling collection's nullable key column does not preserve per-association uniqueness where NULLs are distinct. Every normal row for one collection leaves the other collection's key NULL, so the widened constraint accepts duplicate membership. Confirmed with a NULL-distinct probe: the original (roles_id, role_id) constraint kept one row, and UNIQUE(roles_id, deny_roles_id, role_id) admitted two identical (1, NULL, 7) rows. PostgreSQL's default NULL-distinct unique constraints do the same. With version false, two sessions that each initialize the same empty collection and add the same existing role can both insert.
| } | ||
| for (Column column : keyColumns) { | ||
| if (!uniqueKey.containsColumn(column)) { | ||
| uniqueKey.addColumn(column); |
There was a problem hiding this comment.
Adding the sibling key here weakens the constraint it is meant to extend. For roles and denyRoles sharing a join table, the implicit key becomes UNIQUE(roles_id, deny_roles_id, role_id). A real roles row is (roles_id=1, deny_roles_id=NULL, role_id=7). PostgreSQL, MySQL, and default H2 treat those NULLs as distinct, so that row can be inserted twice. The old (roles_id, role_id) constraint rejected the duplicate.
Keep a per-association uniqueness constraint on (own_collection_key, element) for NULL-distinct dialects. Where the dialect treats NULLs as equal, use a null-safe or partial index instead of one widened constraint that is NULL for every normal row.
Please add a regression that rejects duplicate membership of the same role in the same collection, and still allows that role in the other collection and in another owner. The current specs do not catch this.
There was a problem hiding this comment.
Reworked in 9066f90, by dialect. You are right that one widened key rejects nothing where nulls are distinct, and that is most of what Grails runs on. The constraint now takes the shape the database needs, through SharedJoinTableUniqueness, keyed on the dialect the way RegexDialectPattern is:
- Default (H2, PostgreSQL, MySQL, MariaDB, and SQL Server and DB2, whose Hibernate unique delegates render a filtered unique index): one unique key per association over its own key column and the element. The binder keeps the key Hibernate made for the first association and creates the missing one for each later association.
- Oracle: one unique key over the key columns of every sharing association and the element. Oracle compares the non-null columns of a partially null row, so a row collides only with a row of its own association for the same element, while a key per association would reject the rows of the other association, null in that key column and equal in the element.
The binder is no longer constructed inside a method: it is built once in CollectionBinder with the metadata context, injected into CollectionSecondPassBinder, which now owns key creation, and the GrailsSecondPass interface is gone.
Tests: SharedJoinTableHibernate7Spec runs the scenario on H2, PostgreSQL, MySQL, MariaDB and Oracle through Testcontainers, as RLikeHibernate7Spec does: the same member twice in one association of one group, from two sessions on a version false owner, fails with DataIntegrityViolationException, while the same member in the other association, in another group, and denied by two groups is accepted. SharedJoinTableUniqueKeyBinderSpec renders the DDL for each dialect with the dialect's own exporters, SQL Server's filtered index and DB2's exclude null keys index included, so the two without containers are asserted too.
| int before = children.collectionTable.uniqueKeys.size() | ||
|
|
||
| when: | ||
| new Object() |
There was a problem hiding this comment.
This does not test the guard. new Object() never calls the binder, and children has both a table and a key, so neither advertised condition is exercised. The assertion passes no matter what bind does.
Replace it with a real call that hits each early return (no table, inverse, one-to-many, null key), or delete the feature. The bidirectional one-to-many feature above already covers the inverse case through the public mapping path; keep that.
There was a problem hiding this comment.
Replaced in 9066f90: the feature now calls bind on the bidirectional collection from the metadata, the one whose key the other side owns, and asserts the table gains neither a unique key nor an index. The dialect resolution has a row per dialect, and the DDL feature applies the uniqueness to a hand-built shared table and renders it with each dialect's exporters.
…table-key-8.0.x # Conflicts: # grails-data-hibernate7/core/src/test/groovy/grails/gorm/tests/MultipleUnidirectionalHasManySpec.groovy
…shape each database needs From the review of #16701. One unique key widened with the key columns of every association sharing the table holds a null in every row, so on a database that treats nulls as distinct, which is H2, PostgreSQL, MySQL and MariaDB, and on SQL Server and DB2, whose Hibernate unique delegates render a filtered unique index, it rejected nothing: the duplicate membership the original key caught went through. What a row of a shared table needs is a constraint over the key column of its own association and the element. SharedJoinTableUniqueness holds the shape by dialect, keyed on the dialect class as RegexDialectPattern is: DEFAULT gives every association a unique key of its own, keeping the one Hibernate creates for the first; ORACLE keeps the single key over every key column and the element, since Oracle compares the non-null columns of a partially null row, so a row collides only with a row of its own association for the same element, and a key per association would reject the rows of the other association there. The binder is no longer constructed inside a method: CollectionBinder builds it once with the metadata context and hands it to CollectionSecondPassBinder, which now owns key creation (createCollectionKeys, bindSetSecondPass); the list and map binders end their passes with it, the three second passes call their binders only, and the GrailsSecondPass interface is gone. SharedJoinTableHibernate7Spec runs the scenario on H2, PostgreSQL, MySQL, MariaDB and Oracle through Testcontainers: the same member twice in one association of one group, from two sessions on a version-less owner, fails with DataIntegrityViolationException, while the same member in the other association, in another group, and denied by two groups is accepted. SharedJoinTableUniqueKeyBinderSpec applies the uniqueness to a hand-built shared table and renders the DDL with each dialect's exporters, SQL Server's filtered index and DB2's exclude-null-keys index included, resolves every dialect, and calls bind on an inverse collection to show it is left alone. The upgrade notes and the one-to-many chapter say which key each database gets.
…from its dialect A bracket may quote an identifier or open an array, and the two never coexist on one database, so SqlText no longer guesses from the character before it. SqlSyntax, keyed on the Hibernate dialect, says what a text may hold: TRANSACT_SQL (SQL Server, Sybase) quotes identifiers with brackets, closed by the first ] not written ]], never nested; SQL (every other database) nests brackets like parentheses, so ANY(ARRAY [CAST(? AS INTEGER)]) and "items"[?] keep their placeholders; HQL has neither. Every reader resolves the syntax from what it holds: the criteria builder from its session factory, the query creator from Hibernate's criteria builder, the function contributors from the function contributions, the detached-criteria extension from the entity's datastore. SqlProjection.of, SqlGroupProjection.of and the new SqlRestriction.of take it; the restriction record no longer counts placeholders itself. The alias and unquoting regexes of SqlProjection move into SqlText, so nothing outside it scans SQL. The dialect resolution shared by RegexDialectPattern and SqlSyntax is one interface, DialectKeyed, which the shared join table uniqueness of #16701 adopts after this. HibernateCriteriaBuilderTransactSqlSpec runs H2 in its SQL Server compatibility mode under SQLServerDialect, so bracket identifiers, a ? inside one, a bracket alias with an apostrophe and a bracket group-by name go through the real render path. The H2 and container specs that used bracket aliases on databases without bracket identifiers use double quotes and backquotes now.
jamesfredley
left a comment
There was a problem hiding this comment.
The shared-set unique key is per association, and the absence test would fail if the binder were removed. Not approving. The same binder also forces set semantics on ordinary lists and maps.
jdaugherty
left a comment
There was a problem hiding this comment.
Not approving yet. The direction stands: the key is updatable, the shared table is unique per association in the shape each database needs, and the five-database spec covers the two cases that tell the shapes apart. One regression blocks it, the one jamesfredley raised on SharedJoinTableUniqueness, and it is wider than lists and maps: a Collection-typed hasMany (a bag) goes through SetSecondPass as well, so it gets the same unique key. Reproduced on H2 at 9066f90 against its base 8b7515a for six properties, each of which saves a repeated element on the base and fails with DuplicateKeyException here; the table of results is on SharedJoinTableUniqueKeyBinder.bind. The fix is a one-line guard (collection.isSet()) plus tests for the three kinds.
Two non-blocking notes: GrailsSecondPass was public in v8.0.0 and this patch removes it, and the H2 row of SharedJoinTableHibernate7Spec is skipped along with the container rows when Docker is absent.
| /** Gives the collection a uniqueness constraint over its key columns and its element on its join table. */ | ||
| public void bind(Collection collection) { | ||
| Table table = collection.getCollectionTable(); | ||
| if (table == null || collection.isInverse() || collection.isOneToMany() || collection.getKey() == null) { |
There was a problem hiding this comment.
This guard lets every collection with a join table through, and createCollectionKeys is now called from the list and map binders as well as from bindSetSecondPass, which the bag of a Collection-typed property also goes through (CollectionBinder.registerSecondPass, the else branch). On the DEFAULT path the table of each of those gets a unique key over (key, element) that Hibernate never made: their primary key is (key, index), or absent for a bag, so coveredByPrimaryKey and present are both false and SharedJoinTableUniqueness line 99 adds one.
Reproduced at 9066f90 against its base 8b7515a with one spec on H2 (:grails-data-hibernate7-core:test): six properties of two owners, each saved with a repeated element and read back after a session clear.
| property | base | this PR | unique index the PR adds (H2 information_schema.indexes) |
|---|---|---|---|
List<String> nicknames = ['a', 'a'] |
passes | DuplicateKeyException |
(p16701_owner_id, nicknames_java_lang_string) |
List<Role> roles = [role, role] |
passes | DuplicateKeyException |
(p16701_owner_roles_id, p16701_role_id) |
Map<String, String> attrs = [k1: 'v', k2: 'v'] |
passes | DuplicateKeyException |
(p16701_owner_id, attrs_elt) |
Map<String, Role> roleMap = [a: role, b: role] |
passes | DuplicateKeyException |
(p16701_owner_role_map_id, p16701_role_id) |
Collection<String> labels = ['x', 'x'] |
passes | DuplicateKeyException |
(p16701_bag_owner_id, labels_java_lang_string) |
Collection<Role> bagRoles = [role, role] |
passes | DuplicateKeyException |
(p16701_bag_owner_bag_roles_id, p16701_role_id) |
The base schema has no unique index on any of the six tables. Oracle is not affected, since its path only widens a unique key that already exists, so this is every other database.
A list, a map and a bag may hold an element twice, so the binder has to skip them: return here unless collection.isSet() (a sorted set is a Set too). With that in place the existing coveredByPrimaryKey and present checks already leave a set on a table of its own alone, so the "shared" part needs no sibling lookup.
There was a problem hiding this comment.
Fixed in f1bdd12: bind returns unless collection.isSet(). I reproduced your six properties first: with the guard removed, a list, a map and a bag with a repeated element fail to save with a duplicate key on H2, PostgreSQL, MySQL and MariaDB. A bag went through the set second pass and got the key as well, as you found. With the guard, the set on a table of its own is still left alone by the primary key and present checks, as you said.
| denyRoles.key.updateable | ||
| } | ||
|
|
||
| void "an association with a join table of its own keeps the unique key Hibernate creates for it"() { |
There was a problem hiding this comment.
When the guard is narrowed, please cover the three collection kinds it excludes here, from the metadata as this feature does: a List, a Map and a Collection (bag) hasMany whose tables gain no unique key. And in MultipleUnidirectionalHasManySpec, or a spec of its own, a save and read-back of a list with a repeated element, a map with one value under two keys, and a bag with a repeated element. The module suite was green at this commit, so nothing in it saves a repeated element into a list, map or bag today.
There was a problem hiding this comment.
Added in f1bdd12. SharedJoinTableUniqueKeyBinderSpec binds, from the metadata, a List<String>, a List<Role>, a Map<String, String>, a Map<String, Role>, a Collection<String> and a Collection<Role> hasMany, on the H2 and the Oracle dialect, and asserts none gains a unique key or an index (the lists and maps have their primary key, the bags none); it fails for all twelve rows without the guard. MultipleUnidirectionalHasManySpec saves and reads back a repeated element in each of the six on plain H2, and SharedJoinTableHibernate7Spec does the same on H2, PostgreSQL, MySQL, MariaDB and Oracle. Each collection that points at the same class maps its own join table in those entities, because two associations to one class with a list among them share a table, which is a separate matter.
|
|
||
| import org.hibernate.mapping.Collection; | ||
|
|
||
| public interface GrailsSecondPass { |
There was a problem hiding this comment.
GrailsSecondPass is a public interface that shipped in v8.0.0 (added in 9c8ba25), and this removes it in the 8.0.1 patch. Nothing in the repository references it any more, and a mixin with one default method is unlikely to have implementors outside, but a patch release deleting a public type is not something a user can see coming. Deprecating it for one release, or at least a line in the PR body saying it is gone, would be the safer shape. Non-blocking.
There was a problem hiding this comment.
Kept in f1bdd12 as a deprecated type, with its original default method and a javadoc that says the second passes no longer implement it and what creates the keys now. Nothing in the repository references it.
| * column breaks where nulls are distinct. | ||
| */ | ||
| @Testcontainers | ||
| @Requires({ isDockerAvailable() }) |
There was a problem hiding this comment.
The class-level @Requires skips the H2 row together with the container rows, so without Docker the rejection of a repeated member in the same association runs nowhere: MultipleUnidirectionalHasManySpec writes to the shared table but never asserts that a duplicate is refused, and the binder spec checks the metadata, not the database. An H2-only feature for that case in MultipleUnidirectionalHasManySpec would keep the regression test in the plain module run. Non-blocking.
There was a problem hiding this comment.
Added in f1bdd12, in MultipleUnidirectionalHasManySpec so it runs without Docker: the same role allowed twice by one account of a shared table is refused with a DataIntegrityViolationException, while denying the same role by the same account is accepted, because it is the other association. The container spec keeps the rejection across the other databases.
…e's unique index treats a null, and only a set gets it Whether a unique index leaves a row with a null in its key out or compares the other columns is one fact about each database. UniqueIndexNulls states it once, with the reason for each database, and the shape of the uniqueness of a shared join table follows it: a key per association where nulls are distinct, one key over every association's key column where they are compared. Dialects are resolved through DialectKeyed, as the regular expression operator is. Only a set gets the constraint. A list is keyed by (owner, index) and a map by (owner, key), and their elements repeat by design: the unique key over the owner and the element made a list ['x', 'x'], or a map with one value under two keys, fail to save. SharedJoinTableHibernate7Spec saves both on H2, PostgreSQL, MySQL, MariaDB and Oracle, and the binder spec fails without the guard.
…st, a map and a bag keep repeated elements A bag, a Collection-typed hasMany, goes through the set second pass as well and got the unique key over the owner and the element, so a repeated element failed to save, as it did for a list and a map. The binder acts on collection.isSet() only. SharedJoinTableUniqueKeyBinderSpec covers the six kinds from the metadata, MultipleUnidirectionalHasManySpec saves and reads back a repeated element in each kind and refuses the same member twice in one association of a shared table on plain H2, and SharedJoinTableHibernate7Spec does the saves on the five databases; all of them fail without the guard. GrailsSecondPass, public since 8.0.0, is kept, deprecated, for code that implements or calls it.
jamesfredley
left a comment
There was a problem hiding this comment.
The earlier comment is fixed. Only a set gets the shared-table unique key. A list, a map, and a bag can repeat values. That thread is resolved. Not approving. Enabling the shared key also enables a bulk delete that removes the other collection, and a few uniqueness shapes still reject valid rows.
| if (keyColumns.isEmpty() || elementColumns.isEmpty()) { | ||
| return; | ||
| } | ||
| uniqueness.apply(table, keyColumns, elementColumns, new ImplicitNaming(table)); |
There was a problem hiding this comment.
Hibernate skips a unique key when a set element has a CLOB or BLOB, because that column cannot be indexed. This call adds the key anyway. On H2, unique (owner_id, clob_column) fails table creation.
Keep that exclusion. Do not treat a missing key on an unshared table as an incomplete shared-table mapping.
There was a problem hiding this comment.
Fixed in 1e99b6e: the unique key is not added when an element column is a large object, as in Set.createPrimaryKey. A basic collection's element cannot be mapped to a LOB through the GORM mapping (type: applies to the property), so the check is tested against a real CLOB column from the metadata (SharedJoinTableUniqueKeyBinderSpec, hasLobColumn).
…and leave a large object element without a key Hibernate removes all the rows of a collection with one statement on its key column. Two collections that share a join table and a key column, each with an element column of its own, lost each other's rows when one was cleared, now that the key is updatable. Each collection of a shared table is restricted to the rows that have a value in its element columns, which limits that delete and the load alike. A collection is shared when another non-inverse binding of the metadata uses the same table. The unique key over the key column and the element is not added when an element column is a large object, which a database cannot index and for which Hibernate leaves the set without a key too (Set.createPrimaryKey). Claude-Session: https://claude.ai/code/session_01MDHn7fC3hsGJ27vFHD2o1p
…n table, and keep the second-pass mixin On Oracle, a join table with one key column and an element column per association kept Hibernate's key over the key column and the first element, so the rows of the second association, null in that element, collided with each other. The one key now takes the key and element columns of every association that shares a column with it, which covers both shapes. SharedKeyColumnHibernate7Spec runs the mapping on five databases and fails on Oracle without it. SetSecondPass, ListSecondPass and MapSecondPass implement the deprecated GrailsSecondPass again, so code that casts to it keeps compiling for the release the interface is kept. Claude-Session: https://claude.ai/code/session_01MDHn7fC3hsGJ27vFHD2o1p
jamesfredley
left a comment
There was a problem hiding this comment.
The two earlier comments are fixed. Clearing one shared collection no longer deletes the other, and a set with a LOB element no longer gets a unique key. Those threads are resolved. Not approving. Removing one element of a shared set can still delete the other element.
| // second key column to leave null. A to-many that is not bound as a collection (a serializable or user type) | ||
| // has no join table and no Hibernate collection, so it is neither counted nor asked for one. | ||
| key.setUpdateable(property instanceof HibernateToManyCollectionProperty || unidirectionalCount <= 1); | ||
| key.setUpdateable(true); |
There was a problem hiding this comment.
When a shared-table set has two elements, removing one deletes both. Hibernate's row-delete predicate omits the nullable element columns, and is not null does not tell the two elements apart. The pending removal tests in SharedJoinTableBulkDeleteSpec show this. The key used to be non-updatable, so this path is new.
Fix the row delete, or rewrite the surviving rows, before enabling updates. Please remove one element and assert the other remains.
#16745 was closed as a duplicate of #16691, the issue the reporter opened first. Claude-Session: https://claude.ai/code/session_01MDHn7fC3hsGJ27vFHD2o1p
…ts restriction Hibernate 7 tests the emptiness of a collection with an exists subquery over a join to the collection. For a many-to-many collection it puts the collection's restriction (the where fragment of its join table) on the join to the element, which the select 1 of that subquery never renders, so is empty and is not empty lose it; size() and member of apply it to their subquery directly. A collection that shares a join table and a key column with another one carries such a restriction, so the test saw the rows of the other collection (#16748). RestrictedCollectionSqmTranslatorFactory extends Hibernate's SQM translator and, for a many-to-many collection of an entity that has a restriction, translates `is empty` as `size(...) = 0` and `is not empty` as `size(...) > 0`. Every other collection keeps Hibernate's exists subquery, a restricted one-to-many among them (its restriction stays on the main join). GORM registers it as hibernate.query.sqm.translator unless the application sets one; a dialect with a translator of its own (Sybase) keeps it. HQL, criteria and single-table update and delete are covered; Hibernate's multi-table mutation converter is not reached, so an update or delete of an entity whose hierarchy spans several tables still sees the other collection's rows. SharedJoinTableBulkDeleteSpec: the pending HQL feature now passes, with new features for criteria isEmpty/isNotEmpty, a negated is empty, an update restricted by is empty, associations with a key column each, and a collection on a join table of its own; without the registration the shared-table features fail. The spec cleans its rows after each feature. RestrictedCollectionSqmTranslatorFactorySpec checks the registration, an application's own translator, a dialect's own translator, and a restricted one-to-many of a table-per-hierarchy subclass. Claude-Session: https://claude.ai/code/session_01MDHn7fC3hsGJ27vFHD2o1p
…ressions and native SQL reading Db2 is supported officially, but GORM for Hibernate 7 classified it from a guess (#16746). From IBM's documentation and the DDL Hibernate 7.4 emits for each Db2 dialect: - UniqueIndexNulls: Db2 for Linux, UNIX and Windows (DB2Dialect) gets a unique index with `exclude null keys`; a unique index treats a null as any other value and that option leaves out only an all-null key, so a partly null key is compared, as on Oracle: COMPARED. Db2 for IBM i and z/OS (DB2iDialect, DB2zDialect, which extend DB2Dialect and are listed first) get `create unique where not null index`, under which nulls are not equal: DISTINCT. A shared join table on Db2 LUW therefore gets the one unique key over the columns of every association, as on Oracle; a key per association would reject the rows of the other association. - RegexDialectPattern: Db2 LUW (11.1+) and IBM i match with REGEXP_LIKE; before, Db2 fell back to LIKE, so rlike matched a regular expression as a LIKE pattern. Db2 for z/OS runs REGEXP_LIKE only through the Db2 Analytics Accelerator and keeps the LIKE fallback of other databases. - SqlSyntax: block comments nest on Db2 for IBM i and z/OS. Tests: the dialect specs gain the three Db2 dialects; the shared join table binder spec the DDL each renders for both shapes. The container specs SharedKeyColumnHibernate7Spec, SharedJoinTableHibernate7Spec and RLikeHibernate7Spec gain a Db2 LUW row (Testcontainers Db2Container, icr.io/db2_community/db2:11.5.9.0); IBM publishes that image for amd64 only, so the row runs on the amd64 CI runners and is skipped on arm64. com.ibm.db2:jcc and testcontainers-db2 are test dependencies, as ojdbc11 is for Oracle. Docs: the rlike databases in finders and createCriteria, and Db2 in the shared join table upgrade note. Claude-Session: https://claude.ai/code/session_01MDHn7fC3hsGJ27vFHD2o1p
…er specs The six specs that run a feature against each database (SharedKeyColumn, SharedJoinTable, SqlProjection, RLike, SqlRestriction and NativeIdGenerator Hibernate7Spec) each created PostgreSQL, MySQL, MariaDB and Oracle (and now Db2) containers of their own, as untyped @shared fields the Testcontainers Spock extension does not manage, started lazily in a where row and never stopped. They piled up until the test JVM exited: after three specs three Oracle containers of about 2.5 GB each held the memory Docker had, and every later Oracle row timed out after three minutes waiting for the database (locally, three Oracle rows failed in every full run). Started on its own the same image is ready in about 20 seconds. DatabaseContainers holds one container per database; the specs point their fields at it and keep starting it lazily, so the first row that needs a database starts it and every later spec finds it running. Each spec creates its tables with create-drop, so it starts from an empty schema. The six specs now pass on every database locally, with one container of each running at the peak, in about 2.7 minutes instead of over 13; on CI the Db2 container starts once for its three rows. The specs with typed container fields, which the extension starts and stops per spec, are left as they are. Claude-Session: https://claude.ai/code/session_01MDHn7fC3hsGJ27vFHD2o1p
|
Two more commits, both local fixes found after the last review:
The Hibernate bug itself is drafted for the Hibernate Jira. |
|
Stacked on the upstream fix for #16691 (PR #16757):
|
Hibernate 7: support Db2 in unique key null handling, regular expressions and native SQL reading
…ble-key-8.0.x Stack on the fix for issue #16691, resolving the join table row persister behavior for unindexed collections and enabling single-element removal in SharedJoinTableBulkDeleteSpec.
….x' into fix/h7-shared-join-table-key-8.0.x
|
Stacked on the upstream fix for #16691 (PR #16757):
|
1 similar comment
|
Stacked on the upstream fix for #16691 (PR #16757):
|
|
Stacked on the upstream fix for #16691 (PR #16757):
|
🔎 No tests executed 🔎🏷️ Commit: e635103 Learn more about TestLens at testlens.app/docs. |
jdaugherty
left a comment
There was a problem hiding this comment.
The regression from my last review is fixed. Only a set gets the shared-table unique key, and a list, a map and a bag keep their repeated elements. SharedJoinTableUniqueKeyBinderSpec checks their tables from the metadata, and MultipleUnidirectionalHasManySpec and SharedJoinTableHibernate7Spec save them and read them back. The six properties from my last round now save on this head, and GrailsSecondPass is deprecated instead of removed. I ran the specs this PR touches at e635103, together with JoinTableCollectionPersisterSpec from #16757: all 132 features pass on H2, PostgreSQL, MySQL, MariaDB and Oracle. The Db2 rows are skipped on arm64. The full :grails-data-hibernate7-core:test suite also passes locally on this head, with no failures.
Approving, with two non-blocking notes inline:
- The
<element> is not nullrestriction also applies to the default shared table, where each association has its own key column. It separates nothing there, but it adds a predicate to every load and clear of that table and turns itsis emptyinto a count. Restricting only the collections that share a key column would avoid that. - In the upgrade note, the sentence about lists and maps changes what "that" refers to in the next sentence, and a bag belongs in it too. A sentence on the new
hibernate.query.sqm.translatordefault would match the one onhibernate.persister.resolverin the next paragraph.
| * collections linear. The collection of a shared table that is bound first finds none, and the one bound after | ||
| * it restricts both. | ||
| */ | ||
| private static boolean hasColumnsOfOthers(Collection collection, Table table) { |
There was a problem hiding this comment.
Non-blocking. hasColumnsOfOthers is also true for the default shape, hasMany = [allow: Member, deny: Member] with no mapping. There each association has its own key column and both share the element column, so both collections get the same member_id is not null. The binder spec pins this for SjtOwner (roles.where == 'sjt_role_id is not null', lines 104-105). The predicate separates nothing in that shape, because each association's rows are already selected and deleted by its own key column. Captured with a statement inspector on H2 at e635103:
load: ... where a1_0.p16701_group_allow_id=? and (a1_0.p16701_member_id is not null)
clear: delete from p16701_group_p16701_member where p16701_group_allow_id=? and ((p16701_member_id is not null))
is empty: ... where (select count(*) from p16701_group_p16701_member d1_0 where (d1_0.p16701_member_id is not null) and pg1_0.id=d1_0.p16701_group_deny_id)=0
An owner with a table of its own keeps not exists(select 1 ...) for is empty. Because the default shape now hasWhereRestrictions(), RestrictedCollectionSqmTranslatorFactory also turns its emptiness test into a count. The restriction is needed only when the sharing collections have a key column in common, which is the joinTable shape of SjtbdAccount in SharedJoinTableBulkDeleteSpec. Restricting only that case would leave the SQL of the most common shared shape as it was, and would keep the translator override to the one case that needs it.
| Under Hibernate 7, an owner with two or more unidirectional `hasMany` associations wrote no rows to their join tables: an element added to any of them was lost at flush (issue https://github.com/apache/grails-core/issues/16665[#16665]). This is fixed when each association has a join table of its own, which is the default for associations to different classes (`item_roles` and `item_tags` for `static hasMany = [roles: Role, tags: Tag]`) and also the case when two associations to the same class map different tables with `joinTable`. Each such join table is written, as under Hibernate 5. | ||
|
|
||
| Two unidirectional associations to the same class share one join table by default, named after the owner and the element class, with a key column per association (`item_role` with `item_roles_id` and `item_deny_roles_id` for `static hasMany = [roles: Role, denyRoles: Role]`). Such a table is read, but new elements are not written, and neither are those of associations that `joinTable` maps onto one table. That is issue https://github.com/apache/grails-core/issues/10811[#10811] and is not fixed by this change; name a `joinTable` per association to give each its own table. | ||
| Two unidirectional associations to the same class share one join table by default, named after the owner and the element class, with a key column per association (`item_role` with `item_roles_id` and `item_deny_roles_id` for `static hasMany = [roles: Role, denyRoles: Role]`). Such a shared table is now written as well: each association keeps its rows unique by its own key column and the element. A list or a map is left as it was: its join table is keyed by the index, and its elements may repeat. On most databases that is a unique key per association; on Oracle and on Db2 for Linux, UNIX and Windows, which compare the non-null columns of a partially null row in a unique key, it is one unique key over the key columns of both associations and the element. A schema created by an earlier Grails 8 release has the single unique key Hibernate made for the first association: `dbCreate` of `update` adds the second one, and on Oracle and Db2 for Linux, UNIX and Windows a migration has to replace that key with the wider one. |
There was a problem hiding this comment.
Two small things in this paragraph. The list-and-map sentence sits between "each association keeps its rows unique by its own key column and the element" and "On most databases that is a unique key per association", so "that" now reads as the list's key. Moving it after the Oracle and Db2 sentence fixes the reading, and a bag belongs in it too, since it is also left without the key.
Also, GORM now sets hibernate.query.sqm.translator unless the application sets it. The next paragraph says that an application with its own hibernate.persister.resolver keeps it and loses the #16691 fix. A matching sentence for the translator would say the same about is empty on associations that share a key column (#16748).
Fixes #16699
Fixes #16748
What
An owner with two unidirectional
hasManyto the same class and nojoinTablemapping shares one join table between them, with a key column each.CollectionKeyColumnUpdatermade the key of each collection not updatable in that case, the rule from #10811. Hibernate 7's collection persister writes no rows for a collection whose key is not updatable, so everyaddTo*on such an owner was dropped silently, even withfailOnError: true(the second shape of #16665, and @jdaugherty's probe on #16686).CollectionKeyColumnUpdaternow keeps the key nullable and always updatable; the sibling counting is gone. The Hibernate 5 binder made every collection key non-updatable, which Hibernate 5 tolerated; on Hibernate 7 the flag only loses data.Set.createPrimaryKeycreates none when the table has one already), so the rows of the second collection had no uniqueness at all.SharedJoinTableUniqueKeyBinder, built once inCollectionBinderand run byCollectionSecondPassBinderright after the keys of each collection are created, completes what is missing in the shape the database needs, throughSharedJoinTableUniqueness, which followsUniqueIndexNulls: one fact per database, how its unique index treats a null in its key, resolved throughDialectKeyedas the regular expression operator is (Validate a unique group with a null value against the rows whose group value is null #16583's default reading of a null in a unique constraint is the same fact):Only a set gets this. A list is keyed by (owner, index) and a map by (owner, key), and their elements repeat by design, so they are left as Hibernate binds them, as are a unique key the mapping declares explicitly, a collection with a join table of its own, and a bidirectional one-to-many.
SharedJoinTableHibernate7Specsaves a list['x', 'x']and a map with one value under two keys on H2, PostgreSQL, MySQL, MariaDB and Oracle.Tests
SharedJoinTableHibernate7Spec(new): the scenario on H2, PostgreSQL, MySQL, MariaDB and Oracle through Testcontainers, asRLikeHibernate7Specdoes. The same member twice in one association of one group, from two sessions on aversion falseowner, fails withDataIntegrityViolationException; the same member in the other association, in another group, and denied by two groups is accepted. The last case is the one a key per association breaks on Oracle, the first the one a single key over every key column breaks where nulls are distinct.SharedJoinTableUniqueKeyBinderSpec: the unique keys of the shared table and of two tables of their own from the boot metadata;bindon the bidirectional collection adds nothing; every dialect resolves to its uniqueness; the uniqueness applied to a hand-built shared table and rendered with each dialect's own exporters, SQL Server's filtered index and DB2'sexclude null keysindex included, so the two databases without a container are asserted on their DDL.CollectionSecondPassBinderSpec:createCollectionKeyscreates the keys and then runs the binder.CollectionKeyColumnUpdaterSpec: the shared-table case expects an updatable key.MultipleUnidirectionalHasManySpec: a role added to each of two associations sharing a table is read back through its own association; one role denied by two groups is written once per group; rows put in a shared table survive a flush that adds to one association.Verified:
:grails-data-hibernate7-core:test3,858 tests, with the five-database spec green against live containers;checkstyleMain,codenarcMain,codenarcTestclean.Docs
The Grails 8 upgrade notes (26.11) and the one-to-many chapter of the GORM for Hibernate 7 guide say that a shared join table is now written and which key each database gets, and that a schema from an earlier Grails 8 release needs the second key added, or on Oracle the first key replaced.
Out of scope
Two unidirectional hasMany Lists (or Maps) sharing a join table are a separate, pre-existing defect: the second list's primary key overwrites the first's and both index columns are NOT NULL. Filed as #16700.
On the 9.x branches the native generator will need to state the same uniqueness on a shared
@JoinTable; the differential against the classic oracle flags it at merge-up.