Repository navigation
Validate a unique group with a null value against the rows whose group value is null - #16583
borinquenkid wants to merge 24 commits into
Conversation
Covers the Requestmap, Language and Folder shapes of #14503 in the TCK (run by the Hibernate 7 module) and in the UniqueConstraintSpec of the in-memory datastore, including a subclass, several nullable group members and a change of a saved row's group value. They fail on 9.0.x. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The unique constraint left a group property out of its query whenever the
value was null, so the constraint checked the constrained property alone:
('en', null) conflicted with ('en', 'US'), and a top level folder conflicted
with a folder of the same name under any parent. Every datastore was affected.
A null group value now adds an isNull condition, so it conflicts only with a
row whose value of that property is also null. Non null values, the to-one
handling and the dirty check are unchanged.
Fixes #14503
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## 9.0.x #16583 +/- ##
==================================================
+ Coverage 65.3123% 65.3640% +0.0516%
- Complexity 32320 32419 +99
==================================================
Files 2430 2443 +13
Lines 120429 120649 +220
Branches 22290 22332 +42
==================================================
+ Hits 78655 78861 +206
Misses 31781 31781
- Partials 9993 10007 +14
🚀 New features to boost your workflow:
|
Neo4j compiles isNull on an association to a node property test, but an association is a relationship there, so the condition is always true and a top level folder conflicts with a folder of the same name under a parent. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
# Conflicts: # grails-doc/src/en/guide/toc.yml # grails-doc/src/en/guide/upgrading/upgrading90x.adoc
…t-null-group-9.0.x # Conflicts: # grails-doc/src/en/guide/toc.yml # grails-doc/src/en/guide/upgrading/upgrading90x.adoc
…pache/grails-core into fix/unique-constraint-null-group-9.0.x # Conflicts: # grails-doc/src/en/guide/upgrading/upgrading90x.adoc
…t-null-group-9.0.x
jamesfredley
left a comment
There was a problem hiding this comment.
Approving. A unique group with a null value is now compared with isNull, so two rows that share a null group member are still rejected, and a row with a non-null group value is not treated as a duplicate of a null. Non-null groups, updates, and inheritance are unchanged. The Neo4j association case is still pending and is a pre-existing limitation, not a regression from this change.
…t-null-group-9.0.x # Conflicts: # grails-doc/src/en/guide/upgrading/upgrading90x.adoc
jdaugherty
left a comment
There was a problem hiding this comment.
Requesting changes, mainly on the shape of the fix rather than the code.
This decides the open question of #14503 on its own. In my comment there I laid out the two ways to fix this: skip the lookup when a group value is null (the SQL unique predicate, and how PostgreSQL, MySQL, MariaDB and H2 treat the composite index Hibernate generates from this constraint), or compare null as a value with isNull (what this PR does, and how Oracle and SQL Server treat that index). We said we would decide as a team before a fix went in, and we never did. Since then the thread has asked for both: @jagiro (comment) wants the isNull reading by default with skip-on-null as an opt-in, and @homicki (proposal), on the day this PR opened, proposed the opposite default with unique: [group: ['locale'], nulls: 'match'] as the opt-in, with the Folder and Translation shapes showing that neither reading fits every domain. That proposal also covers #14656, the mirror of this issue.
What I would like instead: support both and make it a toggle. Today's behaviour has to stay reachable as well. A null group value acting as a wildcard is a legitimate reading that applications may prefer: a Requestmap with a null httpMethod applies to every method, so overlapping with ('/x', 'POST') is exactly what some of them want rejected, and the replacement the upgrade note offers, name unique: true, is not equivalent. So three modes, selectable per constraint and ideally with a GORM-wide default setting so an application picks its rule once:
- wildcard: a null group value leaves the property out of the lookup (today)
- skip: a null group value skips the lookup (SQL semantics)
- value: a null group value matches only null (this PR)
Which mode is the default is the decision we owe on the issue. Let's settle it there before this merges, not in the upgrade note. Details inline.
Also:
- The constrained property itself still follows the skip rule, so this change gives one composite key two different null rules (inline on the
else). - The current head (40c6b5f) has no CI run. The previous head was green apart from the canary's mongo container failure. The branch is about 700 commits behind 9.0.x and the upgrade note's section number collides with what landed there since (inline).
- The description says a new
upgrading90xpage with atoc.ymlentry, and that the TCK spec runs "here" on Hibernate 7. The diff appends to the existing page, and the spec runs on every TCK consumer: the Hibernate 5, Hibernate 7, MongoDB and Neo4j cores. Worth correcting, since the description becomes the squash message. - Keep "Fixes #14503" only if the toggle lands in this PR; otherwise make it "Relates to" so the discussion stays open, and link #14656 either way.
| `@JsonPropertyOrder` or sorted alphabetically, and `@JsonView`, `@JsonFilter`, `@JsonAnyGetter`, `@JsonIgnoreType`, | ||
| `@JsonFormat` and `@JsonSerialize` on a property, and mix-ins, other than the property names they give, do not apply. | ||
|
|
||
| ==== 5. A Null Value in the Group of a Unique Constraint Is Compared as a Value |
There was a problem hiding this comment.
9.0.x already has sections 1 to 16 here (XML web support, Spring JSON conversion and the rest landed after this branch was cut), so this becomes a second 5. when merged. Renumber after merging 9.0.x in; as of today it is 17.
There was a problem hiding this comment.
Renumbered when 9.0.x was merged: the section is 17 at this head. The section is rewritten for the toggle, so the number is the only thing that stays.
There was a problem hiding this comment.
9.0.x has 19 sections now (18 RxGORM on RxJava 3, 19 RxJava 1 and 2 deprecated), so this is 20 after the next merge; the file conflicts until then.
| Applications that relied on the old behaviour are affected: a save that was rejected because a `null` group value matched every row of the constrained property now succeeds when the rows differ in that group property. | ||
| The old behaviour was a defect and has no replacement. | ||
| To keep a rule such as "the name is unique whatever the value of `parent`", declare the constraint on the one property, `name unique: true`. | ||
| If the database must also reject the second `('en', null)` row, remember that most databases do not treat `null` values of a unique index as equal; the validation does, the index does not. |
There was a problem hiding this comment.
Hibernate generates the composite unique index from this same constraint, so under the value reading the validator disagrees with the index it produced on PostgreSQL, MySQL, MariaDB and H2, and agrees with it on Oracle and SQL Server. The skip reading is the reverse. That is the main reason both modes need to exist. Rather than "most databases", say which databases match which mode so a reader can pick the one that agrees with their index.
There was a problem hiding this comment.
Replaced "most databases" with the table, and made it the default's rule. I checked it rather than took it from memory: on H2, PostgreSQL, MySQL, MariaDB and Oracle containers, UniqueNullsHibernate7Spec inserts ('en', null) twice and compares the database's answer with the validation's. Oracle rejects it (it compares the non-null columns of a partly null key), the other four accept it. One correction to the list in the review: SQL Server and DB2 read skip, not value, because Hibernate creates the index over nullable columns as a filtered unique index (where col is not null) and DB2's as exclude null keys, so rows with a null are left out. That is from the DDL Hibernate renders; I have no SQL Server or DB2 container. So the default is skip everywhere except Oracle, where it is value.
There was a problem hiding this comment.
Verified for Hibernate 7 against hibernate-core 7.4: SQLServerDialect and DB2Dialect both use AlterTableUniqueIndexDelegate, so skip is right there and my list was wrong for 7. On Hibernate 5.6 it is the other way round: there is no filtered-index delegate, SQLServerDialect gets DefaultUniqueDelegate and a plain unique constraint, so SQL Server (and Sybase) read value while this table says skip. Inline on the Hibernate 5 UniqueNullsPerDialect.
There was a problem hiding this comment.
The reference table now has the per-database list. The only piece left is the upgrade note's Db2 bullet (new comment).
|
|
||
| In this example `username` must be unique in `group` and `department`. There might be same logins in different groups or different departments. | ||
|
|
||
| A `null` value of a property in the group is compared as a value: it conflicts only with a row whose value of that property is also `null`. With `group unique: 'department'`, a group without a `department` may have the same name as a group of a department, but two groups with that name and no `department` conflict. This is the same for a group that has several properties and for an association in the group, such as a `parent` that is `null` for the top level nodes of a tree. |
There was a problem hiding this comment.
With the toggle this paragraph should describe the modes and the default, and say how a null value of the constrained property itself is treated (skipped today), so the whole rule for a composite key is in one place.
There was a problem hiding this comment.
Done in 70a0b02: unique.adoc describes the four readings in a table, the reading each database gets under database, the map form, the setting, and says the constrained property follows the same reading. The GORM configuration page lists the setting.
There was a problem hiding this comment.
Verified, with two corrections inline: the worked example's rows, and the SQL Server/DB2 row holding for Hibernate 7 only.
There was a problem hiding this comment.
Verified, including the corrections from round 2.
…t-null-group-9.0.x
…unique index does, selectable per constraint The unique constraint now has four readings of a null in the constrained property or in its group: wildcard (a null group value matches every row, the behaviour before this change), skip (a row with a null is never a duplicate), value (a null matches only another null) and database, the default, which follows the unique index of the database so that the validation and the index agree. A constraint selects its reading with unique: [group: ..., nulls: ...]; the grails.gorm.unique.nulls setting selects it for every constraint without one. The constrained property itself follows the same reading. The database reading comes from the datastore: Hibernate 7 and 5 key it on the dialect like the other per-database enums (Oracle compares the non-null columns of a partly null key, so it reads value; H2, PostgreSQL, MySQL, MariaDB, SQL Server and DB2 leave such rows out of the index, so they read skip), and a datastore without such an index reads value. The mapping builders and Property read the map form of the constraint as its group, so it does not create a single-column unique index. Tests: the in-memory datastore covers every reading, an association in the group, the setting and the rejection of bad options; the TCK covers each reading and the database-following one on every datastore; UniqueNullsHibernate7Spec checks on H2, PostgreSQL, MySQL, MariaDB and Oracle that the validation agrees with whether the database accepts the same row twice. Relates to #14503, #14656
…erve collection uniqueness groups Query the H2 compatibility mode once at datastore startup via information_schema.settings, falling back to the JDBC URL, and cache the resolved UniqueNullsPerDialect to avoid repetitive runtime queries during validation. Support Collection, Iterable, and Array instances for composite uniqueness groups in Property and HibernateMappingBuilder, and document H2 compatibility mode semantics.
f7cda45 to
330b688
Compare
|
Updated for Round 5 review comments:
|
|
@jdaugherty @jamesfredley fixes for the round 5 points are pushed (head 330b688, 9.0.x merged in): H2 compatibility mode is read once at datastore start, a |
jdaugherty
left a comment
There was a problem hiding this comment.
Round 6 at e8fdab6, which is 330b688 with 9.0.x merged in: the merge is clean and the note stays section 21. The round-5 asks are done. The H2 mode is read from information_schema.settings once, when the datastore is built, and kept; a Set group gives a composite key; the section is 21. Here the covering specs pass on Hibernate 5 and 7, MongoDB and Neo4j, UniqueNullsHibernate7Spec with its containers.
On Hibernate 7, H2 in Oracle, SQL Server and Derby mode agrees with its index for a partly null and for an entirely null key, also when the mode is set outside the URL (;INIT=SET MODE Oracle), and a secondary datasource in another mode gets its own reading.
What is left:
- H2 in
MODE=DB2readsvalueon Hibernate 7 while its index treatsnullvalues as distinct, so the validation rejects a row the database accepts. Comment onUniqueNullsPerDialect. DatabasePlatformis new public API in the core module, and it states something other than whatdatabasereads. Comment there.- On Hibernate 5 the H2 mode handling cannot take effect: in those modes the lookup fails on
limit ?. Comment onAbstractHibernateDatastore. - Docs: a link without a target and the H2 sentence in the upgrade note, and the list of exceptions for an entirely null key.
The first two need settling before merge, the second because it cannot be taken back once released.
| String modeSpec = (h2Mode != null && !h2Mode.trim().isEmpty()) ? h2Mode : jdbcUrl; | ||
| if (modeSpec != null) { | ||
| DatabasePlatform platform = DatabasePlatform.resolveForH2(modeSpec); | ||
| if (platform == DatabasePlatform.ORACLE || platform == DatabasePlatform.DB2_LUW) { |
There was a problem hiding this comment.
H2 in MODE=DB2 keeps null values distinct in a unique index, as REGULAR mode does: on H2 2.5.252 a second ('en', null), (null, 'US') and (null, null) are all accepted under unique (a, b). This line reads that mode as Oracle, so a datastore on jdbc:h2:mem:...;MODE=DB2 answers value, the validation rejects a second ('en', null) and the database accepts it. The Hibernate 5 table has no DB2_LUW in this condition and reads skip for the same database.
A mode does not bring the index of the platform it is named after. Three of H2's modes change how a unique index treats null (Mode.nullsDistinct): Oracle compares a partly null key and exempts an entirely null one, MSSQLServer and Derby compare every key. Every other mode (DB2, PostgreSQL, MySQL, MariaDB, HSQLDB, STRICT, LEGACY) is skip. Please take DB2_LUW out of this condition and add a DB2 row to the mode matrix of UniqueNullsHibernate7Spec; its validationRejects == indexRejects fails for that row today.
There was a problem hiding this comment.
Fixed in b273f8e: MODE=DB2 is read as skip now. Only Oracle, MSSQLServer and Derby change how H2's unique index treats a null, and every other mode keeps nulls distinct, so the mode lookup is a private nested enum of those three and everything else falls through to the dialect. UniqueNullsHibernate7Spec has a DB2 row where validationRejects == indexRejects, and SupportedDatabaseSpec pins the reading.
| * | ||
| * @since 9.0 | ||
| */ | ||
| public enum DatabasePlatform { |
There was a problem hiding this comment.
This enum becomes public API of grails-datastore-core, and what it states is not what database reads. SQL_SERVER is value here and skip under Hibernate 7; DB2_LUW exempts an entirely null key here and does not under Hibernate 5. The reading follows the index Hibernate renders, which is why the table is kept per Hibernate module. The only caller is the H2 mode lookup, which needs three outcomes. It compares the constant and answers from its own table, so uniqueNulls and entirelyNullKeyExempt here are never read; POSTGRESQL, MYSQL, MARIADB, DB2_IBM_I, DB2_ZOS, H2 and DEFAULT are never compared; and H2Mode.SYBASE names a mode H2 does not have (Unknown mode "Sybase" on connect).
Please drop DatabasePlatform and map the mode to the reading directly: Oracle is value with the exemption, MSSQLServer and Derby are value without it, anything else falls through to the dialect. That fits in UniqueNullsPerDialect.forDialect of the Hibernate module, next to the table it belongs to, and leaves nothing H2-specific in the core module.
There was a problem hiding this comment.
Fixed in b273f8e: DatabasePlatform and H2Mode are removed from core, with H2ModeSpec. Core keeps only what every datastore can answer (UniqueNulls, the setting, Datastore.getUniqueNulls()) plus a database-free ClassKeyed lookup in an internal package, with Javadoc that says it is internal and carries no compatibility promise. Each SQL module declares its databases once in SupportedDatabase, keyed by its own dialect classes, and the H2 mode mapping lives there. H2Mode.SYBASE is gone with it.
| return getResolvedUniqueNulls().isEntirelyNullKeyExempt(); | ||
| } | ||
|
|
||
| protected UniqueNullsPerDialect getResolvedUniqueNulls() { |
There was a problem hiding this comment.
On Hibernate 5 this cannot take effect for H2. With H2 2.4 in MODE=Oracle, MSSQLServer or Derby, the constraint's lookup fails before it compares anything:
Syntax error in SQL statement "select this_.id as y0_ from ... this_ where this_.language=? and this_.country is null [*]limit ?"
Hibernate 5.6's H2Dialect limits with limit ?, which H2 2.x accepts only in the modes that have it. That is not from this change, any query with a row limit fails the same way there. It does leave queryH2Mode(), getJdbcUrl() and the H2 branch of the Hibernate 5 UniqueNullsPerDialect without a case in which the validation can agree with the index, and without a test at the level of the datastore: the module's only test of them is the enum's. I would take the H2 mode handling out of the Hibernate 5 module and make the two H2 rows of the table in unique.adoc Hibernate 7 rows.
There was a problem hiding this comment.
Fixed in b273f8e: Hibernate 5 no longer reads the H2 mode. queryH2Mode, isH2Dialect and getJdbcUrl are gone from AbstractHibernateDatastore, H2 is read as its regular mode there, and the two H2 rows of unique.adoc are Hibernate 7 rows.
| given: | ||
| manager.destroy() | ||
| manager.grailsConfig = [ | ||
| 'dataSource.url' : "jdbc:h2:mem:uniqueNullsH2Mode${h2Mode};MODE=${h2Mode};LOCK_TIMEOUT=10000".toString(), |
There was a problem hiding this comment.
Every row carries the mode in the URL, so the matrix passes with queryH2Mode() returning null: the URL fallback answers. A row whose mode the URL pattern cannot see pins the query, for example ;INIT=SET MODE Oracle, which reads value and agrees with the index here.
There was a problem hiding this comment.
Fixed in b273f8e: the matrix has INIT=SET MODE Oracle and INIT=SET MODE MSSQLServer rows next to the URL ones, so only the query of the database settings can find them.
|
|
||
| static mapping = { | ||
| version false | ||
| language unique: (['country'] as Set) |
There was a problem hiding this comment.
Non-blocking: the reported case was the Set in the constraint alone, and with the group repeated here the feature also passes through the builder's list form. Without this line the key is still composite here, so dropping it makes the feature pin the constraint path; the mapping form has its own feature in HibernateMappingBuilderSpec.
There was a problem hiding this comment.
Done in 62c9a95: the mapping block no longer repeats the group, so the feature pins the constraint path alone and the key is composite from the constraint.
| // Db2 for Linux, UNIX and Windows: `exclude null keys` leaves out only a key that is null in every column | ||
| DB2(UniqueNulls.VALUE, true, DB2Dialect.class), | ||
|
|
||
| DEFAULT(UniqueNulls.SKIP, false, Dialect.class); |
There was a problem hiding this comment.
Verified at e8fdab6 on Hibernate 7: the mode is read from information_schema.settings once, when the datastore is built, and kept. In Oracle, SQL Server and Derby mode the validation agrees with the index for ('en', null) and for (null, null), also with the mode set outside the URL (;INIT=SET MODE Oracle), and a secondary datasource in Oracle mode gets its own reading. MODE=DB2 is read as Oracle and should not be, and on Hibernate 5 the lookup cannot run in these modes: new comments on UniqueNullsPerDialect and AbstractHibernateDatastore.
| */ | ||
| void setUnique(Map options) { | ||
| Object group = options.get('group') | ||
| if (group instanceof List) { |
There was a problem hiding this comment.
Verified at e8fdab6: with the Set group declared only in the constraint, the key is composite on Hibernate 5 and 7. ('en', 'GB') saves next to ('en', 'US') and the index rejects a second ('en', 'US').
| `grails-async-rxjava3` carries a `null` value as the RxJava 1 factory and every other promise factory do: `createPromise()` and `createPromise(Class)` give a promise whose `get()` returns `null`, a task that returns `null` resolves to `null`, and a `null` value keeps its place in the list that `onComplete`, `onError` and `waitAll` deliver. | ||
| `grails-async-rxjava2` does not: there `createPromise()` and `onComplete` over a list holding a bound promise of `null` fail with a `NullPointerException`, and a task that returns `null` resolves to the `java.lang.Void` class. | ||
|
|
||
| ==== 20. A Null in a Unique Constraint's Composite Key Follows the Database |
There was a problem hiding this comment.
Verified: section 21, and 9.0.x merges without a conflict.
| |A row with a `null` in some of the key columns is indexed, and its other columns are compared, so `('en', null)` twice is a violation. | ||
| |=== | ||
|
|
||
| A datastore that keeps no such index, such as MongoDB, reads `database` as `value`. |
There was a problem hiding this comment.
Verified: the H2 rows are in the table and the em dashes are gone. The rows say Hibernate 5 and 7; see the comment on AbstractHibernateDatastore.
| In Grails 9 the constraint reads a `null` in its key, in the group and in the constrained property, as the unique index of the database does, so that the validation and the index agree: | ||
|
|
||
| * `skip` on H2, PostgreSQL, MySQL, MariaDB, on SQL Server with Hibernate 7, and on Db2 for IBM i and z/OS with Hibernate 7, whose index leaves a row with a `null` out: `('en', null)` is never a duplicate. | ||
| * `value` on Oracle, on Db2 for Linux, UNIX and Windows with Hibernate 7 (`exclude null keys` indexes partly null keys), on Db2, SQL Server and Sybase with Hibernate 5 (a plain unique constraint compares `null` values), and on a datastore with no such index such as MongoDB: a partly null key `('en', null)` conflicts only with another `('en', null)`. On Oracle and on Db2 for Linux, UNIX and Windows with Hibernate 7, a key that is `null` in every column is omitted by the index, so `(null, null)` and an optional single property with value `null` are accepted. See the per-database table in the <<unique,unique>> constraint reference. |
There was a problem hiding this comment.
Verified: one list now. The H2 sentence added with it needs another pass, new comment on that line.
…tabases Review of #16583: DatabasePlatform was public API in grails-datastore-core, stated a reading other than the one `database` reads, and named an H2 mode H2 does not have. Behind it, the databases were listed again in four dialect-keyed enums per Hibernate module and in a copy of isH2/queryH2Mode in each datastore. Core keeps only what every datastore can answer: UniqueNulls and the setting. A generic, database-free ClassKeyed lookup moves in as internal API. Each SQL module declares its databases once in SupportedDatabase, keyed by its own Hibernate dialect classes, since the two Hibernate versions render different SQL for some databases: - Hibernate 7: SupportedDatabase carries the SQL syntax, the regular expression pattern and how the unique index treats a null. RegexDialectPattern, DialectKeyed and UniqueNullsPerDialect are removed; SqlSyntax and UniqueIndexNulls read from it; the datastore asks it for the reading. H2 compatibility modes are a private nested enum: Oracle, MSSQLServer and Derby change the index, every other mode (DB2 included) keeps nulls distinct, so MODE=DB2 is no longer read as Oracle and Sybase is no longer named. - Hibernate 5: its own SupportedDatabase for the regular expression and the unique reading; RlikeExpression reads it. H2 modes are not read on Hibernate 5, where a limited query fails in the modes that change the index. Behavior: rlike on PostgreSQL with Hibernate 7 is now ~* as on Hibernate 5, so the match ignores case as it did on 5 (confirmed against PostgreSQL, MySQL, MariaDB and Oracle containers). Hibernate 5 rlike on a database with no regex operator now falls back to like, as Hibernate 7 does, instead of throwing; the reference docs state both. Oracle9Dialect and OracleDialect now work with rlike on Hibernate 5. Docs: the upgrade note and unique.adoc state the H2 mode rules, the entirely-null-key exceptions and the link target. Claude-Session: https://claude.ai/code/session_01Dmki4SQGwRSj8pRJ5KCWyX
…lone The mapping block repeated the group, so the feature also went through the builder's list form; HibernateMappingBuilderSpec covers that form. The key is composite from the constraint alone. Claude-Session: https://claude.ai/code/session_01Dmki4SQGwRSj8pRJ5KCWyX
|
Thanks for the class placement. I wanted cohesiveness and the first pass had leakage; this one is much better at consolidating db variance. What changed since round 6 (b273f8e, 62c9a95):
Not run here: PITest (its opt-in is in the #16819 stack) and the Db2 container (this machine is arm). |
jamesfredley
left a comment
There was a problem hiding this comment.
Not approved. rlike rewrites a column expression that already contains ?2. A unique group that contains a null is compared the way that database's index compares it, and a null Neo4j association tests the relationship.
|
|
||
| private boolean isOracleDialect(Dialect dialect) { | ||
| return (dialect instanceof Oracle8iDialect); | ||
| return SupportedDatabase.forDialect(dialect).regexPattern().replace("?1", columns[0]).replace("?2", "?"); |
There was a problem hiding this comment.
The second replace runs after the column expression is inserted. A mapped formula or quoted column that contains ?2, such as code?2, becomes code?. The query then uses the wrong column. Replace ?2 before inserting the column.
…it is RlikeExpression inserted the column and then replaced ?2, so a formula or a quoted column whose text contains ?2 had that text replaced too. The value placeholder is replaced first and the column is inserted last. Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
|
@jamesfredley Fixed in c93e38f, on top of 62c9a95.
|
jamesfredley
left a comment
There was a problem hiding this comment.
rlike no longer rewrites a column expression that already contains ?2, and a unique group that contains a null is compared the way that database compares null. Not approving. The null policy is read from the default connection, so a uniqueness check on another connection can accept a duplicate that connection rejects.
| throw new IllegalStateException("Cannot validate object [$target]. It is not a persistent entity") | ||
| } | ||
|
|
||
| Datastore datastore = GormRegistry.findDatastore(constraintClass) |
There was a problem hiding this comment.
Input: a uniqueness check inside a named connection whose null policy differs from the default connection. The policy is read from the default connection, so a duplicate that the named connection rejects is accepted. Resolve the policy through the same connection the criteria use.
There was a problem hiding this comment.
@jamesfredley I could not reproduce this as a null-policy bug, so the policy lookup is unchanged. The spec is in 68e04e7 (UniqueNullsConnectionHibernate7Spec): two connections, the default in H2's regular mode (reads skip) and the second in MODE=Oracle (reads value). An entity mapped to the second is validated with value and rejects the duplicate; one on the default connection keeps skip.
The case you describe, an entity mapped to every connection and validated inside second.withNewSession, does read the default connection's policy. It also runs its query on the default connection: UniqueConstraint's criteria always uses ConnectionSource.DEFAULT. When I made the policy follow the block, the query failed with No Session found for current thread, which is how I found it. The policy and the query agree today; both ignore the block.
Making the check follow the connection block is a separate routing fix, tracked in #16934 (it is the same gap that keeps UniqueWithMultipleDataSourcesSpec ignored on Hibernate 5). I kept it out of this pull request, which only changes how a null in a unique key is read.
…the entity's connection indexes them Two connections on different readings, the default in H2's regular mode (skip) and the second in MODE=Oracle (value): an entity mapped to the second is validated with value, one on the default with skip.
# Conflicts: # grails-doc/src/en/guide/upgrading/upgrading90x.adoc
✅ All tests passed ✅Test SummaryTip Inspect all details with the TestLens CLI: CI / Build Grails-Core (Windows JDK 25 shard 2) > :grails-startup-progress:test
🏷️ Commit: d2509f4 Test FailuresStartupProgressSpec > tells a watching page why the start failed while #stage, then releases the port > tells a watching page why the start failed while running a hook, then releases the port (:grails-startup-progress:test in CI / Build Grails-Core (Windows JDK 25 shard 2) | Attempt 1/2)Learn more about TestLens at testlens.app/docs. |
jamesfredley
left a comment
There was a problem hiding this comment.
Not approving. A uniqueness check on a named connection still uses the default connection's null policy, and a supplied session factory never learns the database's null policy.
| throw new IllegalStateException("Cannot validate object [$target]. It is not a persistent entity") | ||
| } | ||
|
|
||
| Datastore datastore = GormRegistry.findDatastore(constraintClass) |
There was a problem hiding this comment.
An entity mapped to two connections, with language unique on country and country nullable. The default connection skips a null. The other connection treats a null as a value. Store ('en', null) on the other connection, then validate the same pair inside that connection's transaction. Validation returns true, and the save then fails on the unique index. This lookup names no connection, so it reads the default connection's policy.
|
|
||
| protected String queryH2Mode() { | ||
| try { | ||
| DataSource ds = getDataSource(); |
There was a problem hiding this comment.
A datastore built from a supplied session factory has no stored data source, so this throws and the null policy is cached as skip. A duplicate key that is partly null then passes validation and fails in the database. Read the mode from the session factory when no data source is stored.
Problem
UniqueConstraintadded a condition for a property of the unique group only when its value was notnull. Anullgroup value dropped the property from the query, solanguage unique: 'country'checkednew Language(language: 'en')aswhere language = 'en'and rejected it because('en', 'US')exists. The same happened for theRequestmap(url unique: 'httpMethod') of the issue and for a self-referencingFolder(name unique: 'parent'). This is stricter than any reading ofnull: in SQLnullvalues never conflict, and ifnullis a value it conflicts only with anothernull. All datastores were affected.Fix
The reading of a
nullin a unique key is a choice, so the constraint now has four readings, for the group and for the constrained property alike:wildcard: anullgroup value matches every row (the behavior before this change).skip: a row with anullin its key is never a duplicate (SQL).value: anullmatches only anothernull.database(default): the reading of the database's own unique index, so the validation and the index agree.A constraint selects its reading with
unique: [group: ..., nulls: ...], andgrails.gorm.unique.nullsselects it for every constraint without one. The datastore supplies thedatabasereading: Hibernate 7 and 5 key it on the dialect, like the other per-database enums (Oraclevalue; H2, PostgreSQL, MySQL, MariaDB, SQL Server and DB2skip), and a datastore without such an index, such as MongoDB, readsvalue. Hibernate's mapping builders andPropertyread the map form as its group, so it does not create a single-column unique index.Behaviour change (who is affected)
The default changes from this branch's earlier hard-coded
valueand from the old wildcard behavior todatabase. Applications that relied on anullgroup value matching every row get a successful validation when the rows differ in the group property;nulls: 'wildcard'(orgrails.gorm.unique.nulls: wildcard) keeps the old rule. The upgrade guide ((section 20)) and theuniqueconstraint reference describe the readings and the reading each database gets.Tests
UniqueNullsSpec(in-memory datastore): every reading on a nullable key and on aFolder-shaped association key, the setting, and rejection of bad options.UniqueConstraintNullsSpec(TCK, every datastore that registers the constraint): the same probes per reading, and the database-following reading checked against what the datastore reports.UniqueNullsHibernate7Spec: on H2, PostgreSQL, MySQL, MariaDB and Oracle containers, the validation agrees with whether the database accepts the same row twice; the setting overrides the database, and a constraint option overrides the setting.UniqueConstraintNullGroupValueSpecandUniqueConstraintSpeckeep their cases, namingvalueexplicitly.:grails-datastore-core(360),:grails-datamapping-core(2198),:grails-datamapping-core-test(729),:grails-data-hibernate7-core(3870). Hibernate 5 compiles and passes checkstyle and CodeNarc; its tests, MongoDB and Neo4j were not run here.Relates to #14503 and #14656 (the mirror of that issue); the default is the question left for the team on #14503.
Update (b273f8e, 62c9a95)
SupportedDatabase); the SQL syntax, regular expression pattern and unique-index null handling read from it.DatabasePlatformandH2Modeare removed from core, which keepsUniqueNullsand a database-freeClassKeyedlookup in aninternalpackage.MODE=DB2readsskip; H2 compatibility modes are read on Hibernate 7 only.rlikeon PostgreSQL with Hibernate 7 is now~*, as on Hibernate 5, so the match ignores case (checked against PostgreSQL, MySQL, MariaDB and Oracle containers); on a database with no regex operator Hibernate 5 now falls back tolike, as Hibernate 7 does, instead of throwing. The reference docs state both.SqlSyntax,UniqueIndexNulls,RegexDialectPatternandDialectKeyed, which came from 8.0.x, so expect a conflict when 8.0.x merges up.https://claude.ai/code/session_01Dmki4Tgxp7dLBhzpe2xT93