You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Use canonical Kafka schema settings with Jackson 3.1.8 - #738
Generated Kafka configurations still emitted the unsupported schema.registry.auto.register.schemas key and matching external override names. Use auto.register.schemas in the producer, consumer and Streams templates, expose corresponding values entries only for the configurations each generator emits, and document the required migration. Related: networknt/light-kafka#264. light-example-4j is out of scope.
Keep Jackson pinned at 3.1.8. Adapt configuration-only YAML scalar normalization to APIs available in 3.1.8, preserving Core/legacy booleans, empty/valueless nodes, non-finite numbers, numeric parsing, quoting, anchors, encoding and stream limits. Ordinary JSON scalars bypass the event pass; unchanged text avoids an output copy.
Merge current master and preserve the removal of Jackson-specific CI. The PR uses master's plain Maven CI and makes no workflow or qualification source-pin changes relative to master.
Validation:
Full local mvn -o -q clean install: 86 passed, 6 skipped, zero failures/errors, without a Jackson version override.
Generation regression covers all eight Kafka flag combinations for both server and service generators, exact correspondence between emitted files and values entries, and actual canonical/old override injection.
YAML regressions cover valueless flow/explicit keys, empty leading documents, non-finite values, Unicode/encoding, anchors, quoting and numeric limits. These edge-case regressions also passed against a separate master + Jackson 3.2.4 baseline.
git diff --check passes. Fresh remote Maven CI is separate from the local cached-dependency evidence above.
Old external override names remain deliberately unused under the existing compatibility policy. The README and generated values warn that an old false override alone leaves the canonical true default; rename overrides before regeneration and register schemas before selecting canonical false. Light's intentional serializer legacy fallback is preserved. No deployment is included. Companion 2.x PR: #739.
I ran GeneratorConfigYaml from this branch against master with Jackson 3.2.4 as a baseline, and generated Hybrid server output with single Kafka flags.
1. Empty-scalar rewrite corrupts valueless keys and empty documents (GeneratorConfigYaml.java:95)
The " null" rewrite splices text at the empty scalar's start mark. For valueless keys that mark is the next token, not the position right after :.
? a / ? b gives {"a null? b null":""}; baseline {"a":null,"b":null}.
---\n---\na: 1 gives the string "null--- a"; baseline null.
Suggested fix, verified in a scratch copy: enable YAMLReadFeature.EMPTY_STRING_AS_NULL (present in 3.1.8) on the factory builder and drop the value.isEmpty() ? " null" branch. All three cases then match the baseline, and x: and - still give null.
2. values.yml entries don't match the generated kafka ymls (values.rocker.raw:69) HybridServerGenerator emits kafka-streams.yml when kafkaConsumer is true and never emits kafka-consumer.yml. With only kafkaConsumer=true, the server gets kafka-streams.yml reading ${kafka-streams.auto.register.schemas:true}, but values.yml only gets kafka-consumer.auto.register.schemas. Following the README and setting that to false does nothing. With only kafkaStreams=true, values.yml has a kafka-streams entry but no kafka-streams.yml exists. Please gate each values entry on the same condition as its yml in each generator.
3. Non-finite floats now throw (GeneratorConfigYaml.java:96) .inf, -.inf, +.inf, .nan and .INF resolve to FLOAT in both resolvers, so they are never rewritten. Jackson 3.1.8 then throws StreamReadException: Malformed numeric value. The baseline returns strings such as "Infinity".
4. The old override key is silently ignored (kafkaProducerYml.rocker.raw:30, plus consumer and Streams)
An existing kafka-producer.schema.registry.auto.register.schemas: false (honored by Light's serializer fallback, per the README) is no longer referenced. The producer falls back to true and starts auto-registering with no warning. Consider keeping the old key as a fallback or failing loudly. This is a plausible risk, not something I reproduced.
5. Efficiency (GeneratorConfigYaml.java:47)
The regex fast path is gone, so every config gets a full snakeyaml event parse plus Jackson's parse. The BOOL and NULL branches also rewrite plain true/false/null to themselves. Guard with !core.equals(JSON_SCALARS.resolve(value, true)) and consider a cheap pre-check.
6. Test gaps
GeneratorConfigScalarCompatibilityTest has no cases for valueless keys, empty leading documents or non-finite floats.
KafkaSchemaRegistrationGenerationTest enables all three flags and uses a convoluted skip condition. It asserts the dead server kafka-consumer entry as correct and duplicates the final consumer render check.
7. Cleanup
normalizeBooleans and the class Javadoc still describe legacy booleans, but the method now handles Core null/int/float too.
The README's "Jackson 3.1.8 source validation" section holds merge-gating prose that goes stale after merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated Kafka configurations still emitted the unsupported
schema.registry.auto.register.schemaskey and matching external override names. Useauto.register.schemasin the producer, consumer and Streams templates, expose corresponding values entries only for the configurations each generator emits, and document the required migration. Related: networknt/light-kafka#264. light-example-4j is out of scope.Keep Jackson pinned at 3.1.8. Adapt configuration-only YAML scalar normalization to APIs available in 3.1.8, preserving Core/legacy booleans, empty/valueless nodes, non-finite numbers, numeric parsing, quoting, anchors, encoding and stream limits. Ordinary JSON scalars bypass the event pass; unchanged text avoids an output copy.
Merge current master and preserve the removal of Jackson-specific CI. The PR uses master's plain Maven CI and makes no workflow or qualification source-pin changes relative to master.
Validation:
mvn -o -q clean install: 86 passed, 6 skipped, zero failures/errors, without a Jackson version override.git diff --checkpasses. Fresh remote Maven CI is separate from the local cached-dependency evidence above.Old external override names remain deliberately unused under the existing compatibility policy. The README and generated values warn that an old false override alone leaves the canonical true default; rename overrides before regeneration and register schemas before selecting canonical false. Light's intentional serializer legacy fallback is preserved. No deployment is included. Companion 2.x PR: #739.