Skip to content

Use canonical Kafka schema settings in generated configs (2.x) - #739

Merged
stevehu merged 3 commits into
2.xfrom
fix/264-canonical-schema-registration-2x
Nov 3, 2026
Merged

stevehu merged 3 commits into
2.xfrom
fix/264-canonical-schema-registration-2x

Conversation

@stevehu

@stevehu stevehu commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Migrate producer, consumer and Streams templates to auto.register.schemas with matching canonical values.yml placeholders and generated values entries. Related: networknt/light-kafka#264; light-example-4j is explicitly out of scope.

Document the required override rename: old Confluent settings were ignored; canonical false requests lookup-only and requires registered schemas. Preserve the deliberately supported Light serializer legacy fallback. Existing defaults remain true.

Validation: mvn -o -q -pl light-hybrid-4j -am -Dmaven.repo.local=/home/steve/workspace/artifacts/implementation/light-4j/jackson-3-migration/m2-phase5-baseline test passed 13 tests with no failures/errors/skips, including all eight Kafka flag combinations for both generators, exact correspondence between emitted configs and values entries, and canonical/old override injection; git diff --check passes. Local validation uses the preserved Jackson 2 cache. No Jackson 3 changes or deployment are included; remote CI remains separate.

The values template now receives the actual consumer/Streams file emission decisions, preserving the existing server consumer-flag-to-Streams behavior. Generated values and the README prominently require migration of old false overrides; no legacy alias is introduced.

@stevehu

stevehu commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Review findings

These apply to the shared template changes. I verified the first one on the #738 branch, and 2.x has the same generator structure.

1. values.yml entries don't match the generated kafka ymls (values.rocker.raw)
HybridServerGenerator emits kafka-streams.yml when kafkaConsumer is true and never emits kafka-consumer.yml. With only kafkaConsumer=true, the server's kafka-streams.yml reads kafka-streams.auto.register.schemas, but values.yml only has kafka-consumer.auto.register.schemas. Setting that to false has no effect. With only kafkaStreams=true, values.yml has a dead entry. Please gate each entry on the same condition as its yml.

2. The old override key is silently ignored (kafkaProducerYml.rocker.raw, plus consumer and Streams)
An existing kafka-*.schema.registry.auto.register.schemas: false is no longer referenced, so the template falls back to true and begins auto-registering with no warning. Consider a fallback or a loud failure. This is a plausible risk, not reproduced.

3. Test (KafkaSchemaRegistrationGenerationTest)
It enables all three flags, so it hides item 1. The skip condition is convoluted, and assertTrue(values.contains(client + ".auto.register.schemas: true")) asserts the dead server kafka-consumer entry. The final direct consumer render check duplicates the service-folder assertions.

@stevehu
stevehu merged commit d01c7a8 into 2.x Nov 3, 2026
1 check passed
@stevehu
stevehu deleted the fix/264-canonical-schema-registration-2x branch November 3, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant