Repository navigation
feat(build-logic): add opt-in PITest mutation testing support and skill - #16819
borinquenkid wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Mutation-testing configuration can silently skip intended classes and tests, blanket exclusions omit handwritten behavior, and the added tests do not verify actual mutant execution or configuration-cache reuse.
5 open issues
Avoid blanket method-name exclusions that skip handwritten behavior · New Install namespace fallbacks before module groups are assigned · New Set an all-tests default independently of the production-class selector · New Add TestKit coverage for auto-enablement, Spock discovery, and cache reuse · New Use existing production classes and matching spec packages in examples · New
What changed in this PR
Adds opt-in PITest mutation testing to Grails’ shared build conventions, with targeted execution controls and contributor guidance for assessing test effectiveness.
Changes:
- Adds PITest enablement through the analysis extension, Gradle properties, and direct task requests.
- Configures mutation targets, reports, concurrency, and Groovy bytecode exclusions, with configuration-level tests.
- Introduces a mutation-testing skill and links it from existing contributor guidance.
| File | Description |
|---|---|
gradle.properties |
Pins PITest engine, Gradle plugin, and JUnit 5 plugin versions. |
build-logic/plugins/src/test/groovy/org/apache/grails/buildsrc/GrailsCodeAnalysisPluginSpec.groovy |
Tests enablement, target selectors, and exclusions. |
build-logic/plugins/src/main/groovy/org/apache/grails/buildsrc/GrailsCodeAnalysisPlugin.groovy |
Implements PITest activation and configuration defaults. |
build-logic/plugins/src/main/groovy/org/apache/grails/buildsrc/GrailsCodeAnalysisExtension.groovy |
Exposes the enablePitest() opt-in. |
build-logic/plugins/build.gradle |
Adds the PITest Gradle plugin dependency. |
AGENTS.md |
Adds mutation testing to the skill index. |
.agents/skills/test-fixer/SKILL.md |
Links assertion-depth work to the new skill. |
.agents/skills/README.md |
Lists mutation testing as a contributor-only skill. |
.agents/skills/mutation-testing/SKILL.md |
Documents commands, reports, exclusions, and semantic mutation checks. |
Review effort: Balanced
- Integrate info.solidsoft.pitest (1.19.0 / PITest 1.19.6 / pitest-junit5-plugin 1.2.2) into GrailsCodeAnalysisPlugin and GrailsCodeAnalysisExtension - Auto-enable PITest when the pitest task is requested on the command line or via grails.code-analysis.enabled.pitest[.projects] - Disable upstream blanket FGROOVY filtering so @CompileStatic Groovy classes are mutated alongside Java classes while excluding synthetic Groovy compiler helpers - Lazily evaluate targetClasses defaults and default targetTests to * independently of targetClasses - Add .agents/skills/mutation-testing/SKILL.md covering targeted PITest execution and the Java/Groovy semantic mutation checklist - Reference the mutation-testing skill from AGENTS.md, CLAUDE.md, GEMINI.md, and companion skills
4bc4260 to
95f25fe
Compare
PITest only mutates the classes of the module it runs in, so a class could not be mutation-tested when its specs live in another module (JpaQueryBuilder is in grails-datastore-core, its specs in grails-datamapping-core-test and the Hibernate modules). -Pgrails.code-analysis.pitest.mutableProjects=:other-project[,...] adds the compiled main classes of those projects as mutable code and runs their classes task first. A path that does not exist fails the build with a message that names the property. grails-datamapping-core-test, a test-only module, now applies grails-code-analysis so it can run PITest. Covered by a TestKit build with two projects: the class of the other project is mutated and killed with the property, left alone without it, and a missing path is refused.
|
@jamesfredley added |
jamesfredley
left a comment
There was a problem hiding this comment.
Not approving. A mutation run of an extracted test never sees that test.
| defaults | ||
| }) | ||
| ) | ||
| pitest.targetTests.set( |
There was a problem hiding this comment.
A module whose tests are extracted for the shared suite runs those tests normally. A mutation run of one of those tests reports no coverage. The extracted classes are not on the mutation classpath. Put them there.
A module that runs the shared TCK extracts its specs from the tck jar into a directory only its Test tasks read, because JUnit does not find tests inside a jar. PITest finds tests the same way, so a mutation run of a TCK spec examined no test and reported every mutant as having no coverage. The jar is extracted again for PITest, apart from the Test tasks' directory (their cleanup task deletes it), and that directory is added to the mutation classpath. A module without a tck configuration is unchanged. Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
|
Fixed in 659f040. The extracted classes were not on the mutation classpath, as you say. A module that runs the shared TCK extracts the specs from the Run on Hibernate 5 with Other shapes I did not run: |


Summary of Changes
build-logic:grailsCodeAnalysis.enablePitest()opt-in and-Pgrails.code-analysis.enabled.pitest[.projects]overrides inGrailsCodeAnalysisPluginandGrailsCodeAnalysisExtension.pitesttask when invoked directly on the command line (e.g.,./gradlew :<module>:pitest).$getStaticMetaClass,getMetaClass,setMetaClass,invokeMethod,getProperty,setProperty,$getLookup,pfget$*,pfset$*,super$*,this$*), closure classes (*$_closure*,*$_*), and runtime call sites.-Pgrails.code-analysis.pitest.targetClasses,-Pgrails.code-analysis.pitest.targetTests,-Pgrails.code-analysis.pitest.threads, and-Pgrails.code-analysis.pitest.mutators.GrailsCodeAnalysisPluginSpecto verify PITest opt-in, CLI property overrides, and Groovy synthetic exclusions..agents/skills/mutation-testing/SKILL.mdand updated skill indexes inAGENTS.md,.agents/skills/README.md, and.agents/skills/test-fixer/SKILL.md.