Repository navigation
Make Grails Forge the one generator and grails the one command line - #17114
jdaugherty wants to merge 284 commits into
Conversation
…ter a comment The scanner of GSP pages reported a token on the last line of a page on line 1, and reported everything after a comment of several lines too early: it removes a comment from the text it scans and kept the places its lines began at as they were before. The line a compile error, a tag error and an expression error name, and the line mapping a compiled page keeps, were wrong for such pages, which is most templates of a Forge feature pack: they start with a licence header. A token on the last line is now reported there, and the lines of a removed comment still count. Part of apache#16348
The features of an application are now selected in rounds until nothing is added: the named ones, what they require, the defaults for what nothing has chosen and what those defaults require. Features that require each other in a circle are selected together, a requirement that cannot be met names the way that led to it, and what is refused does not depend on the order the features are named in. The conflicts of a feature pack are checked against the final features, with the defaults and with what other features bring in, when the context is made. Before, a pack found them only when it was applied, after the templates of the packs before it had run, and a caller that only built the context did not see them. The create commands take --exclude-features, which leaves a default out. A name that is no feature, that cannot be selected by name, that is selected too or that a feature of the application requires is refused. The option takes names: the coordinates of a plugin are refused there. The exclusions are recorded, sorted, in .grails/project.yml, and in grails-forge-cli.yml when there are any; the code generation commands read them from the state of the project first. An option given on the command line is a choice and no longer gives way to a feature without a word: --data hibernate5 with the feature gorm-hibernate7 made a Hibernate 7 application, and --reloading devtools with grails-micronaut one without DevTools. Both are refused now. An option that is not given is a default and gives way as before. Part of apache#16348
… an error A template was compiled every time it was rendered. It is now compiled once for the pages it is rendered with, kept by a digest of its feature, the directives and its source: once for a generation for the packs that generation is given, whose compiled pages are still not shared between generations or between packs, and once for a run of Forge for the packs Forge has in itself, which the hosted Forge renders from many threads at once. 200 generations with asciidoctor went from 3,690 ms to 644 ms on one thread and from 1,207 ms to 162 ms on eight. An error in a scriptlet, a compile error and a tag the parser refuses now name the line of the template, and a compile error no longer names the generated class. Which tags are instructions to the generator and that every other tag is written out is pinned by a spec and stated in the guide, with the ways to write a view whose own GSP has to survive. The descriptor keeps the template API it declares, the model is made for that number in one place, and the Javadoc table, the guide table and the names a template is given are held to each other by a spec. Part of apache#16348
An exclusion could leave an application that does not build: a REST API without views-json lost the hidden grails-web feature, and with it the Grails Gradle plugin. An exclusion is now refused when the application would be without a hidden feature it has when nothing is excluded, and when it names the Grails Data implementation the option chooses, which the options and the record of a project cannot say is absent. An option that is given is a choice in the generator and no longer only in the create commands: Options says which options were given, the generator refuses an application that would not have what one of them asks for, a given "none" included, and the hosted Forge marks the options a request has. What is recorded of an option that was not given is what the application has, so an application made with the feature of Hibernate 7 and no option is recorded as one. Every refusal of a selection is a FeatureSelectionException, which the create commands report as a mistake in the command line (exit code 2). A name that was given is shown in one bounded line that is no markup; names that are no features are all reported, sorted. A conflict and a missing requirement name the way that led to them, a long one by its ends. What a default requires of an exclusive group is selected before the group is given its default. --list-features checks the exclusions and marks the defaults without them. The design says that a cycle among requirements is a mutual requirement and is accepted. Part of apache#16348
…om a false pass A review of checkForgePack found that the class path Forge runs with was not Forge's own: groovy.version forced its Groovy, and a BOM property the project overrides put a strict constraint on it. The tooling configuration is now left out of everything the Grails plugins do to the versions of every configuration of a project. A check that was ended before it was done passed: a template that ends the JVM gave exit 0 and no output. Forge now writes one line for a pack it checked, and the task takes exit 0 for a pass only with that line. A spec of Forge holds the class name and the line to the constants of the task. The task also says what went wrong where Gradle or Java did not: a Forge that is in no repository of the build, with the override to name another; a toolchain older than Forge needs; and a Forge older than the plugin expects. Publishing a plugin to a repository now checks its pack, which publish did not depend on; publishToMavenLocal and the jar still do not. The sample applications of the check go through the Java versions too, and the guide lists them and what they leave unrendered, says that problems of the descriptor come before those of the templates, and that the check stops at 50. Part of apache#16348
They are there to catch an input that takes the parser far longer than its size should, not to measure it. With other builds running on the machine one of them ran out of its twenty seconds. Part of apache#16348
A review of taking a feature pack by the coordinates of its plugin found that a repository could keep the command waiting without end, end it in a stack trace, and have a password of GRAILS_REPO_URL printed. An answer that stops, before or after its first bytes, is now ended at the time an answer has, and a file at the time a file has. Whatever a repository answers, or the client throws for it, is one line that names the repository, with exit code 1; so is a jar that holds no usable pack, with its problems and without the usage of the command. An entry of GRAILS_REPO_URL a pack cannot be taken from is left out with a warning, since the variable is written for the generated build, and no message repeats the user, password or query of an address. A redirect is checked for its host and port, and is not followed from a repository out there to this computer or its network. The checksum of a jar a repository gave and answers 401 or 403 for is taken as not published. The build of a snapshot has to be the version with a time and a number, of the artifact that was asked for, and the newest one is taken. The copy of the jar is hashed again after the pack is read, so that the digest that is recorded is that of what was read. The feature list names the jar it took, and a pack that declares its plugin in another version than the one it was taken from is warned of. Part of apache#16348
A review found the state under .grails short of its own claims. The reader built everything a file held before it bounded any of it, and every command read the state: an 8 MiB index cost seconds and gigabytes. The lines of a file are now counted and measured before the YAML library sees it, and its events before anything is built; the code generation commands read the manifest and the lock only, and a create command and --version read nothing of the directory. The lock was covered by nothing. It now ends with the digest of itself, and its options and features have to agree with the manifest. It no longer names the operating system, which no generated file depends on, and it says which repositories GRAILS_REPO_URL gave the build, without a path or a password. The generator renders everything and reads the state back before it writes the first file, with one set of limits for writing and reading. The manifest is the commit point: a generation that did not finish is replaced by the next one, and a .grails that is a file or a link is refused before anything is written. The kept files of --inplace that are not as generated are named. The copy of a .gitignore in the baseline was read by Git: it is kept under another name, and .grails/.gitignore has every copy committed. grails-forge-cli.yml is no part of the baseline, and the commands warn when it disagrees with the state. The hosted difference leaves out all of .grails, so that applying it does not break the state. A file is compared with the baseline without regard to line endings when it is text. A package with the name of a Windows device is refused where the name is given: a usage error of the create commands, and 400 from the hosted endpoints, whose ZIP is now made before its response begins. Part of apache#16348
…s and in a long page The table a compiled page keeps of the line of the page each line of its class was written for had one entry for each line the parser ended, and none for the line breaks inside what it printed. After an expression, a scriptlet or a tag attribute of several lines, a compile error and a line of a stack trace were mapped to a later line than they are on. The table also lost an entry each time it grew, at 1,000, 2,000 and 4,000 lines of the class. Every line break that is written is now recorded, and the table grows before an entry is put in it. The scanner looked at the start of every line of the page each time it removed a comment, which made a page of many comments slow to parse: 30,000 comments in 90,000 lines took over 20 seconds. It now keeps where comments were removed and finds the line of a token from that, in the 1.4 seconds the removal itself takes. What is only found where a page ends, such as a tag that is never closed, is reported on the last line of the page, also when the page ends with a line break, where it was reported on a line after it. The upgrade note says for which pages the line changes and that precompiled pages carry the lines. Part of apache#16348
A template was compiled as a page of a web application without tag
libraries: head, title, meta and body anywhere in a file were rewritten
to the capture tags of a layout, and every tag the generator does not
run was written anew, its attributes evaluated, so params="[id: 1]" came
out as {id=1} and a map that names a variable of the view failed. Of the
655 views of the example applications, 14 came out as typed.
The ten tags the GSP compiler compiles into a page are the instructions
to the generator. Every other tag, of any namespace, is now text and is
written byte for byte, with the expressions in it evaluated. A backslash
in front of a scriptlet, a directive, a comment or an instruction tag
writes it for the view, as a backslash in front of an expression does.
Layout preprocessing and static compilation are off for a template. A
spec makes a template of each of the 655 views the way the guide says
and has all of them written back as they are.
What an error is, is asked of what caused it, so writing too much and a
stack overflow are said as such inside an expression; an interrupt
leaves the thread interrupted; a template the parser refuses is compiled
once; and the key of a compiled page has the length of each of its
parts.
The guide, the Javadoc of the template API and the views are inputs of
the tests, and the examples of the guide are run from the guide.
Part of apache#16348
The scripts of the Gradle and Grails wrappers are written byte for byte, and Git converted them like any text: a clone with core.autocrlf=true had gradlew with the line endings of Windows, and a commit with it set stored gradlew.bat with those of Unix. The scripts then did not run where they are meant to, and to the baseline of the project they were changed files. Every generated application now has a .gitattributes at its root, written by the Gradle build feature beside .gitignore: the .bat files are checked out with CR LF and gradlew and grailsw with LF on every clone, and the wrapper jars are never converted. Nothing else is said: how the text files of an application are stored stays with the settings of whoever commits. Checked with Git for a commit and a clone with core.autocrlf false, input and true each: every file of every clone is now as generated, and the baseline under .grails, whose own .gitattributes unsets text, keeps its bytes. Part of apache#16348
…is true of a failed check A jar of Forge's class path that a repository named and did not have was left out without a word: the check then gave the wrong hint, a missing class, or passed. What could not be resolved or fetched is now found with the configuration and fails the task before Forge is started, by file and module. Forge that was found is no longer said to be missing when what it needs is, and a build whose tooling came out that way does not keep its configuration cache entry, so the next build looks in the repositories again. The task tells a check that passed and had other lines written beside its own from one that never said it checked, gives an end with 1 and no problem a hint as an end with 0 has, and gives the hints for an older Java and a missing class only for what the launcher of Java writes, not for any text with those words. Forge is started without the lines SLF4J writes for having no provider. The BOM property overrides plugin no longer imports the Grails plugin: its extension takes the names of configurations to leave alone, and the Grails plugin names the tooling configuration there. Part of apache#16348
…per than asked A second review of taking a feature pack by the coordinates of its plugin found that the metadata of a snapshot could overflow the stack of the command, that a redirect to this computer written as one number was followed, and that the digest that is recorded could be that of other bytes than the pack was read from. The jar is now read into memory and the pack is read from that array, by a reader of the directory of a jar that the loader has for bytes: it is written to no file, so there is nothing to swap, to delete or to sweep. The metadata of a snapshot is read as a stream, to a bounded depth and number of elements, and nothing a repository sends leaves the search but as its one failure, an overflowed stack included. What watches the time of a file is ended whatever ends the file. A host is a name or an address written as one, and other ways of writing an address are refused; a redirect is not followed to an address that is private, shared, multicast or carries such an address. No address is repeated in a message as it was given, since a password can be where a parser does not look for it, and the generator's own messages about GRAILS_REPO_URL follow that rule. An entry of that variable that is not an absolute path is not asked, the warnings about its entries are few, and a pack that cannot be used is told from two that cannot be used together by the type of the refusal. Part of apache#16348
A second review of the state of a generated project found more that a directory or the environment could do to the command line. grails-forge-cli.yml was parsed without bounds and its values taken as they were: a value that is none, a file that is no YAML or only a part of a record, ended every command that reads a project in a stack trace. It is now read as the state is, each value is held to what it can be, and a file that cannot be used gives one line of warning and a directory without a project. An entry of GRAILS_REPO_URL was written into the build file as it was, so that a line break in it was code of the build; this is older than the state. An entry with a control character, a quote or a $ is refused without being repeated, and what is written is escaped. The lock has of an address the scheme, host, port and path only, names fifty repositories and counts the rest. A name with a control character or a path separator, or one a file system does not take, is refused where it is given, as a mistake in the command line, and no longer by the read-back of the state. .grails/.gitignore now has every file of the state committed, also under a global *.yml or .grails/*, and a state whose baseline is there without its manifest or lock is said to be what an ignore rule leaves behind. A file where the project needs a directory, and a directory that cannot be written to, are seen before anything is written; a failure while writing names the file by its path in the project and says what was written before. Part of apache#16348
The table of lines a compiled page keeps took a carriage return by itself for the end of a line of the class. The Groovy compiler counts line feeds and nothing else, so after a stray carriage return in an expression, a scriptlet or a tag attribute every later error was named on an earlier line than it is on. Only a line feed is counted now, and a spec asks the compiler what it counts. An expression that is never closed is named on the line it starts on, not on the line the text in front of it starts on. An escaped expression was written with its backslash where the scanner cut the text in front of it at 64,000 characters, between the backslash and the expression. The text is no longer cut there. A bracket in an expression that closes nothing was an EmptyStackException of the scanner. It is left for the compiler, which names the line. The rows of the spec of lines now each fail without the fix they are for, and the upgrade note says what changed and what did not. Part of apache#16348
The generator read more in a template than the guide said. The short
directive of a page, which every scaffolded view starts with, was run
and had no escape, a declaration in braces failed the template, and the
form for which a page strips a tag took text out. They are text now, as
every tag is that is no instruction, so what the generator reads is
what the guide lists: expressions, what starts with <% or %{, and the
ten instruction tags.
Backslashes in front of those, and in front of any tag, are an escape
two by two: two write one, and one left over has what follows written
and not read. A backslash in front of a tag that is text is no longer
written, so every tag of a view can be escaped.
An instruction with env, which would ask for the environment of the
generator, and one whose test is not one expression, which is always
true, are refused with their line. A template with more than a page can
hold, or nested too deeply for the compiler, is told so in words, and a
name that is missing because a g:set is text gets a hint. A tag that is
text costs a page less than before: 1,926 pairs fit where 800 did.
The views the specs write back are 31 copies under the test resources;
every view of the sources is tried with -PallRepositoryViews, 745 of
745. The pack check runs in the environment the command line generates
in.
Part of apache#16348
An application could be made without gsp or without the asset pipeline and did not work: a web application without gsp answered its home page with an error, since its URL mappings are to GSP views, and without the asset pipeline its pages wrote their asset tags out as text. A feature now says which features what it generates relies on, and an exclusion that would leave it without one of them is refused. Selection never adds what is relied on, so no selection without exclusions changes. Over HTTP an option counts as given only when its value is not the default. The Forge UI sends every option with every request, so a feature ticked there beside an option left alone was refused. The default servlet container and the default reloader stand aside, when their option was not given, for a feature of their group that is selected or required: a feature pack that requires Jetty gets Jetty and not Tomcat with it. An option that was given still does not give way. Of the Grails Data implementation only the default gives way, to a selected Hibernate 7, as before: another value is the choice of the caller whether it is marked as given or not, and is not swapped. What is recorded of the options is what the application has: a plugin has no servlet container and is recorded with none, where it was recorded with Tomcat. A JDK the generator does not support is a mistake in the command line, like every other value an option cannot have. Part of apache#16348
…slices The Forge CLI gets a gradle command that runs the Gradle wrapper of the project the current directory is in. Every argument after the name of the command reaches Gradle as the one argument it was: the command takes the rest of the command line before picocli reads it for options and has none of the options the other commands share, so -x, -v, --stacktrace and -- are Gradle's. The shell's command of the same name joins its arguments into one string, which makes two task names one task that does not exist and never passes an option on. The project is the nearest directory upwards with a grails-app directory or a record of a generated project, and the wrapper the nearest one above it, both within the Git repository. Gradle runs in the project directory, so in a build of several projects a task name selects that project. The command ends with 0 when Gradle does, with 1 and one line when Gradle fails, when there is no project or no wrapper, and with 2 and its usage without arguments. On Windows an argument a batch file would not receive unchanged is refused. Cancelling the command or ending the CLI ends the Gradle it started. The Tooling API is not used: it is on the class path of the shell only and its version is not managed by the BOM. The design document gains the cut of step 4 into nine slices, with what of each item of section 2.5 exists in the Forge CLI, in the shell only, or nowhere at 40c07df93c, and the order the slices depend on. Part of apache#16348
The Forge CLI gains `feature add FEATURE... [--dry-run] [--diff]`, which adds features of Grails to the application in the current directory from the state the generator wrote under .grails. The application is generated again in memory as its manifest records it, which has to give the manifest, lock and baseline index the project has, and once more with the features added, through the generator and the selection the create commands use. Every file the two differ in is compared with the file of the project: an unchanged file is replaced, an edited text file gets the changes of the feature merged into it line by line and keeps its edits, and overlapping or adjacent changes, removed files, links, binary files changed on both sides and new files that meet another file are conflicts. With a conflict nothing is written. The files and the new manifest, lock and baseline are written as one change under a lock of the project, with copies of the replaced files and a journal in .grails/transactions. A failed write is rolled back, and an interrupted change is rolled back by the next feature add; a file that was changed meanwhile is left as it is and named. Refused: a project without state or with state that does not read, an application another generator version made, one that is not generated again as recorded, features of external packs, and a feature that would replace a feature, change an option or leave a generated file out. ProjectState.of is public so that the state the generator writes for the proposed application can be read from its bytes. Part of apache#16348
Checking a pack renders its templates for applications that are made up. Nothing told the author of a pack whether an application created with it can be built: whether its dependencies resolve, its build file blocks work and its sources compile. The Grails plugin now registers the opt-in task forgePackTest for a plugin with a pack in src/main/forge. It starts the new entry point PackFixtures of Forge in a JVM of its own, on the class path of grailsForgePackTooling, which generates one application for each fixture with the feature of the pack selected, through the generator and the file output handler the create commands use. Without fixtures there is one for each application type of the descriptor. The task then builds each application with its own Gradle wrapper, by default with assemble, writes the log of each under build/forge-pack-test and fails with the fixture, the failed tasks and what Gradle said went wrong. Nothing depends on the task: it builds applications, which takes minutes. PackFixtures has a contract as strict as that of PackCheck: one line for each fixture, a summary as the last line, and the exit codes 0, 1 and 2. A fixture the generator refuses is a result and not a failure of the run. A generated application without a settings file gets one that names it, since Gradle refuses to build a directory below the settings file of another build. What checkForgePack and forgePackTest share moved from CheckForgePackTask to the new base class ForgePackToolTask, and their common wiring to GrailsPluginGradlePlugin.configureForgePackTool. The check behaves as before. Part of apache#16348
A name given to --features of the create commands that is neither a feature of the framework nor that of a pack that was given is looked up in a feature catalog, which says which plugin has the feature, in which versions, and which SHA-256 the jar of each has. The pack is then taken as it is for coordinates that were typed, with the digest of the catalog as the one the jar has to have, so a catalog adds a name and nothing else to how a pack is trusted. The catalog is read by the command line alone, as untrusted JSON within limits, only when a name is looked up, and through the transport of the jar. The latest compatible release is chosen; a snapshot and a deprecated plugin are used by their coordinates. A name of the framework stays the framework's, and a name two plugins have is refused with the full names to choose from. --feature-catalog and GRAILS_FORGE_CATALOG name the catalogs, --list-catalog-features lists them, a copy of an https catalog is kept for a day, and --refresh and --offline say whether the network is asked. The lock of a project records the catalog and the digest of what was read of it. The plugin index does not record packs yet: the format read here is written down in the design as the contract to confirm with it. Part of apache#16348
A name that is not known was looked for in the plugin index, and when that could not be read the command failed with exit code 1 where it had refused the name with exit code 2. The catalog nobody named no longer makes a name worse off: when it cannot be read, the refusal is the one of a feature that does not exist, followed by a line that says the catalog could not be read and why. It is waited for for 5 seconds, and a failure is remembered beside the copies for a day, which --refresh ignores. A catalog that was named and cannot be read still fails the command. Whether the plugin index is read at all when nothing is named is one constant, PackCatalogs.DEFAULT_ON. A plugin Forge has a version for is taken from a catalog in that version and no other, and refused with the coordinates to insist with when the catalog has no pack for it, so a name does not choose a version the build of the application would not have. The specs of the CLI now fail on a request to another computer, with its address, and cover the default catalog through the command. Part of apache#16348
…to the Forge CLI The Forge CLI gets create-unit-test, create-integration-test, create-functional-test, create-restful-controller, create-domain-resource, create-command and create-script, which until now were commands of the profiles only. Each writes to the paths the profile command wrote to and is there for the application types its profile had it for. The name of the class is held to what a new name can be before a path is built from it: an identifier and a package of Java, no path, no control character, no device name of Windows; anything else is a mistake in the command line. A file that is there already is not replaced without --force, and then the command writes none of its files and fails, as the shell's did. All code generation commands, the six earlier ones too, now take their templates from one place: src/main/templates of the project when the template is there, and the template of the CLI otherwise. The locations and the two syntaxes are the shell's, so a template of a profile or of the Quartz plugin works unchanged: artifacts/ and testing/ with @artifact.name@ tokens, which are replaced and never run, and Job.groovy and artifacts/Script.groovy as Groovy templates, which scripts rendered and which are run. A project template is read as UTF-8 within a bound and only from inside the project, and one that cannot be read or rendered fails the command with one line naming the template and its line. The templates of the CLI are resources in that same form, which replaces the GSP views of grails-forge-cli and its use of the GSP plugin; the six earlier commands write what they wrote, except that the test of an interceptor asks for the controller by its property name, which an interceptor of two words needs. install-templates copies the templates of the commands a project has into src/main/templates, keeping what is there unless forced. Pack artefacts are not part of this: the pack descriptor has no section for them. The design document says what such a section needs. Part of apache#16348
A jar can be read from its end, by its directory, or from its start,
entry by entry, and the format lets the two disagree. The reader of
the bytes of a jar did only the first, and in its own way: of a jar
with a second directory and end record in the comment of its end
record it took another pack than the JDK reads from the same file, as
it did of a jar whose end record counts fewer entries than its
directory holds and of one with a ZIP64 end record, and it took jars
the JDK refuses. One file with one SHA-256 was so two packs: the one a
person looks at and the one that is generated from.
There is one reader now, PackJar, for a jar taken by coordinates and
for a jar given as a file, which is read into memory under the same
limit of 64 MiB. It takes a jar in one form only and refuses anything
else with a line that says what is wrong:
- the jar is its entries from the first byte, each directly after the
other, then the directory, then the end record of 22 bytes, with
which it ends: no comment, and no bytes in front, between or after.
A jar after prepended bytes was read before and is refused, since
bytes that belong to no entry are where a second jar is put
- one disk, one count, the directory exactly where and as long as the
end record says, nothing of ZIP64, and no mark of an end record in
the directory
- the directory is exactly the entries counted, in their order
- every entry, of the pack or not: stored or deflated, not encrypted,
no flags but the deflate options, the data descriptor and UTF-8; a
name that is strict UTF-8, relative, without control characters,
backslashes, empty steps, "." or ".."; no comment; extra fields
that are whole and of six kinds that carry times and owners, so
not the one that is a second name for the entry, which Python and
the zip tools take for its name and the JDK does not
- no name twice, and no file where another entry has a directory
- the header in front of an entry repeats the directory. With a data
descriptor, the checksum and the sizes are in the descriptor and in
the header are all zero, as the JDK and its jar tool write them,
or all there too, as Maven does
- the content of every entry is inflated and checked: it ends where
its compressed length ends and has its length and its CRC-32
What is read is bounded: fewer than 65535 entries, 4 MiB of names and
256 MiB of content together, and no name is made a string, so a jar
of 65535-byte names is refused in a line instead of ending the
command with an OutOfMemoryError.
Of 2,878 jars of a Gradle cache 2,854 are read; the others have a
comment, a ZIP64 extra field, an entry named "/" or 4 MiB of names.
A jar built by Gradle with zip64 = true is refused.
In a template, an escaped "<%" directly in front of a "{" left the
"%{" to start a scriptlet, which took the text up to the next "}%"
out or failed to compile; "\<%{" is what the way of the guide makes
of "<%{". The percent sign is written with the "<" now.
The address of a repository with an at sign in its query or fragment
is not shown: "https://admin:12345?x@host/" showed the user and the
start of the password as the host and the port.
Part of apache#16348
On a busy machine GrailsPluginGradlePluginForgePackSpec failed with "configuration cache cannot be reused because file 'src/main/forge/feature.yml' has changed" for a build before which nothing had touched that file. Every test of a spec has its project in the same directory, which is deleted and copied again between two builds of one TestKit daemon. With file system watching on, the daemon started a build with what it remembered of the descriptor of the test before, stored that as the input of the configuration the descriptor has been since the pack is packaged, and the next build saw the file as changed. The failure is the same with the plugin sources as they were before forgePackTest, and goes away with watching off, so the builds of GradleSpecification now run without it. forgePackTest itself reads no file of the pack while the build is configured. A spec now holds that: after an edit of a template, a new file, a file Gradle does not look at, and a descriptor that is spoiled or taken away, the configuration cache is reused and the task runs again. Part of apache#16348
The Forge CLI gets the commands of the shell that are a Gradle build of the project the current directory is in: compile, clean, assemble with its other names war and package, dependency-report, test-app, and in a plugin project package-plugin and publish-plugin. Each names the Gradle tasks the profile command of the same name runs and turns its own options into arguments of Gradle one for one, so a test filter or a property value with spaces or quotes reaches Gradle as the one argument it was; the shell joins them with spaces and splits them again. What follows -- on the command line is not read by the command and is given to Gradle unchanged. An option a command does not have is a mistake in its use and is not passed on. The commands run Gradle through GradleRunner, which is the code of the gradle command moved to a class of its own: one place that finds the wrapper, builds the command line, shows the output as it is written, turns the exit code into 0 or 1 with one line, and ends Gradle when the command is cancelled. The gradle command uses it and behaves as before. test-app keeps the selection of unit and integration tests, the filters and the options of the shell. Its --continuous is the continuous build of Gradle, which runs the tests at once and again when their inputs change; the watcher of the shell runs nothing when it starts and ends at once outside the interactive shell. The environment is test for test-app, production for assemble and development for dependency-report unless -Dgrails.env names another one; the environment in front of the command belongs to the dispatch of the CLI and is not read yet. stats and bug-report need no Gradle. stats prints the table of the shell, from the same directories and with the same rule for a line of code. bug-report writes the ZIP file of the shell, with what is under grails-app and src and build.gradle, but follows no link, leaves out files that are keys by their name, does not replace an existing file without --force, and writes nothing above 10000 files or 256 MiB. The configuration of the application is in the file as it was in the shell's, and the command says so. install runs nothing and names the Gradle task that installs a plugin into the local Maven repository: the task the profile command runs does not exist in a project that publishes with maven-publish. The option for system properties has a map type of its own. picocli asks the factory of the CLI for the map of an option, and the factory answers for java.util.Map with a bean of the application, into which the properties of one command line would be put and stay for the next. Part of apache#16348
The Forge CLI gains a catalog of the commands it has, built from the picocli model of the command line, so a command that is registered is in it without declaring anything a second time. Short forms: cA is create-app, cDC is create-domain-class and f a is feature add, by the rule of the Grails shell (ScriptNameResolver), applied to the name of the command and of a command of a command before picocli reads the command line. The name of a command wins over a short form, a short form that fits several commands is refused with their names and exit code 2 where the shell runs the first that fits, and nothing but the name of a command is rewritten: the arguments of create-app and of gradle reach the command as they were typed. What stands where a command is expected and is none is refused with the commands most like it, and is repeated without control characters or markup. help [COMMAND...] prints what --help prints, for a command of a command too, and help --json writes the commands with their options as one JSON document (format 1, described in the design and the guide), with what kept commands of the project from being found as its diagnostics. The document is shaped so that the commands of plugins can be described by it. The code generation commands, add-property and feature add work in every directory of a project: the record is read and the files are written in the project the current directory is in, found by ConsoleProject within its bounds. In the project directory they do what they did. The artefact commands and install-templates write through the same output, so they and the templates of a project under src/main/templates follow the project directory too. The CLI declares groovy-json, which it no longer gets from the GSP plugin, for the document. The interactive shell completes the commands there are after the last command instead of those it started with. Part of apache#16348
…gether The catalog, the Gradle built-ins and the artefact commands were written apart. Together, a command that is not offered in a project is refused in the catalog's words and not picocli's, the built-ins add names to the suggestions for a mistyped command, and assemble has aliases in the JSON catalog. Part of apache#16348
feature add undid an interrupted change from a journal in the project, .grails/transactions/current, before it looked at anything else. A cloned repository or an archive could carry such a journal and have the command replace or remove any file whose digest it named, and move a file in from outside through a linked backup directory. The journal of a change is now kept in the home directory of whoever runs the command, .grails/forge/transactions/<digest of the project path>/journal, in directories that are the user's alone. The staged files and the copies of the replaced files stay in the project, beside their targets, under a random name the journal records, so that every replacement is still a rename within one file system. What a project has under .grails/transactions is never read; the command says that it is there. Recovery reads the state before and after the change first, holds every path of the journal to the files of those two baselines, wants every directory of the change to be a directory, and names every file it puts back. The lock has to be a regular file: a pipe of its name is refused instead of opened. Undoing looks at every file before it touches one. A file the change wrote that was changed since stops it with nothing put back; a change whose every file was written is complete and only loses its journal; a file the change had not come to is left as it is, so an edit made while a change runs no longer blocks every later command. A feature that would take the place of a default (a database driver for h2, grails-layout, Hibernate 7) is still refused, with the two ways there are to it, and --diff and --dry-run now show what it would add and remove. The merge is its own bounded Myers diff: at most 200,000 lines and 2,000 changed lines a side, beyond which a file is a conflict at once. A change counts as touching the lines beside it that are like its own or that another shortest alignment would not keep, which makes the merge never bolder than git merge-file on 28,000 seeded cases; a committed reference holds 4,000 of them. A conflict shows the lines of both sides and what to do. The diff that is shown is cut at 400 lines a file and 2,000 in all, without format characters and carriage returns. Also: no window in which a second command fails on the directory of the lock, a file whose name differs in case is a conflict, the guide uses grailsw and says where conflicts are to be expected. Part of apache#16348
… files or none A cloned repository could choose both the place and the content of a file a code generation command writes: a link where a directory of the sources is expected, and a template of the project with any text. The commands and install-templates now write a file only when every directory from the project down to it is a directory and not a link, wherever the link leads, is no file, and is there under the very name that is asked for; a link in the place of the file is replaced when the command is forced and never written through, and a directory there is left alone. The rule is a subclass of the output handler in the CLI, used by these commands only: creating an application, in a new directory or in place, is unchanged. No template of a project is run any more. artifacts/Script.groovy has the tokens of the other templates, and Job.groovy, the location of the Quartz plugin's template, has its names in braces replaced and refuses, with the line, what the template engine of Groovy would have evaluated. The engine is no dependency of the CLI again. A command writes all of its files or none: every target is checked before the first is written, a name longer than 255 bytes among them, every text is rendered first, and when a write fails after all, what the command wrote before it is taken back. A failure is one line with the path of the file in the project and what is wrong. The six earlier commands follow the same rule, so a file that is there now fails them where it was kept with a warning; the files they generate are what they were. Names: Book.groovy is the name of a file and refused, the first letter has to have a capital of one letter, a hyphen stands between two words, and a script is named with letters a to z and digits. A template is found under the very names of its location and is refused when it has a NUL. Part of apache#16348
While an application starts, grails-startup-progress answers every request on its port with the Grails-Startup-Phase header, but only until the web server has the port. From then on requests reach the application, which answers without the header while BootStrap and the startup hooks still run, and only the status path goes on naming the phase. A client that is no browser could not follow the start to its end, because nothing told it where that path is: it is configurable and sits below the context path. Every answer that names the phase now also names that path, in the Grails-Startup-Status-Path header. A client asks any path, takes the status path from the answer and asks it until the phase is READY, or FAILED. The names of the two headers, the default status path, the phases and the request the command line sends are written down once, in run-app-protocol.properties among the test resources of this module. The specification here asks a starting application with those bytes and expects those answers, through a start that reaches its end and one that fails; the specification of the command line reads the same file. The two programs share no class, so the file is what keeps them from drifting apart unnoticed. Part of apache#16348
run-app --detach, and run-app in interactive mode, took an application for started when HEAD / got a response without the Grails-Startup-Phase header. With grails-startup-progress that is the moment the web server takes the port over: BootStrap and the startup hooks still run, and the command said that the application answers while it was still starting. A start that failed was waited for until Gradle ended or the time was over, which with a process that stays after a failed start is the whole of --ready-timeout. The command now follows the start where the application says how far it is. It takes the status path from the first answer that names a phase and asks that path until the phase is READY. When the first answer is the application's own, it asks the default status path once, which finds an application that was past its progress page already. FAILED ends the wait at once: the command says so, stops what it started, shows the end of the log and ends with 1. Without startup progress nothing changes: a response of HTTP, whatever its status, is all there is to go by, and the one extra request is answered as any path the application does not have. The guide describes the two headers for any program that waits for an application. The request stays a HEAD on a loopback address; only the status line and the headers are read, within the limit there was; nothing of an answer is shown; a status path is taken only when it is a plain path of the same port. A web server that expects TLS answers a plain request with 400, which was taken for the application: the request is now made over TLS then, and stays there once TLS answered. stop-app decides that the application is gone by its port being free. A port that nothing had taken, as for an application that is still starting, says nothing by being free, and stop-app said "Application stopped" all the same. It now looks at the port before it ends Gradle, and says instead that Gradle was stopped and that the end of the application was not seen. The exit code is 0 as before. Part of apache#16348
…ning stop-app ends the Gradle that run-app started and relies on the Gradle daemon to end the application, which is a process below the daemon and not below what stop-app ends. That was held only by a shell script written to behave that way, and the one feature that ran Gradle itself turned the daemon off. WarmGradleDaemonSpec runs the wrapper of this build against a build whose bootRun starts a program that answers on a port, with a daemon that is started by an earlier build. It holds that the application is a child of that daemon, that stop-app returns with the port free and the process of the application gone, and that the daemon is still there and idle afterwards. The same for a run-app that stays with the application, when its Gradle is ended, is killed, or the command is cancelled. Gradle does what the command relies on in all four cases, so nothing in the commands changes. The daemon is kept apart by a registry of its own and a JVM argument no other build has, in the Gradle user home there is. A user home of its own would download the distribution, and one that links to the distributions there are has Gradle's cleanup delete those it did not use. Part of apache#16348
The runner of a script command asked for the console in order to put it back afterwards, and asking creates the interactive console when there is none: a JLine terminal, a line reader and the history file of the user. The console of a script made a second set. So the JVM the build starts for a script created ~/.grails_history, saved the history when it ended, and had what JLine and the JDK say of a terminal in its error output. On the CI runners JLine warned, in some runs of the JVM and not in others, that the history file is accessible by other users, and on Windows with JDK 25 the warning of the JDK about native access, which ends with an empty line, came with the terminal. GrailsConsole.hasInstance() says whether there is a console without making one, and the runner removes the console of the script when there was none before. The console of a script opens no terminal, no reader and no history. A script that asks for input is asked once and has no answer, which is what it had from the empty input of the build; the question with a list of answers no longer asks again without end. The spec gives the JVM a home directory of its own and expects nothing in it, expects the standard output as it is, and leaves the empty line of a warning of the JVM out of the error output with the warning. Part of apache#16348
The spec runs the wrapper twice in one JVM, where a user runs two. The
class loader of the first run still had the jar of the command line open
when the second was asked to download it again, and on Windows a file
that is open cannot be replaced: the download was not moved into its
place, and the text the spec had put there was read as a jar ("zip END
header not found"). The class loader is closed between the two, as the
end of the first JVM closes it.
Part of apache#16348
The descriptor of the command grails-core registers by hand is a resource that goes into the jar as it is checked out, and the spec compares it to the bytes the compiler writes for a command, one line that ends with a line feed. On Windows the file was checked out with CRLF, so the spec failed there and a jar built there differed from one built elsewhere. Part of apache#16348
The table of lines of a GSP page has an entry for each line of the generated class as the Groovy compiler numbers them. Groovy 6.0.0 counts line feeds and nothing else, and the writer of the class was changed to do the same. The snapshot of Groovy 6.0.1 counts a carriage return by itself as the end of a line, so against it every error after a stray carriage return in an expression, a scriptlet or a tag attribute was named on a later line of the page than it is on, and the spec that pinned what the compiler counts failed. The writer now asks the compiler on the class path, once, and counts a carriage return that no line feed follows when the compiler does. A carriage return in front of a line feed is one line break for both. The spec of what the compiler counts has both answers and takes the one of the compiler that is there, and a spec of the writer covers both, since only one compiler is on the class path of a build. Part of apache#16348
The spec first shows that the console writes escape sequences without the option, with picocli told that the output is a terminal. The prompt is written by JLine, for which a file is a dumb terminal, and whether that keeps the colours of the prompt depends on COLORTERM in the environment: it is set on a developer machine and not on the Linux runners, where the two rows that write nothing but the prompt found no escape sequence. JLine is now told as well, so the spec does not depend on the environment. Part of apache#16348
…ler does The upgrade guide and the upgrade skill said that a line of a page ends as a line does for the Groovy compiler. The compilers of Groovy 6.0.0 and of the snapshot after it differ on a carriage return by itself, and the line of a page that an error names does not depend on which is on the class path, so the notes say what holds for a page. Part of apache#16348
The specification of stop-app for an application that had not taken its port compared the first two lines of the output. The wrapper of the specification is a shell that waits for a child, and such a shell says on its output that the child was ended, sometimes between the two lines. Only the lines of the command are compared. Part of apache#16348
The commands that are a build of the project created a GradleRunner each, which starts the Gradle wrapper as a process. They now say what they run Gradle for and are given what runs it by GradleBuild.of, the one place that decides it; GradleRunner is the implementation there is, so nothing a user sees changes. A build that is left running, run-app --detach, asks for the process runner by name, since nothing else can outlive the JVM of the command line. The Gradle Tooling API becomes a dependency of the command line, in the version of the wrappers: gradleToolingApiVersion is back in the gradle.properties of the root, with the comment that keeps it in step with the wrapper files, and the Gradle skill names it again. Nothing uses it yet. Part of apache#16348
compile, clean, assemble, war, package, test-app, dependency-report, package-plugin, publish-plugin, run-command, the commands of the application and the scripts by name, run-script, console, shell and a run-app that stays with the application are now run through the Tooling API of Gradle, from the JVM of the command line, by ToolingApiRunner. grails gradle and run-app --detach keep running the Gradle wrapper as a process: a passthrough has to be Gradle as it is on its own command line, and a build that is left running has to outlive the command line. The split is made in GradleBuild.of and nowhere else, and the environment variable GRAILS_GRADLE_RUNNER=process has every command run the wrapper, as the way out where a build does not run through the Tooling API. The Gradle is still the one the wrapper of the project names and no other. The checks of ConsoleProject.gradleWrapper are made before a connection is, a project without a wrapper or without wrapper properties that name a distribution is refused, and the Tooling API is never left to pick a version by itself. Where it would not look for the wrapper properties in the place the wrapper has them (a wrapper script that is a link, a wrapper above a project with settings of its own), and for arguments the program gradle reads itself (--no-daemon, --daemon, --foreground, -g, -p, -Dorg.gradle.*), the wrapper is run as before. Tasks and their options go to Gradle as one list of arguments in the order the commands build them, so task options reach their task and the configuration cache sees one command line per command. The build gets none of the system properties of the command line's JVM, only what -D gives it. --non-interactive gives it an input that has ended and GRAILS_NON_INTERACTIVE=true. Cancelling goes through a cancellation token; a failed build is reported as "Gradle did not succeed: the build failed", since there is no exit code to name. The refusals for what a Windows batch file cannot receive stay with the wrapper and do not apply here. The existing specs of the commands keep a script in the place of the wrapper and assert what it was asked; the test task sets GRAILS_GRADLE_RUNNER=process for them. ToolingApiBuildSpec runs the same commands against a real build with a daemon of its own and asserts what Gradle read and what each task was given. Part of apache#16348
A run-app that stays with the application runs bootRun through the Tooling API now, so there is no Gradle client process for it to record and for stop-app to end. It records itself instead: the record in .grails/run/app.json names the command line's own process and start instant and says so with "commandLine": true. While the build runs it looks for the stop request stop-app writes beside the record (app.stopping, which names the process it is for), and cancels its own build when it finds one; the Gradle daemon then ends the application. stop-app, for such a record, writes the request and waits until the run-app has removed its record or ended, then for the port, as before. It ends no process on that path. It still refuses a record whose process has ended, did not begin at the recorded instant or is not the user's, and a record it does not own; a process that does not answer within 30 seconds is left alone, the request is taken back and the command fails. Because nothing is ended, a command line may stop a run-app of its own JVM. A record of a Gradle that was started, as run-app --detach and GRAILS_GRADLE_RUNNER=process write it, is handled as it was: it has no new key, and that Gradle is ended. The readiness logic is untouched. ToolingApiRunAppSpec covers a run stopped from another process, one interrupted (SIGINT to the command line, and the thread interrupt of the interactive console), one whose command line is killed, records that are not what they say, and --detach. Part of apache#16348
The interactive console now asks its builds of one Tooling API connection for the project it is in, instead of making one per command: a build after the first begins without the cost of a connection, and far without that of starting the Gradle wrapper. The connection is closed when the console is left, replaced when a command is run for another project, and replaced when gradle-wrapper.properties has changed, as after update-wrapper. The checks of the wrapper are still made before every build, and a build that failed or was cancelled leaves the connection usable. A command line that runs one command makes its connection and closes it, so nothing of the Tooling API outlives the command. Two things follow from how the Tooling API reads input, which the first console spec with more than one line showed: it reads the stream it is given as soon as a build starts, whether the build reads or not, in a thread that stays in the read after the build has ended. In the console that thread would take the next line typed at the prompt. So a build of the console, and of a command line that runs in a caller's JVM, is given an input that has ended; only a command line that is a program of its own, whose JVM ends with the command, gives the build its input. And shell, which is a session at the terminal, runs the Gradle wrapper as a process again: a process is given the terminal itself and ends with everything that reads it. That is a third purpose in GradleBuild.of. Part of apache#16348
run-app --detach starts the Gradle wrapper through /bin/sh and relied on
"set -m" to put the build in a process group of its own, so that Ctrl-C
in the terminal of the command line is not sent to it. bash, the sh of
macOS, grants job control without a terminal. dash, the sh of Debian and
Ubuntu, does not ("can't access tty; job control turned off"), so there
the build stayed in the process group of the command line, with the
interrupt ignored as every background command of such a shell has it.
LifecycleCommandsSpec asserted a separate group unconditionally and
failed on Linux.
The start line now uses setsid when the system has the program, which
Linux does: the build is in a session, and so a process group, of its
own whatever the shell is. A background command of a shell without job
control leads no process group, which is what lets setsid become the
command without starting another process, so the recorded process is
still the build. Without setsid, as on macOS, the job control of the
shell is asked for as before, and where that is refused too the build
keeps the interrupt ignored. setsid comes first because dash, on a
terminal, takes the foreground for itself when job control is turned on.
DetachedBuildSpec holds each of the three paths to what it gives, with
the shell named and a PATH that has a setsid or none, so the dash paths
are run on macOS as well. LifecycleCommandsSpec now asserts the
guarantee, a group of its own or the interrupt ignored, and not the
outcome of bash.
Part of apache#16348
Run on a terminal, the built command line showed two faults in what Gradle wrote through the Tooling API. The output and the error output of the JVM keep what they are given for a while, each by itself, so the report of a failed build could be written before the task output that preceded it. Each of the two streams is now written through as Gradle writes to it; Gradle writes both from one thread, in the order they are to be read. With setColorOutput(true) Gradle draws its progress below what it wrote and moves the cursor up to draw it again, but writes the error output beside that console and not through it. On a terminal the last lines of a failure report, "BUILD FAILED" among them, were drawn over. The wrapper does not have this fault, since it gives Gradle the terminal itself and Gradle then writes both through one console. So a build that is run through the Tooling API writes plain text, on a terminal too: no colours and no progress display, every line once and in order. grails gradle and GRAILS_GRADLE_RUNNER=process still show Gradle as the wrapper shows it. Part of apache#16348
The guide said that every command starts the Gradle wrapper of the project and that the Tooling API is not used. "Building with Gradle" now has a section, How the grails Command Runs Gradle, with the two ways, which commands use which and why, and what differs through the Tooling API: the wrapper is checked and not started, where the wrapper is started all the same, system properties, plain output, input, the message of a failed build, cancelling, Windows, the Gradle versions it runs, and the environment variable GRAILS_GRADLE_RUNNER. The sections on run-app and stop-app say what a run-app that stays with its application records and how stop-app ends it, and how a detached build is kept from the interrupt of the terminal. The Command Line chapter names the variable and says where the JVM options of the build come from either way; the tour, interactive mode, the reference pages of run-app, stop-app, gradle and shell, upgrade notes 44, 47 and 51 and the upgrade skill follow. The threat model entry for the project directory and the Gradle wrapper said that code of a project runs only by starting its wrapper; it now names both ways, what is trusted on each, and that no Gradle is ever chosen for a project. Part of apache#16348
…esign The design said that the Tooling API is not used and gave as the reason that the dependency was not on the class path of the command line. Revision 45 records the maintainer's decision for a hybrid, what is run through the Tooling API and what as a process, what was measured, what it costs, how run-app goes by startup progress, and what the first run of continuous integration showed. Part of apache#16348
The project owner decided where each way of running Gradle is the default. A command line that runs one command starts the Gradle wrapper of the project as a process again, for every command, exactly as before the hybrid: Gradle has the terminal with its colours and its progress, reads what is typed, and a failed build is reported by its exit code. Running such a build through the Tooling API spared under 100 ms and cost that output. The interactive console runs the builds it asks for itself through the Tooling API, on one connection: there a build after the first takes about 60 ms where starting the wrapper takes about 380. gradle, shell and a run-app that leaves the application running start the wrapper in the console as everywhere, and everything that fell back to the wrapper on the Tooling API path still does. GradleBuild.of decides it by ConsoleCommands.isInteractive, which only the session of the console sets, so a command line that a caller runs in its own JVM (GrailsCli.execute) is no console and starts the wrapper. GRAILS_GRADLE_RUNNER stays the one switch and has three values: auto, or unset, is this rule; process starts the wrapper in the console too; tooling-api runs the build commands through the Tooling API on a one-shot command line as well, with the run-app record that names the command line and stop-app asking through app.stopping. Any other value runs nothing and names the three. The test task no longer forces the process runner: it sets auto, so that the environment of whoever runs the specs does not decide, and the specs that put a script in the place of the wrapper pass with it as they are, none of them needing the wrapper forced. GradleRunnerChoiceSpec holds the rule itself against a real build. The guide, the interactive mode page, the environment table, the tour, upgrade notes 44, 47 and 51, the upgrade skill, the reference pages and the threat model say what is the default where; the upgrade notes no longer claim changed output or messages for a one-shot command line. Part of apache#16348
The design recorded the hybrid of the Tooling API and the wrapper process in a line of its change log. A subsection of its own now has the rule, how it came about, what is the same on both paths, what each is better at, what was measured, what the hybrid costs and what would change it. The process was a shortcut of the implementation and is said to have been one. Revision 46 records that the default is by where a command is run: the wrapper for a command line, the Tooling API in the interactive console. Part of apache#16348
The Grails wrapper installed whatever it downloaded. A jar that was cut off stayed in the wrapper home, and since a release is never downloaded again the user was stuck until the file was found and deleted by hand. A download is now written beside the place of the jar and moved there in one step only after it matches the strongest checksum file the repository has (SHA-512, SHA-256, SHA-1, MD5) or, without one, after every entry was read against its CRC, and it opens as a jar that has its start class. Nothing is left behind when a download fails. An installed file that cannot be started is said in one line and downloaded again once. A repository whose metadata cannot be read, whose download fails or is not whole is a miss of that repository, said in one line that names it and repeats nothing of the file, and the next repository is tried. When every repository missed, the error names each with its reason, without a stack trace. A release that an overridden repository lacks gets the line a snapshot already had. A home that cannot be written to is said, with GRAILS_WRAPPER_HOME as the way out, before any repository is asked. cleanupOtherVersions is removed: it never removed a version that had a jar, and every installed version is kept on purpose, so that projects of different versions do not download again. Requests now have a connect and a read timeout, and the user and password of a repository URL are left out of every message. Two wrappers that download one version into one home each write a file of their own, so neither finds part of a jar. Part of apache#16348
The wrapper section of the guide says which repositories are asked, what a whole download is, that the checksum is from the repository the jar is from and so finds damage and not an untrustworthy repository, what is printed when a repository is passed over, that every downloaded version is kept, and what an unwritable home leads to. The release instructions say that the check of the wrapper does not replace verifying the checksums by hand. Part of apache#16348
Upgrade note 50 and the matching entry of the upgrade skill name the new lines of the wrapper and the texts they replace, for jobs that read its output, and say that a repository must not serve a stale checksum file beside the jar. Part of apache#16348
The wrapper now installs the command line it downloads only when it matches the checksum the repository serves beside it. The threat model says what that is worth: it finds a download that was cut off or corrupted, and it is no protection against a repository that is malicious, since the checksum comes from where the jar comes from. Part of apache#16348
With grails-startup-progress, run-app --detach and run-app in the interactive console went on asking the status path after the web server had taken the port, and returned only when the phase was READY, which is after BootStrap. That reversed what was documented and decided: run-app returns when the server answers. An application without the module was also asked once for the default status path, which it does not have. The command asks for the root again. An answer that names a phase other than READY is the server of the progress page, and the command waits. An answer without a phase, or with READY, is the web server of the application, and the command returns, also while BootStrap runs. The request for the default status path is gone, and so is its constant. A phase of FAILED still ends the wait at once. For that the status path is still asked, but only while the progress server has the port and only when its answer names the path: a failed start holds the port to say so only for a client that asked there within the last seconds, and releases it at once otherwise, so a client that asks for the root alone never hears of the failure. That is the one use left of the Grails-Startup-Status-Path header, which therefore stays; the section of the guide that offered it to other programs is removed, and run-app-protocol.properties no longer has the default status path. The guide, the upgrade note and the upgrade skill say again that run-app does not wait for BootStrap, and keep that a failed start is reported at once. The retry over TLS, the bounds on what is read and what stop-app says are as they were. Part of apache#16348
Revision 47: run-app returns when the web server of the application answers and does not wait for BootStrap, which the maintainer decided against revision 45; the wrapper installs a download only when it is whole; and the decisions that leave things as they are. Part of apache#16348
The specs of Forge have only ever run on Linux and macOS. Reading them for what a Windows checkout and file system change found these, none of which is behaviour of the product: The templates of the generator, of the bundled packs and of the artefact commands are packaged as they are checked out and keep their line ends when they are rendered, so with core.autocrlf a Forge built on Windows writes CRLF files and the specs that compare generated text fail. They are pinned to LF in .gitattributes, with the golden files the artefact specs compare as text, as the command descriptors already are. Specs asked for POSIX file modes without asking whether the file system has them, made paths and file names Windows does not accept (a ?, a |), took a path without a drive for an absolute one, compared a path with its escaped form in a build file and in JSON, joined expected console lines with LF where the command prints lines, and split a table of the guide on LF. One feature that drives the /bin/sh stub of gradlew is now skipped on Windows like its neighbours. Part of apache#16348
The command line starts gradlew.bat, refuses what a batch file cannot receive, checks ownership without POSIX attributes and replaces files that may be in use, and none of that has run on Windows: buildForge has Ubuntu cells only. forgeWindows is one cell, Java 21 with indy=true (the cheaper setting in the recent runs), that runs the specs of grails-forge-core, grails-forge-cli and grails-cli. test-core is left out: it builds generated applications for about 20 minutes on Ubuntu and the specs of the command line already build some with the wrapper. It is a test job, so it is gated on the CI policy and publish waits for it. The build cache may serve the compilation of the framework, but DO_NOT_CACHE_TESTS keeps the Test tasks out of it, so that a result an Ubuntu cell stored is not taken for this one. buildForge also passes -PallRepositoryViews now: the feature of PackTemplateViewsSpec that takes every view of the checkout needs nothing else, costs seconds, and never ran in CI. Part of apache#16348
The Coverage workflow had no Forge job, and Forge had no JaCoCo at all. Its modules now apply the grails-jacoco convention plugin of build-logic, but only when jacocoAggregateReport or jacocoTestReport is requested by name, the way the dependency analysis plugin is applied: an ordinary build of Forge starts no agent and writes no report. coverage-forge runs the aggregate report, less the specs of test-core whose work is in generated builds that are not measured, and uploads it with the others. For grails-gradle the workflow saw only the test JVM, about 40% of the lines, because the functional specs run their builds in TestKit daemons. On a coverage run the plugins module now hands the JaCoCo agent to GradleSpecification, which writes it into org.gradle.jvmargs of the TestKit directory, so the daemons write build/jacoco/testKit.exec and the aggregate report reads it: 85% of the lines locally. The agent goes into the daemons only; through JAVA_TOOL_OPTIONS it would reach the test JVM a second time and fail there. Part of apache#16348
🚨 TestLens detected 36 failed tests 🚨Failed Jobs without Test Failures❌ Groovy Snapshot Canary Build / Build Grails (shard 1) Test SummaryTip Inspect all details with the TestLens CLI: CI / Test Grails Forge (Windows, Java 21, indy=true) > :grails-cli:test [grails-forge]
CI / Test Grails Forge (Windows, Java 21, indy=true) > :grails-forge-cli:test [grails-forge]
🏷️ Commit: 20eb5c0 Test Failures (first 10 of 36)AddPropertyCommandSpec > the class is found from any directory of the project, and the path that is said is the one in the project (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))
ApplicationCommandsSpec > a command of the build is not run without the Gradle wrapper of the project, and not outside the project (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))ApplicationCommandsSpec > a file of the build is said to be possibly out of date when #what, wherever below the commands of the application > a file of the build is said to be possibly out of date when a command was deleted, wherever below the commands of the application (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))
ApplicationCommandsSpec > the commands of a project above that is not the user's own are not there, whatever its build lists (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))
CommandCatalogSpec > in every directory of a project the commands are those of the project, and they write in the project (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))CommandCatalogSpec > the commands are not those of a project in another repository, or of a project above the nearest one > the commands are not those of a project in another repository, or of a project above the nearest one [directory: @753c9953 delegate=org.grails.forge.cli.CommandCatalogSpec@2ce10371 owner=org.grails.forge.cli.CommandCatalogSpec@2ce10371 thisObject=org.grails.forge.cli.CommandCatalogSpec@2ce10371 resolveStrategy=0 directive=0 parameterTypes=[class java.io.File] maximumNumberOfParameters=1 bcw=inaccessible thisType=inaccessible mopPer… (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))CommandCatalogSpec > the commands of a project above that is not the user's own are not there, and a tool is told why (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))
ConsoleCommandsSpec > bug-report holds sources that are 64 directories deep, and writes nothing for a project that has deeper ones (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))
ConsoleCommandsSpec > the project is looked for above the current directory, and Gradle runs in the project (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))
CreateControllerCommandSpec > test app with controller (:grails-forge-cli:test [grails-forge] in CI / Test Grails Forge (Windows, Java 21, indy=true))Rerun Controls (first 20 of 36)Note Checks are currently running using the configuration below. Select tests to mute in this pull request: 🔲 AddPropertyCommandSpec > the class is found from any directory of the project, and the path that is said is the one in the project Reuse successful test results: 🔲 ♻️ Only rerun the tests that failed or were muted before Click the checkbox to trigger a rerun: 🔲 Rerun jobs Learn more about TestLens at testlens.app/docs. |
One generator, one command line
Implements the design of #16348 for Grails 9: Grails Forge is the only application generator,
grailsis one command line, and the Grails shell and the profiles are removed. The design, with its status and a change log of every decision, is ingrails-forge/FORGE-CLI-DESIGN.md(revision 47).Fixes #14932
Fixes #14187
Requires apache/grails-plugins-metadata#326. That pull request has the indexer record the feature pack of a plugin version in the plugin index (
grails-plugins-index.json), which is what this command line reads to know which plugins bring a feature pack. Until it is merged and the index is written again, the index has no pack entries: a feature of a plugin cannot be selected by its name with--features, andplugin-infocannot name the feature a plugin is added with. The features of Grails itself, a pack named by its coordinates or by a directory, andlist-plugins/plugin-infofor everything else do not depend on it. The two are to be merged together, the metadata change after this one.This is a draft: see "Not established" at the end.
What it does
grailsis the former Forge CLI, extended to do what the shell did in a project.grails-shell-cli, the seven profile artifacts,-t/--typeandGRAILS_PREFERRED_SHELLare gone. Gradle is run in two ways, by where a command is run. A command line that runs one command starts the Gradle wrapper of the project as a process, so Gradle has the terminal. In the interactive console the builds (compile,test-app,assemble, the commands of the application and its plugins) run through the Gradle Tooling API on one connection the console keeps: a further build there takes about 50 ms where a wrapper process takes about 350 (one machine, a warm daemon, builds with nothing to do).grails gradle ...,run-app --detachandshellare always the wrapper.GRAILS_GRADLE_RUNNER(auto,process,tooling-api) overrides the rule. The design has the reasons, in "How Gradle is run: a hybrid, and why".create-app,create-restapi,create-plugin,create-web-pluginand the newcreate-rest-api-plugin. Most features are declarative feature packs (61 bundled); a plugin can ship its own insrc/main/forge, checked bycheckForgePackandforgePackTest..grails/project.yml,.grails/forge.lock.ymland.grails/baseline/say how a project was generated.feature add [--replace]changes it later with a three-way merge;project showandproject regenerateread it;project migratewrites a record for an application that was not generated by Grails 9, including one of Grails 7 or 8 as a step of its upgrade.grailsCommandCatalog, run withrunCommand. A command that only works on files can implementStandaloneCommandand is then run without starting the application.s2-quickstart,s2-create-*,s2-create-acl-domains,init-oauth2,s2ui-*),query-tailand the scaffolding scripts are compiled commands, and the Quartz and Geb scripts are replaced bycreate-jobandcreate-functional-testof the command line.run-appand a failed start.run-app --detachandrun-appin the console return when the web server of the application answers, as before. Where the application has the startup progress page, a start that fails is reported at once and no longer waited out.test-appruns every test phase of the project through one Gradle task,testApp; a phase is selected by a switch of its name (--unit,--integration,--functionalfor afunctionalTesttask).list-pluginsandplugin-inforead the plugin index of apache/grails-plugins-metadata and a list bundled with the release.project migrate.Where to start reading
grails-doc/src/en/guide/commandLine/tour.adoc.grails-doc/src/en/guide/upgrading/upgrading90x.adoc. Note 55 maps every command line of Grails 8 (the shell, the Forge CLI, plugin commands and scripts, and how a command line is written) to Grails 9.grails-skills/upgrade/skills/grails-upgrade/SKILL.md.Please review in particular
THREAT_MODEL.md,threat-model.yaml): the entries for feature packs, the plugin index, and the project directory and its Gradle wrapper.ConsoleProject.distrustLinked).s2-quickstart: the old script generated, in every current application, a user class that hashed the password through a service that is not injected into domain instances. The command generates the password encoder listener instead, and upgrade note 54 tells people who used the script to check.Differences of the command line from Grails 8
The complete list is upgrade note 55. The ones most likely to be met:
gradle.propertiesnames Grails 9,./grailsw update-wrapperwith the wrapper committed by Grails 7 or 8 replacesgrailsw,grailsw.batandgrails-wrapper.jarby those of Grails 9, to be committed.update-wrapperis a command in Grails 9 as well. On Windows a file that is in use is written beside the old one asNAME.new, with the one step that is left.--profile,create-profileand the other profile commands; aprofiledependency fails the build;grails.profileis not read.--featuresadds to the defaults and refuses unknown names;--exclude-featuresdrops a default.grails prod war) is refused; use-Dgrails.env=after the command. A command of a plugin that is typed by its name runs fordevelopment, and warns when-Dgrails.env=follows it: name the environment withrun-command.test,run,stopare no longer other names oftest-app,run-app,stop-app;quitisexit;openand!are removed.stop-appstops only whatgrails run-appstarted.create-*commands fail when a file exists, unless--force; project templates are text and are not executed.grailsonly aftergrails gradle grailsCommandCatalog;run-commandworks without it.~/.grails/settings.groovyis not read; proxies go toGRAILS_OPTS.GRAILS_SAFE_DIRECTORY).UrlMappings, no controller, no views.cache-ehcachefeature givesorg.grails.plugins:cache-ehcache:5.0.0-RC1: 3.0.0 could not be resolved by a generated application. It is a release candidate, and only compilation is checked here.Defects found and fixed along the way
Most of these are in code that existed before this change and were found by replacing it, by writing its tests, or by a coverage audit of the branch. None is backported, although several also affect 7.0.x and 8.0.x, the first five of the Gradle plugins among them.
Grails Gradle plugins
-Dgrails.*was not passed to forked JVMs when the configuration cache entry was reused.runScriptdid not run a script any more, and its JVM did not end when the script could not be run.shelltask succeeded when the Grails shell could not be started.org.apache.grails.gradle.grails-markupcould not be applied to any build: the task re-declared three properties of its parent that were then never set.Scaffolding: the scripts and the commands did different things
The plugin had every operation twice, as a script for the shell and as a compiled command, and the two had drifted. They are one set of commands now, with the behaviour that was right in each:
generate-controller, andgenerate-allthrough it, wrote the service specification tosrc/test/groovyas a command; it is an@Integrationspecification, which the script wrote tosrc/integration-test/groovy.generate-allas a command stopped after the controllers when one of several classes was not found; the script went on.create-scaffold-controller,create-scaffold-serviceandgenerate-scaffold-allas commands refused a class without a source file in the project; the scripts took any name, which is what lets a domain class of a plugin or of another project be scaffolded.install-templatesas a command wrote relative to the working directory and returned nothing when it failed.generate-servicewithout a domain class threw aNullPointerException.Spring Security: the scripts and the commands did different things
s2-quickstartas a command never created the Requestmap class or its settings (the report in Grails 7 - s2-quickstart does not create Requestmap domain #15850: the model was never assigned; that issue is not closed here, because the command was not run in a generated application), always refused--uiOnly, and wroteauthority.className, the static rules and the filter chain undergrails.plugin.authorityinapplication.yml.s2-quickstartas a script generated, in every application withoutgormVersioningradle.properties, a user class that hashes the password through aspringSecurityServicethat is not injected into a domain instance, so the password was stored as given. The command generates the password encoder listener.--saltgenerated a listener that does not compile; it is refused.init-oauth2as a command answered only toinit-o-auth2, a name made from its class name, so the documentedrunCommand -Pargs="init-oauth2 ..."found nothing; it also failed after writing the domain class whengrails-app/confwas missing.s2ui-create-challenge-questionswrote the user class name in the wrong case (AppUserasAppuser), so the generated class did not compile, and accepted a count of 0.s2ui-overridewrote outsidegrails-app/controllersfor a package such as../../outside, and threw for missing arguments.Spring Security REST
Database migration
dbm-create-changeloganddbm-changelog-to-groovywere scripts that handed the shell's context to command classes and could not run anywhere else; they are application commands in both migration modules.Code generation
add-propertyfound its place in a class by matching lines: it could write the property into the body of a method, the constraint outside a one-line or out-of-orderconstraintsblock (the class then did not compile), a secondconstraintsblock into an abstract class, and it rewrote the line endings of the file. It takes its positions from the Groovy parser, only adds lines, and refuses what it cannot change safely.create-domain-resourcewrote a test whose one feature fails (true == false), so the tests of a new REST API application failed as soon as the command had been used.create-functional-testof the Geb plugin wroteBookSpecSpec.groovyforBookSpec, overwrote an existing file without a word, and always wrote aGebSpec, also where the application uses containers.Generated applications
cache-ehcachefeature gave a version of the plugin (3.0.0, of 2021) that no repository of a generated application has, so the build could not resolve its dependencies.views-markupfeature did not compile..gitattributesfor its wrapper scripts.GSP
Core
Holders.reset()threw aNullPointerExceptionon a class path without a Servlet environment.Grails wrapper
publishhad written (only one thatpublishToMavenLocalhad), and went on silently to the next repository whenGRAILS_REPO_URLnamed one that did not have the command line.Command line, trust
Scripts run by the build
Tests and CI
test-core) could not generate one: the Gradle API jar on its class path carries the version file of the Groovy that Gradle runs on, and Spock refused to load.BUILD SUCCESSFULwas satisfied by an earlier build. With that corrected, the last build of the interceptor specification turned out never to have passed: its controller had no action of the name its test calls.gradle.ymlranbin/grails-forge-cli, which the distribution no longer has.Verification
Run locally on macOS, JDK 21, with the test cache off: the Forge build (
grails-forge-core,grails-forge-cli,grails-forge-webwith its integration tests,grails-cli,test-core), the Grails Gradle plugins and model, the root modules this touches (grails-coreandgrails-datamapping-corecommand tests,grails-console,grails-wrapper,grails-scaffolding,grails-bootstrap,grails-geb,grails-quartz, the four Spring Security plugin modules), code style, and the guide with no AsciiDoc warnings.test-corenow compiles generated applications that together contain every bundled feature pack.Line coverage of the changed code, measured before the last round of tests was added: Forge core 90%, Forge CLI 93%, Gradle plugins 96% (with the TestKit JVMs).
Not established
forgeWindows, which the snapshot publish waits for) and a Forge coverage job; the command line has never run on Windows before, the first Windows run is expected to show defects, and they are fixed in this pull request. About 200 command-line features that drive a shell stub ofgradleware skipped on Windows. The Tooling API path in particular was run on macOS only, with the one Gradle version the build has;GRAILS_GRADLE_RUNNER=processis the way out.project migrateis tested on fixtures written for it.test-appwith an added test phase is tested on the Gradle side and on the command-line side separately, not end to end in a generated application.Interceptortrait. It could not be reproduced and is unexplained.Related
./grailsw list-pluginsgrails-plugins-metadata#128 (a source forlist-plugins) is answered by the commands reading the index.