diff --git a/harness/pstack/claude-code/skills/statechart/SKILL.md b/harness/pstack/claude-code/skills/statechart/SKILL.md index 1818f2a73..66670ebd1 100644 --- a/harness/pstack/claude-code/skills/statechart/SKILL.md +++ b/harness/pstack/claude-code/skills/statechart/SKILL.md @@ -1,36 +1,36 @@ --- name: statechart -description: "Draw what an app's user sees as a Mermaid statechart from a TLA+ model. States are gray if they work, red if still wrong, and green if fixed. Each bug TLC finds is a numbered path. Use for /statechart, \"show the PM the bug\", or to turn an agreed chart into a TLA+ skeleton." +description: "Draw how any system moves between states, such as a product flow, a protocol, or a service, as a Mermaid statechart from a TLA+ model. Each bug is a numbered path with blue borders to a state that breaks a rule. Use for /statechart, to show a bug as a path through states, or to turn an agreed chart into a TLA+ skeleton." disable-model-invocation: true --- # Statechart -The chart shows what an app's user sees. A PM reads it to decide whether that behavior is right. Label each state with what is on the screen, such as "Paying". Label each step with what the user does or sees happen, such as "tap Pay" or "the charge goes through". Keep variable and action names out of labels. The scripts call this user the reader, as in `readerActions` and reader steps. +The chart shows how a system moves between states, in the words of the person who reads it. The scripts call that person the reader, as in `readerActions` and reader steps. The reader decides whether the behavior is right. The reader can be a PM who reads a product flow, an engineer who reads a protocol or a service, or a reviewer who reads a design. Label each state with what the reader can observe or cares about, such as "Paying" in an app or "Lease held, write pending" in a lock service. Label each step with the event that moves the system, such as "tap Pay" in an app, "the request reads the count" in a service, or "the lease expires" in a lock service. Keep variable and action names out of labels. -Run `node /scripts/statechart.mjs --help` for the commands and for how to read a chart. Work in a scratch folder outside the repo. The command writes its files next to its input. +Run `node /scripts/statechart.mjs --help` for the commands and for how to read a chart. Work in a scratch folder outside the repo. The command writes its files next to its input. The first run installs TLC and Java with mise, which downloads about 305 MB and needs the network. The help's TLC section tells how to use a TLC and Java that you have. ## Pick the direction -Use `from-spec` when a TLA+ model exists or the code is concrete enough to model. Use `from-chart` when there is no model and the PM should agree on the behavior before anyone writes or changes the code. +Use `from-spec` when a TLA+ model exists, or when the code or design is concrete enough to model. Use `from-chart` when there is no model, and the people who own the behavior should agree on it before anyone builds or changes it. -If the code already has a TLA+ spec, start from a copy of that spec. Add the variables the user sees, and keep the spec's existing invariants. To audit the code against that spec, make the copy model what the code does, and pass the spec unchanged as `--against`. Check it with the config that its authors say it passes, and pass that config as `--against-config` unless it has the spec's file name. One mapping labels both models, so label states and steps only from variables that both have. In a code model or a new spec, tie every variable and action to code that a reviewer can open, and take its name from that code. Make each action one indivisible step in the code. Put the file and line of the code that checks each condition of an action in a comment on that condition, as in `\* pay.ts:120`. +If the code already has a TLA+ spec, start from a copy of that spec. Add the variables that the labels and rules need, and keep the spec's existing invariants. To audit the code against that spec, make the copy model what the code does, and pass the spec unchanged as `--against`. Check it with the config that its authors say it passes, and pass that config as `--against-config` unless it has the spec's file name. One mapping labels both models, so label states and steps from variables that both have. A label that reads a variable that only the copy has must test `"name" in v` first, and label the spec's states another way. When a model describes code, tie every variable and action to code that a reviewer can open, and take its name from that code. Make each action one indivisible step in the code. Put the file and line of the code that checks each condition of an action in a comment on that condition, as in `\* pay.ts:120`. ## Spec to chart -1. Choose the chart's states. When the PM cares about something the screen does not show, such as a second charge while the screen still says "Paying", name the state "Paying, charged twice". Start with few states, and split a state only when the split separates two bugs. -2. Write each rule the PM agrees to as one invariant, with the PM's sentence as its comment. Each rule names what the user sees on a screen or gets from a request, and the code that can change it. Refuse a rule that lacks one of them. For a rule that applies once everything settles, define `Settled == ~ENABLED System`, where `System` is the disjunction of the actions the user does not take. Report each broken rule as a bug. Never weaken a rule to make the chart pass. -3. Run `from-spec`, then fill in the mapping file it writes. Put the labels in `state` and `step`, and list the user's actions in `readerActions` by action name. When two bugs end on the same screen through different code paths, add `cause` and make it name the code path. To color the states by what the PM cares about, such as "Toast shown", return that group from `group`. The legend names the groups, and states that break a rule still get a red border. -4. If there is an open fix, model it from the code and pass that model as `--fixed`. +1. Choose the chart's states. When the reader cares about something that a label would hide, put it in the label, such as "Paying, charged twice" when the screen still says "Paying" after a second charge. Start with few states. Split a state when the split separates two bugs, or when the reader must see a step that leaves the label the same, such as a request that reads a count, because the chart does not draw such a step. +2. Write each rule that the reader agrees to as one invariant, with the reader's sentence as its comment. A rule can be about any state in the model: what a screen shows, what a request returns, a stored value, or a message between parts. When the model describes code, the rule also names the code that can change that state. If the model has no variable for what a rule is about, add the variable. Do not drop the rule. For a rule that applies once everything settles, define `Settled == ~ENABLED System`, where `System` is the disjunction of the actions that happen without the reader. For a rule that applies at a narrower point, such as after each request finishes, write a predicate for that point and make the rule hold where it is true. For a goal that the system must be able to reach, write a rule that says the goal never happens, and pass that rule with `--expect-broken`. The path that breaks it is then a way to the goal. The chart still draws the goal as a state that breaks a rule, and `stories.md` tells it as a bug, so name each goal rule in the hand-off. Report each other broken rule as a bug. Never weaken a rule to make the chart pass. +3. Run `from-spec`, then fill in the mapping file it writes. Put the labels in `state` and `step`. When the chart must separate what the reader does from what happens around the reader, list the reader's actions in `readerActions` by action name. Leave the list empty when every action counts the same. When the overview is too large to read, add `--cover`. The overview then keeps only the shortest path to each settled state, plus every bug path, and the command lists what it leaves out. When two bugs end in the same state for different reasons, add `cause` and make it name each reason. To color the states by what the reader cares about, such as "Order shipped", return that group from `group`. The legend names the groups, and in the PNG a state that breaks a rule is red in place of its group color. +4. If there is a proposed fix, model it and pass that model as `--fixed`. 5. Render each chart with `--png` and look at every PNG. Check that each numbered path reads in order. If a label reads like code, fix `state` or `step` and run the command again. -6. Replay each bug's story against the real code, as a test or a live repro, one step per model action. Drive only entry points that a user or client reaches, and assert only what they see. A test that asserts a flag that it sets, or that searches the source text, proves nothing. When a run proves part of the story and reading the code proves the rest, say which part each proves. When the test copies code that it cannot import, give the commit and line range of the copy, and list what the copy leaves out. If a bug does not reproduce, assume the model is wrong until you show otherwise. Fix the model, not the rule. +6. When the model describes code, replay each bug's story against the real code, as a test or a live repro, one step per model action. Drive only entry points that a user, a client, or another part of the system reaches, and assert only what they can observe. A test that asserts a flag that it sets, or that searches the source text, proves nothing. When a run proves part of the story and reading the code proves the rest, say which part each proves. When the test copies code that it cannot import, give the commit and line range of the copy, and list what the copy leaves out. If a bug does not reproduce, assume the model is wrong until you show otherwise. Fix the model, not the rule. For a design with no code yet, the TLC trace is the evidence, and the hand-off must say so. ## Chart to spec -1. Draw the chart the PM agrees to. Use one region for each thing that changes independently, such as the connection and the sync badge. Add one `%% never` line per rule, with the PM's sentence. Draw "the app closes" as an edge to the state the user sees next. -2. Run `from-chart`. If a rule breaks, the agreed chart has a design bug. Take the bug to the PM. -3. Add the real code's guards and variables to the spec that `from-chart` wrote, and run `from-chart` again. For each difference it prints, ask the PM whether the code is wrong or the chart needs a correction. +1. Draw the chart that the reader agrees to. Use one region for each part that changes independently, such as a device's connection and the job that the device runs. Add one `%% never` line per rule, with the reader's sentence. Draw an event that stops or restarts a part, such as "the app closes" or "the process restarts", as an edge to the state that comes next. Run `render CHART.mmd` to get a PNG of the chart for the reader. +2. Run `from-chart`. If a rule breaks, the agreed chart has a design bug. Take the bug to the people who agreed to the chart. +3. Add the real code's guards and variables to the spec that `from-chart` wrote, and run `from-chart` again. For each difference it prints, ask the reader whether the code is wrong or the chart needs a correction. ## Hand off -Give the PM the PNG and the Mermaid source of each chart. Add one line per bug with the rule it breaks, its steps as the chart numbers them, whether it reproduced, and whether the open fix fixes it. With `--against`, also say whether the spec already lists it: in the configs that `stories.md` names, in the spec's notes on what the model leaves out, or not at all. Do not report that a rule holds while the command lists conditions with no file and line. Take every count you quote from `summary.json`. Attach `stories.md` and the command that regenerates everything. Say what the model leaves out. +Give the PNG and the Mermaid source of each chart to the person who must decide. Add one line per bug with the rule it breaks, its steps as the chart numbers them, whether it reproduced, and whether the proposed fix fixes it. A broken PROPERTY has no path on the chart, so give the TLC counterexample that `stories.md` names for it. With `--against`, also say whether the spec already lists it: in the configs that `stories.md` names, in the spec's notes on what the model leaves out, or not at all. When the model describes code, do not report that a rule holds while the command lists conditions with no file and line. Take every count you quote from `summary.json`. Attach `stories.md` and the command that regenerates everything. Say what the model leaves out. diff --git a/harness/pstack/claude-code/skills/statechart/scripts/bug-states.mjs b/harness/pstack/claude-code/skills/statechart/scripts/bug-states.mjs new file mode 100644 index 000000000..3c8e9f21f --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/bug-states.mjs @@ -0,0 +1,81 @@ +import fs from "node:fs"; +import { BROKEN_MARK, parseValue, stateKey } from "./lib.mjs"; +import { SEP } from "./chart-parts.mjs"; + +export function firstBad(raw) { + const badStates = new Map(); + for (const t of raw) { + if (!badStates.has(t.invariant)) badStates.set(t.invariant, new Set()); + badStates.get(t.invariant).add(stateKey(t.steps[t.steps.length - 1].vars)); + } + const cuts = []; + const seen = new Set(); + for (const t of raw) { + const first = t.steps.findIndex((s) => badStates.get(t.invariant).has(stateKey(s.vars))); + const cut = { invariant: t.invariant, steps: t.steps.slice(0, first + 1) }; + const id = `${t.invariant}${SEP}${stateKey(cut.steps[cut.steps.length - 1].vars)}`; + if (seen.has(id)) continue; + seen.add(id); + cuts.push(cut); + } + return cuts; +} + +export const byOrder = (a, b) => + a.steps - b.steps || + a.trace.steps.length - b.trace.steps.length || + (a.id < b.id ? -1 : a.id > b.id ? 1 : 0); + +const lastKey = (t) => stateKey(t.steps[t.steps.length - 1].vars); +export const badOf = (raw, rule) => new Set(raw.filter((t) => t.invariant === rule).map(lastKey)); +export function reachableAvoiding({ graph: g, stateKeys }, bad) { + const ok = (id) => !bad.has(stateKeys.get(id)); + const next = new Map(); + for (const e of g.edges) { + if (!next.has(e.from)) next.set(e.from, []); + next.get(e.from).push(e.to); + } + const clean = new Set([...g.nodes.values()].filter((n) => n.init && ok(n.id)).map((n) => n.id)); + for (const id of clean) for (const to of next.get(id) ?? []) if (ok(to)) clean.add(to); + return clean; +} + +export const diverges = (c) => c.second?.key !== undefined; +export function readBroken(file) { + if (!file) return []; + const lines = fs.readFileSync(file, "utf8").split("\n"); + const marked = new RegExp(`^<<\\s*"${BROKEN_MARK}",`); + const out = []; + for (let i = 0; i < lines.length; i++) { + if (!marked.test(lines[i])) continue; + let text = lines[i]; + while (typeof parseValue(text) === "string" && i + 1 < lines.length) text += `\n${lines[++i]}`; + const [, vars, broken] = parseValue(text); + out.push({ key: stateKey(vars), rules: broken }); + } + return out; +} +export const badStates = (raw, broken, counts = () => true) => + new Set([ + ...raw.filter((t) => counts(t.invariant)).map(lastKey), + ...broken.filter((b) => b.rules.some(counts)).map((b) => b.key), + ]); +export function labelStatus(m, keys, bad) { + const good = reachableAvoiding(m, bad); + const kinds = new Map(); + for (const n of m.graph.nodes.values()) { + const k = keys.get(n.id); + if (!kinds.has(k)) kinds.set(k, new Set()); + kinds.get(k).add(bad.has(m.stateKeys.get(n.id)) ? "bad" : good.has(n.id) ? "good" : "after"); + } + return (k) => { + const has = kinds.get(k); + if (!has) return undefined; + if (has.has("bad")) return has.has("good") ? "mixed" : "wrong"; + return has.has("good") ? "works" : "after"; + }; +} + +export const RANK = ["works", "fixed", "after", "mixed", "wrong", "worse"]; +export const worst = (...all) => + all.filter(Boolean).reduce((a, b) => (RANK.indexOf(b) > RANK.indexOf(a) ? b : a), "works"); diff --git a/harness/pstack/claude-code/skills/statechart/scripts/chart-edges.mjs b/harness/pstack/claude-code/skills/statechart/scripts/chart-edges.mjs new file mode 100644 index 000000000..0f0db11dc --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/chart-edges.mjs @@ -0,0 +1,68 @@ +import { edgeId } from "./chart-parts.mjs"; +import { diverges } from "./bug-states.mjs"; + +export const edgesAlong = (c) => + c.path.slice(1).map((p, i) => ({ + id: edgeId(c.path[i].key, p.key), + from: c.path[i].key, + to: p.key, + label: p.label, + guard: p.guard, + })); + +export function chartEdges({ args, now, secondQ, modelEdges, classes, chosen, coverOff }) { + const edges = new Map(modelEdges); + for (const { second: s } of classes.filter(diverges)) { + if (modelEdges.has(s.edge)) continue; + if (!edges.has(s.edge)) + edges.set(s.edge, { + from: s.from, + to: s.key, + labels: new Set(), + secondOnly: true, + apart: s.apart, + }); + edges.get(s.edge).labels.add(s.overview); + } + const pathNumbers = new Map(); + (chosen ? edgesAlong(chosen) : []).forEach((step, i) => { + if (!pathNumbers.has(step.id)) pathNumbers.set(step.id, []); + const shown = step.guard ? `${step.label} [${step.guard}]` : step.label; + pathNumbers.get(step.id).push({ n: i + 1, label: step.label, shown }); + if (!edges.has(step.id)) + edges.set(step.id, { from: step.from, to: step.to, labels: new Set([step.label]) }); + }); + const bugSteps = classes.flatMap(edgesAlong); + const pathEdges = new Map(); + for (const step of bugSteps) { + if (edges.has(step.id)) continue; + if (!pathEdges.has(step.id)) + pathEdges.set(step.id, { from: step.from, to: step.to, labels: new Set() }); + pathEdges.get(step.id).labels.add(step.label); + } + for (const [id, e] of pathEdges) edges.set(id, e); + if (!args.cover || coverOff) return { edges, pathNumbers, coverEdges: null }; + + const coverEdges = new Set(bugSteps.map((step) => step.id)); + const bfs = (q, targets) => { + const parent = new Map([...q.initial].map((k) => [k, null])); + const outEdges = new Map(); + for (const [id, e] of q.edges) { + if (!outEdges.has(e.from)) outEdges.set(e.from, []); + outEdges.get(e.from).push([id, e]); + } + for (const k of parent.keys()) + for (const [id, e] of outEdges.get(k) ?? []) if (!parent.has(e.to)) parent.set(e.to, id); + for (const t of targets) { + for (let id = parent.get(t); id; id = parent.get(q.edges.get(id).from)) coverEdges.add(id); + } + }; + bfs(now, now.settled); + if (secondQ) + bfs( + secondQ, + [...secondQ.settled].filter((k) => !now.nodes.has(k)), + ); + for (const c of classes.filter(diverges)) coverEdges.add(c.second.edge); + return { edges, pathNumbers, coverEdges }; +} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/chart-parts.mjs b/harness/pstack/claude-code/skills/statechart/scripts/chart-parts.mjs new file mode 100644 index 000000000..eaa391d4e --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/chart-parts.mjs @@ -0,0 +1,21 @@ +export const SEP = "\u0000"; +export const edgeId = (from, to) => `${from}${SEP}${SEP}${to}`; + +export function guarded(guards, label) { + const words = guards?.get(label); + return words && !words.has("") ? `${label} [${[...words].sort().join(" or ")}]` : label; +} +export const addGuard = (guards, label, words) => { + if (!guards.has(label)) guards.set(label, new Set()); + guards.get(label).add(words); +}; + +export const frontMatter = (title) => [ + "---", + ...(title ? [`title: ${JSON.stringify(title)}`] : []), + "config:", + " markdownAutoWrap: false", + " state:", + " useMaxWidth: false", + "---", +]; diff --git a/harness/pstack/claude-code/skills/statechart/scripts/chart-to-tla.mjs b/harness/pstack/claude-code/skills/statechart/scripts/chart-to-tla.mjs index 7dfa337d2..911c58698 100644 --- a/harness/pstack/claude-code/skills/statechart/scripts/chart-to-tla.mjs +++ b/harness/pstack/claude-code/skills/statechart/scripts/chart-to-tla.mjs @@ -5,49 +5,20 @@ import process from "node:process"; import { parseMermaid } from "./lib.mjs"; const USAGE = "Usage: node chart-to-tla.mjs CHART.mmd --module NAME [--dir OUT]"; -const HELP = `${USAGE} - -Writes a TLA+ spec, a TLC config, and a reader mapping from a Mermaid stateDiagram-v2. -statechart.mjs from-chart runs it, then checks each INVARIANT in NAME.cfg alone. - -Each region becomes one variable whose values are the region's states. Each transition label -becomes one action. Regions that share a label move together: every region that can take it does, -and the others stay. A "%% when" line after a transition is its guard: the region can take that -transition only while the other regions are in, or with "not" are not in, the states it names. An -unlabeled transition is its own action. Each -"%% never NAME: A & B -- sentence" line becomes the invariant NAME == ~(A /\\ B). A name in a -never line is a state, or a composite that stands for every state inside it. - - --module NAME The TLA+ module name. Required. - --dir OUT The folder for the files. Default: the chart's folder. - -h, --help Print this help, and exit 0. - -Writes NAME.tla, NAME.cfg, and NAME.map.mjs into OUT, and replaces files with those names. -NAME.cfg has SPECIFICATION Spec, INVARIANT TypeOK, and one INVARIANT per never line. -NAME.map.mjs gives tlc-to-chart.mjs the reader's words: the region states joined by " | ". - -Exit codes - 0 It wrote the files. - 1 It cannot read the chart. The error says why. - 2 A bad argument.`; -const more = "Run node chart-to-tla.mjs --help for the flags."; const args = {}; for (let i = 2; i < process.argv.length; i++) { const a = process.argv[i]; - if (a === "-h" || a === "--help") { - console.log(HELP); - process.exit(0); - } else if (a === "--module") args.module = process.argv[++i]; + if (a === "--module") args.module = process.argv[++i]; else if (a === "--dir") args.dir = process.argv[++i]; else if (!a.startsWith("-") && !args.chart) args.chart = a; else { - console.error(`unknown argument ${a}. ${more}`); + console.error(`unknown argument ${a}.`); process.exit(2); } } if (!args.chart || !/^[A-Za-z][A-Za-z0-9_]*$/.test(args.module ?? "")) { - console.error(`${USAGE}\n${more}`); + console.error(USAGE); process.exit(2); } const dir = args.dir ?? path.dirname(args.chart); @@ -167,9 +138,7 @@ for (const a of actions) { if (still.length) out.push(` /\\ UNCHANGED <<${still.join(", ")}>>`); } out.push(""); -out.push( - actions.length ? `Next == ${actions.map((a) => a.name).join(" \\/ ")}` : "Next == UNCHANGED vars", -); +out.push(actions.length ? `Next == ${actions.map((a) => a.name).join(" \\/ ")}` : "Next == FALSE"); out.push(""); out.push("Spec == Init /\\ [][Next]_vars"); diff --git a/harness/pstack/claude-code/skills/statechart/scripts/compare-charts.mjs b/harness/pstack/claude-code/skills/statechart/scripts/compare-charts.mjs index 587125a9f..730698be4 100644 --- a/harness/pstack/claude-code/skills/statechart/scripts/compare-charts.mjs +++ b/harness/pstack/claude-code/skills/statechart/scripts/compare-charts.mjs @@ -1,51 +1,19 @@ #!/usr/bin/env node import fs from "node:fs"; import process from "node:process"; -import { FULL_LABELS, LEGEND_ID, compareCharts, parseMermaid } from "./lib.mjs"; +import { FULL_LABELS, LEGEND_ID, NO_LOOPS, compareCharts, parseMermaid } from "./lib.mjs"; const USAGE = "Usage: node compare-charts.mjs EXPECTED.mmd ACTUAL.mmd"; -const HELP = `${USAGE} - -Compares two Mermaid stateDiagram-v2 charts: reachable states, start states, and labeled -transitions. statechart.mjs from-chart runs it to check that the spec still draws the chart that -the reader agreed to. - -EXPECTED is the agreed chart. ACTUAL is what tlc-to-chart.mjs drew from the spec in the customer -view with --highlight 0 --full-labels --flat, without --cover, --fixed-dot, or --against-dot. Each -chart compares as the -machine that its steps reach from its start states. A chart with parallel regions compares as its -combined machine, so a guard that blocks a step in only some combinations of regions shows as a -difference. A "%% when ID, ID" or "%% when not ID, ID" line right after a transition is its guard: -the step happens only while another region is in, or not in, one of those states. The label ends -with the guard's words in [ ], which are not part of the label. A combined state is the region -states joined by " | " in region order, which is what chart-to-tla.mjs's mapping writes. A step that does not change the state is not compared, because -tlc-to-chart.mjs does not draw it. An edge label "a or b" counts as two labels. An unlabeled edge -counts only when no labeled edge joins the same two states, because tlc-to-chart.mjs draws only -the labels. - - -h, --help Print this help, and exit 0. - -Prints each difference and then "N difference(s)", or prints "same". Writes no file. - -Exit codes - 0 The charts are the same. - 1 The charts differ. - 2 A bad argument, a chart that it cannot read, or an ACTUAL chart that is not drawn for - comparison.`; -const more = "Run node compare-charts.mjs --help for the flags."; const files = []; for (const a of process.argv.slice(2)) { - if (a === "-h" || a === "--help") { - console.log(HELP); - process.exit(0); - } else if (a.startsWith("-") || files.length === 2) { - console.error(`unknown argument ${a}. ${more}`); + if (a.startsWith("-") || files.length === 2) { + console.error(`unknown argument ${a}.`); process.exit(2); } else files.push(a); } if (files.length < 2) { - console.error(`${USAGE}\n${more}`); + console.error(USAGE); process.exit(2); } const [expectedFile, actualFile] = files; @@ -93,7 +61,9 @@ if (numbered) { process.exit(2); } -const rows = compareCharts(expectedRegions, actualRegions); +const rows = compareCharts(expectedRegions, actualRegions, { + loops: !actualText.includes(NO_LOOPS), +}); let differences = 0; for (const [title, xs] of rows) { console.log(`${title}:`); diff --git a/harness/pstack/claude-code/skills/statechart/scripts/lib.mjs b/harness/pstack/claude-code/skills/statechart/scripts/lib.mjs index 49f1675bc..238f1745e 100644 --- a/harness/pstack/claude-code/skills/statechart/scripts/lib.mjs +++ b/harness/pstack/claude-code/skills/statechart/scripts/lib.mjs @@ -1,15 +1,8 @@ import fs from "node:fs"; -import process from "node:process"; +import path from "node:path"; -export function findTlc() { - const { TLA2TOOLS, JAVA } = process.env; - if (!TLA2TOOLS) - throw new Error( - "Cannot find TLC. Download tla2tools.jar from a TLA+ release:\n https://github.com/tlaplus/tlaplus/releases\nThen set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later).", - ); - if (!fs.existsSync(TLA2TOOLS)) throw new Error(`TLA2TOOLS is ${TLA2TOOLS}, which does not exist.`); - return { jar: TLA2TOOLS, java: JAVA || "java" }; -} +const tuples = new WeakSet(); +export const isTuple = (v) => tuples.has(v); export function parseValue(text) { const src = text.trim(); @@ -45,7 +38,9 @@ export function parseValue(text) { ws(); if (peek("<<")) { i += 2; - return list(">>", value); + const items = list(">>", value); + tuples.add(items); + return items; } if (peek("{")) { i++; @@ -106,7 +101,7 @@ export function parseValue(text) { } // One state as TLC prints it: "/\ x = 1" lines, or "x = 1" for one variable. -function parseState(text) { +export function parseState(text) { const lines = text.split("\n").filter((l) => l.trim() !== ""); const conjuncts = []; for (const line of lines) { @@ -154,7 +149,7 @@ export function ruleSentences(tla) { export const tlaVariables = (code) => [...code.matchAll(/\bVARIABLES?\s+(\w+(?:\s*,\s*\w+)*)/g)].flatMap((m) => m[1].split(/\s*,\s*/)); -function splitCommentsKeepingOffsets(text) { +export function splitCommentsKeepingOffsets(text) { let code = ""; let notes = ""; let block = 0; @@ -199,6 +194,9 @@ function splitCommentsKeepingOffsets(text) { } const IDENTIFIER = /(?)/g; +const QUALIFIED = /(?>|[A-Za-z_]\w*)/g; const CITATION = /[\w./-]+\.[A-Za-z]\w*:\d+|#L\d+/; const DEPTH = { "(": 1, "[": 1, "{": 1, ")": -1, "]": -1, "}": -1 }; @@ -246,24 +244,13 @@ function conjuncts(bare, from, to, lineStart) { return [first, ...cuts.map((c) => c + 2)].map((a, i) => [a, cuts[i] ?? to]); } -export function uncitedConditions(text) { - const { code, notes } = splitCommentsKeepingOffsets(text); - const bare = code.replace(/"(?:[^"\\\n]|\\.)*"/g, (s) => " ".repeat(s.length)); - const variables = new Set(tlaVariables(bare)); - const starts = [0, ...[...bare.matchAll(/\n/g)].map((m) => m.index + 1)]; - const lineOf = (at) => { - let lo = 0; - let hi = starts.length - 1; - while (lo < hi) { - const mid = (lo + hi + 1) >> 1; - if (starts[mid] <= at) lo = mid; - else hi = mid - 1; - } - return lo; - }; - const lineStart = (at) => starts[lineOf(at)]; - const lines = { bare: bare.split("\n"), notes: notes.split("\n") }; +export const blankStrings = (code) => + code.replace(/"(?:[^"\\\n]|\\.)*"/g, (s) => " ".repeat(s.length)); +export function definitions(text, instancedReads = new Map(), parameters = []) { + const { code, notes } = splitCommentsKeepingOffsets(text); + const bare = blankStrings(code); + const variables = new Set([...parameters, ...tlaVariables(bare)]); const defs = new Map(); const nextItem = /^(?![ \t\n]|\/\\|\\\/)/gm; for (const m of bare.matchAll( @@ -271,20 +258,24 @@ export function uncitedConditions(text) { )) { const from = m.index + m[0].length; nextItem.lastIndex = from; - defs.set(m[1], { from, to: nextItem.exec(bare)?.index ?? bare.length, line: lineOf(m.index) }); + defs.set(m[1], { at: m.index, from, to: nextItem.exec(bare)?.index ?? bare.length }); } + const known = (id) => defs.has(id) || instancedReads.has(id); const direct = (body) => { - const ids = body.match(IDENTIFIER) ?? []; + const plain = body.replace(UNCHANGED_CLAUSE, " ").replace(PRIMED, " "); + const qualified = (plain.match(QUALIFIED) ?? []).filter(known); + const ids = plain.replace(QUALIFIED, (q) => (known(q) ? " " : q)).match(IDENTIFIER) ?? []; return { reads: ids.filter((id) => variables.has(id)), - writes: [...body.matchAll(/(? m[1]) - .filter((id) => variables.has(id)), + writes: [...body.matchAll(PRIMED)].map((m) => m[1]).filter((id) => variables.has(id)), acts: PRIMES_OR_UNCHANGED.test(body), - calls: [...new Set(ids.filter((id) => defs.has(id)))], + calls: [...new Set([...ids.filter(known), ...qualified])], }; }; - const own = new Map([...defs].map(([name, d]) => [name, direct(bare.slice(d.from, d.to))])); + const own = new Map([ + ...[...instancedReads].map(([name, reads]) => [name, direct(reads.join(" "))]), + ...[...defs].map(([name, d]) => [name, direct(bare.slice(d.from, d.to))]), + ]); const through = (start) => { const out = { reads: new Set(start.reads), writes: new Set(start.writes), acts: start.acts }; const seen = new Set(start.calls); @@ -301,6 +292,24 @@ export function uncitedConditions(text) { } return out; }; + return { code, notes, bare, defs, direct, own, through }; +} + +export function uncitedConditions(text) { + const { code, notes, bare, defs, direct, own, through } = definitions(text); + const starts = [0, ...[...bare.matchAll(/\n/g)].map((m) => m.index + 1)]; + const lineOf = (at) => { + let lo = 0; + let hi = starts.length - 1; + while (lo < hi) { + const mid = (lo + hi + 1) >> 1; + if (starts[mid] <= at) lo = mid; + else hi = mid - 1; + } + return lo; + }; + const lineStart = (at) => starts[lineOf(at)]; + const lines = { bare: bare.split("\n"), notes: notes.split("\n") }; const cited = (from, to, { above }) => { for (let l = from; l <= to; l++) if (CITATION.test(lines.notes[l])) return true; for (let l = from - 1; above && !lines.bare[l].trim() && lines.notes[l].trim(); l--) @@ -317,7 +326,7 @@ export function uncitedConditions(text) { if (part.acts || [...part.reads].every((v) => changes.has(v))) continue; const first = lineOf(a + bare.slice(a, b).search(/\S/)); const last = lineOf(a + bare.slice(a, b).trimEnd().length - 1); - if (cited(first, last, { above: first > d.line })) continue; + if (cited(first, last, { above: first > lineOf(d.at) })) continue; found.push({ line: first + 1, action, @@ -328,6 +337,33 @@ export function uncitedConditions(text) { return found; } +const TEMPORAL = /\[\]|(?(?!>)|~>|-\+->|\\(?:AA|EE)\b|\b[WS]F_/; + +export function checkedOnlyInInitialStates(spec, property) { + const bodies = new Map(); + const modules = new Set(); + const read = (module) => { + const file = path.join(path.dirname(spec), `${module}.tla`); + if (modules.has(module) || !fs.existsSync(file)) return; + modules.add(module); + const { bare, defs } = definitions(fs.readFileSync(file, "utf8")); + for (const [name, d] of defs) + bodies.set(name, [...(bodies.get(name) ?? []), bare.slice(d.from, d.to)]); + for (const [, names] of bare.matchAll(/\b(?:EXTENDS|INSTANCE)\s+(\w+(?:\s*,\s*\w+)*)/g)) + names.split(/\s*,\s*/).forEach(read); + }; + read(path.basename(spec, ".tla")); + const seen = new Set(); + const temporal = (name) => { + if (seen.has(name)) return false; + seen.add(name); + return (bodies.get(name) ?? []).some( + (body) => TEMPORAL.test(body) || (body.match(IDENTIFIER) ?? []).some(temporal), + ); + }; + return bodies.has(property) && !temporal(property); +} + function unescapeDot(s) { let out = ""; for (let i = 0; i < s.length; i++) { @@ -350,7 +386,7 @@ export function parseDot(text) { for (const line of text.split("\n")) { let m = edgeRe.exec(line); if (m) { - const e = { from: m[1], to: m[2], action: m[3] ? unescapeDot(m[3]) : "" }; + const e = { from: m[1], to: m[2], action: m[3] ? unescapeDot(m[3]).split("(")[0] : "" }; const id = `${e.from} ${e.to} ${e.action}`; if (seen.has(id)) continue; seen.add(id); @@ -408,8 +444,8 @@ export function parseTraces(text) { const [vars, end] = stateAt(i); i = end; if (head[1] === "1") { - current = { invariant, steps: [] }; - traces.push(current); + current = invariant && { invariant, steps: [] }; + if (current) traces.push(current); } current?.steps.push({ action: head[2] === "Initial" ? null : head[2], vars }); } @@ -420,41 +456,36 @@ export function parseTraces(text) { // action property, a bare state predicate, which TLC checks only in the initial states, as a // property of the initial state, and any other formula as a temporal property. export const BROKEN_PROPERTY = - /^Error: (Invariant \S+ is violated|Action property \S+ is violated\.|Property \S+ is violated by the initial state:|Temporal properties were violated\.)/m; + /^Error: (Invariant .+? is violated|Action property .+? is violated\.|Property .+? is violated by the initial state:|Temporal properties were violated\.)/m; // With -continue, TLC prints "Model checking completed" even after broken invariants, and not -// when it stopped early. TLC can also throw a Java exception of its own while it reports a state -// that breaks a rule, and stop early. An error in the rule's formula throws one of TLC's own -// exceptions in place of java.*. +// when it stopped early. export function tlcOutputWarning(text, file) { const lines = text.split("\n"); - const named = (re) => new Set(lines.map((l) => re.exec(l)?.[1]).filter(Boolean)); - const broken = named(/^Error: Invariant (\S+) is violated/); - const failed = [...named(/^Error: Evaluating invariant (\S+) failed\./)]; - const thrown = [...text.matchAll(/^The exception was a (\S+)\n: (?:([\w.$]+):)?(.*)$/gm)].map( - ([, cls, cause, message]) => `${cause ?? cls}: ${message.trim()}`, - ); - const crashed = - failed.length > 0 && - failed.every((r) => broken.has(r)) && - thrown.length > 0 && - thrown.every((t) => t.startsWith("java.")); - const reported = crashed - ? /^Error: (Invariant \S+ is violated|The behavior up to this point is|Evaluating invariant \S+ failed\.|TLC threw an unexpected exception\.|The error occurred when TLC was evaluating the nested)/ - : /^Error: (Invariant \S+ is violated|Action property \S+ is violated\.|Property \S+ is violated by the initial state:|The behavior up to this point is|Temporal properties were violated\.|The following behavior constitutes a counter-example:)/; + const broken = [ + ...new Set( + lines.map((l) => /^Error: Invariant (\S+) is violated/.exec(l)?.[1]).filter(Boolean), + ), + ]; + const reported = + /^Error: (Invariant .+? is violated|Action property .+? is violated\.|Property .+? is violated by the initial state:|The behavior up to this point is|Temporal properties were violated\.|The following behavior constitutes a counter-example:)/; const other = lines.find((l) => l.startsWith("Error: ") && !reported.test(l)); + if (other && broken.length) { + const thrown = /^The exception was a (\S+)\n: (?:([\w$]+(?:\.[\w$]+)+):)?(.*)$/m.exec(text); + const error = thrown + ? `${thrown[2] ?? thrown[1]}: ${thrown[3].trim()}` + : other.slice(7).replace(/\.$/, ""); + const rules = joinAnd(broken); + return `TLC found that ${rules} ${broken.length === 1 ? "is" : "are"} broken, and then stopped on an error: ${error}. The paths that TLC printed before it stopped are complete, but TLC did not check every state, so the charts can miss states that break ${rules}. A smaller constant can let TLC finish.`; + } if (other) throw new Error( `${file}: TLC stopped on an error that is not a broken rule: ${other.slice(7)}`, ); - if (crashed) { - const rules = failed.join(", "); - return `TLC found that ${rules} ${failed.length === 1 ? "is" : "are"} broken, and then failed with ${thrown[0]}. TLC stopped before it checked every state, so the charts can miss states that break ${rules}. The failure is in TLC, not in the rule. Do not weaken ${rules} because of it. A smaller constant can let TLC finish.`; - } // TLC stops at the first broken temporal property, and at an initial state that breaks a // PROPERTY, and then does not print that it completed. if ( - !/^(Model checking completed\.|Error: (Temporal properties were violated\.|Property \S+ is violated by the initial state:))/m.test( + !/^(Model checking completed\.|Error: (Temporal properties were violated\.|Property .+? is violated by the initial state:))/m.test( text, ) ) @@ -508,6 +539,8 @@ export const escapeMermaidQuotes = (s) => String(s).replaceAll('"', "#quot;"); export const LEGEND_ID = "legend"; export const FULL_LABELS = "%% labels in full"; +export const NO_LOOPS = "%% self-loops left out"; +export const BROKEN_MARK = "statechart broken"; export function parseMermaid(text) { const names = new Map(); @@ -764,7 +797,7 @@ function guardHolds(guard, at) { return guard.not ? !hits.some(Boolean) : hits.every(Boolean); } -export function productOf(regions) { +export function productOf(regions, { loops = true } = {}) { const names = Object.keys(regions); const rs = Object.values(regions); const join = (parts) => parts.join(" | "); @@ -793,7 +826,7 @@ export function productOf(regions) { const queue = [...starts]; const step = (from, to, label) => { const [a, b] = [join(from), join(to)]; - if (a === b) return; + if (a === b && !loops) return; out.transitions.push({ from: a, to: b, label }); if (!seen.has(b)) { seen.add(b); @@ -826,9 +859,9 @@ export function productOf(regions) { return out; } -export function compareCharts(expectedRegions, actualRegions) { - const expected = productOf(expectedRegions); - const actual = productOf(actualRegions); +export function compareCharts(expectedRegions, actualRegions, { loops = true } = {}) { + const expected = productOf(expectedRegions, { loops }); + const actual = productOf(actualRegions, { loops }); const edgeSet = (m) => { const labeled = new Set(); const out = new Set(); @@ -873,9 +906,15 @@ export const STATUS_CLASSES = [ "classDef works fill:#f4f6f7,stroke:#566573,color:#1b2631", "classDef wrong fill:#fdedec,stroke:#c0392b,color:#7b241c", "classDef fixed fill:#e9f7ef,stroke:#1e8449,color:#145a32", + "classDef mixed fill:#fdebd0,stroke:#d35400,color:#7e5109", + "classDef after fill:#ffffff,stroke:#808b96,color:#566573", + "classDef worse fill:#f4ecf7,stroke:#7d3c98,color:#4a235a", "classDef worksPath fill:#f4f6f7,stroke:#1f3a93,stroke-width:4px,color:#1b2631", "classDef wrongPath fill:#fdedec,stroke:#1f3a93,stroke-width:4px,color:#7b241c", "classDef fixedPath fill:#e9f7ef,stroke:#1f3a93,stroke-width:4px,color:#145a32", + "classDef mixedPath fill:#fdebd0,stroke:#1f3a93,stroke-width:4px,color:#7e5109", + "classDef afterPath fill:#ffffff,stroke:#1f3a93,stroke-width:4px,color:#566573", + "classDef worsePath fill:#f4ecf7,stroke:#1f3a93,stroke-width:4px,color:#4a235a", ]; export const GROUP_PALETTE = [ @@ -910,6 +949,9 @@ export function groupColorProblem(color) { const BORDERS = { Wrong: ["stroke:#c0392b,stroke-width:4px", "#7b241c"], Fixed: ["stroke:#1e8449,stroke-width:4px", "#145a32"], + Mixed: ["stroke:#d35400,stroke-width:4px", "#7e5109"], + After: ["stroke:#808b96,stroke-width:4px", "#566573"], + Worse: ["stroke:#7d3c98,stroke-width:4px", "#4a235a"], Path: ["stroke:#1f3a93,stroke-width:4px", "#1b2631"], }; export function groupClassDef(base, fill, border) { diff --git a/harness/pstack/claude-code/skills/statechart/scripts/map-view.mjs b/harness/pstack/claude-code/skills/statechart/scripts/map-view.mjs new file mode 100644 index 000000000..e7e844b6d --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/map-view.mjs @@ -0,0 +1,92 @@ +import { isRecord, isTuple } from "./lib.mjs"; + +export function mapView(value, at = "v") { + if (isRecord(value)) + return Object.fromEntries(Object.entries(value).map(([k, x]) => [k, mapView(x, `${at}.${k}`)])); + if (!Array.isArray(value)) return value; + if (isTuple(value)) + return tupleView( + value.map((x, i) => mapView(x, `${at}[${i + 1}]`)), + at, + ); + return arrayView(value.map((x, i) => mapView(x, `${at}[${i}]`))); +} +const arrayView = (items) => + Object.defineProperties(items, { + flat: { value: (depth = 1) => flatten(items, depth) }, + flatMap: { value: (f, self) => flatten(items.map(f, self), 1) }, + }); +const views = new WeakMap(); +const itemAt = (k) => (typeof k === "string" && /^[1-9]\d*$/.test(k) ? k - 1 : -1); +const flatten = (xs, depth) => + Array.prototype.flatMap.call(xs, (x) => + depth >= 1 && Array.isArray(x) ? flatten(views.get(x)?.items ?? x, depth - 1) : [x], + ); +function tupleView(items, at) { + const view = new Proxy(items, { + get(_, k) { + if (k === "0") + throw new Error( + `the mapping reads ${at}[0], but a tuple reads from 1, as in TLA+. Read ${at}[1] for its first item.`, + ); + const i = itemAt(k); + if (i >= 0) return items[i]; + if (k === "toJSON") return () => items; + if (k === "flat") return (depth = 1) => flatten(items, depth); + if (k === "flatMap") return (f, self) => flatten(items.map(f, self), 1); + const x = Reflect.get(items, k); + if (typeof x !== "function") return x; + return x.bind(x === Object.prototype[k] ? view : items); + }, + set(_, k, x) { + if (k === "0") + throw new Error( + `the mapping writes ${at}[0], but a tuple reads from 1, as in TLA+. Write ${at}[1] for its first item.`, + ); + const i = itemAt(k); + if (i < 0) return Reflect.set(items, k, x); + items[i] = x; + return true; + }, + has(_, k) { + if (k === "0") + throw new Error( + `the mapping looks for index 0 in ${at}, with "in" or with an array method such as concat or flat, but a tuple reads from 1, as in TLA+. Use [...${at}] for an array that reads from 0.`, + ); + const i = itemAt(k); + return i < 0 ? Reflect.has(items, k) : i in items; + }, + ownKeys: () => + Reflect.ownKeys(items).map((k) => + typeof k === "string" && /^(0|[1-9]\d*)$/.test(k) ? String(+k + 1) : k, + ), + getOwnPropertyDescriptor(_, k) { + if (k === "0") return undefined; + const i = itemAt(k); + return Reflect.getOwnPropertyDescriptor(items, i < 0 ? k : i); + }, + }); + views.set(view, { items, at }); + return view; +} +const nativeClone = globalThis.structuredClone; +function cloneView(x, seen) { + if (x === null || typeof x !== "object") return nativeClone(x); + if (seen.has(x)) return seen.get(x); + const view = views.get(x); + if (!view && !Array.isArray(x) && Object.getPrototypeOf(x) !== Object.prototype) + return nativeClone(x); + const out = view || Array.isArray(x) ? [] : {}; + const copy = view ? tupleView(out, view.at) : Array.isArray(x) ? arrayView(out) : out; + seen.set(x, copy); + for (const [k, item] of Object.entries(view?.items ?? x)) out[k] = cloneView(item, seen); + return copy; +} +globalThis.structuredClone = (value, options) => { + try { + return nativeClone(value, options); + } catch (e) { + if (e?.name !== "DataCloneError") throw e; + return cloneView(value, new Map()); + } +}; diff --git a/harness/pstack/claude-code/skills/statechart/scripts/model/OutputFolder.cfg b/harness/pstack/claude-code/skills/statechart/scripts/model/OutputFolder.cfg deleted file mode 100644 index 68fd715a8..000000000 --- a/harness/pstack/claude-code/skills/statechart/scripts/model/OutputFolder.cfg +++ /dev/null @@ -1,6 +0,0 @@ -SPECIFICATION Spec -INVARIANT ChartsOneSpec -INVARIANT KeepsYourFiles -INVARIANT NeverLosesChart -INVARIANT ExitTwoLeavesFolder -INVARIANT OverlapNeverRefuses diff --git a/harness/pstack/claude-code/skills/statechart/scripts/model/OutputFolder.map.mjs b/harness/pstack/claude-code/skills/statechart/scripts/model/OutputFolder.map.mjs deleted file mode 100644 index 9618338f6..000000000 --- a/harness/pstack/claude-code/skills/statechart/scripts/model/OutputFolder.map.mjs +++ /dev/null @@ -1,106 +0,0 @@ -const QUIET = ["idle", "exited", "stopped", "orphanTlc"]; -const going = (v) => v.pc.filter((p) => !QUIET.includes(p)).length; -const changed = (before, after) => - after.pc.findIndex( - (p, i) => p !== before.pc[i] || after.aside[i].exists !== before.aside[i].exists, - ) + 1; - -export const readerActions = ["Start", "Edit", "Note", "Stop", "KillNode"]; - -function context(v) { - if (v.pc.every((p) => p === "idle")) return "Before any run"; - if (going(v) === 2) return "Two runs going"; - return going(v) === 1 ? "One run going" : "Done"; -} - -const SPECS = { 1: "the spec", 2: "the edited spec", "1,2": "both specs" }; - -function contents(v) { - const kept = v.aside.filter((a) => a.exists); - const aside = kept.some((a) => a.notes) - ? ", your notes moved aside" - : kept.some((a) => a.by) - ? ", the old chart moved aside" - : ""; - const parts = [ - v.out.by && `chart of ${SPECS[v.out.read.join()]}`, - v.out.notes && "your notes", - ].filter(Boolean); - if (!v.out.exists) return `no folder${aside}`; - return `${parts.join(" and ") || "empty folder"}${aside}`; -} - -export function state(v) { - return `${context(v)}: ${contents(v)}`; -} - -export function group(v) { - if (v.pc.every((p) => p === "idle")) return "Not started"; - return going(v) ? "Running" : "Finished"; -} - -export function step(action, before, after) { - const r = changed(before, after); - switch (action) { - case "Start": { - const where = going(before) ? " in a second shell" : ""; - const how = after.inputOk[r - 1] ? "" : " with a typo"; - return `you start run ${r}${where}${how}`; - } - case "Refuse": - return before.inputOk[r - 1] - ? `run ${r} says the folder has a file that it did not write` - : `run ${r} says the input is wrong`; - case "Tlc": - return `run ${r}'s TLC reads the spec`; - case "Draw": - return `run ${r} draws the chart in its own folder`; - case "Stale": - return `run ${r} says the spec changed while it read it`; - case "Foreign": - return `run ${r} says the folder has a file that it did not write`; - case "Swap": - case "Checked": - return ""; - case "MoveIn": - return `run ${r} moves its folder into place`; - case "MoveAside": - return `run ${r} moves the old folder aside`; - case "Restore": - return `run ${r} puts the folder with your notes back and stops`; - case "Supersede": - return `run ${r} finds another run's folder in place and deletes the old folder`; - case "KeepAside": - return `run ${r} keeps the folder with your notes aside and says where`; - case "RenameFails": - if (after.aside[r - 1].exists) - return `run ${r} cannot rename a folder and says where the old folder is`; - return after.out.exists && !before.out.exists - ? `run ${r} cannot rename a folder, puts the old folder back, and stops` - : `run ${r} cannot rename a folder and stops`; - case "Clean": - return `run ${r} deletes the old folder and finishes`; - case "Edit": - return "you edit the spec"; - case "Note": - return "you save notes in the folder"; - case "Stop": - return `you stop run ${r} with Ctrl-C`; - case "KillNode": - return `you kill run ${r}'s Node, and its TLC keeps running`; - case "OrphanEnd": - return `run ${r}'s orphaned TLC finishes`; - default: - return action; - } -} - -export function cause(v, facts, invariant) { - return { - ChartsOneSpec: "the run read the spec before and after an edit, and still wrote its chart", - KeepsYourFiles: "a run deleted a folder that had your notes in it", - NeverLosesChart: "a run deleted the last chart before another chart was in place", - ExitTwoLeavesFolder: "a run wrote the folder and then said it could not finish", - OverlapNeverRefuses: "a run stopped only because another run overlapped it", - }[invariant]; -} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/model/OutputFolder.tla b/harness/pstack/claude-code/skills/statechart/scripts/model/OutputFolder.tla deleted file mode 100644 index 7765babcb..000000000 --- a/harness/pstack/claude-code/skills/statechart/scripts/model/OutputFolder.tla +++ /dev/null @@ -1,176 +0,0 @@ ----- MODULE OutputFolder ---- -EXTENDS Naturals - -Runs == {1, 2} -NoRun == 0 -NoFolder == [exists |-> FALSE, by |-> NoRun, read |-> {}, notes |-> FALSE] - -VARIABLES pc, inputOk, snap, read, out, aside, spec, noted, wrote, broke, exitCode - -vars == <> - -Init == - /\ pc = [r \in Runs |-> "idle"] - /\ inputOk = [r \in Runs |-> TRUE] - /\ snap = [r \in Runs |-> 0] - /\ read = [r \in Runs |-> {}] - /\ out = NoFolder - /\ aside = [r \in Runs |-> NoFolder] - /\ spec = 1 - /\ noted = FALSE - /\ wrote = [r \in Runs |-> FALSE] - /\ broke = [r \in Runs |-> FALSE] - /\ exitCode = [r \in Runs |-> "none"] - -Set(f, r, v) == [f EXCEPT ![r] = v] -Going(r) == pc[r] \notin {"idle", "exited", "stopped", "orphanTlc"} -Go(r, to) == pc' = Set(pc, r, to) -Exit(r, code) == pc' = Set(pc, r, "exited") /\ exitCode' = Set(exitCode, r, code) - -Start(r) == - /\ pc[r] = "idle" - /\ r = 2 => pc[1] # "idle" - /\ \E ok \in BOOLEAN : inputOk' = Set(inputOk, r, ok) - /\ snap' = Set(snap, r, spec) - /\ Go(r, "check") - /\ UNCHANGED <> - -Refuse(r) == - /\ pc[r] = "check" - /\ ~inputOk[r] \/ out.notes - /\ Exit(r, "2") - /\ UNCHANGED <> - -Tlc(r) == - /\ \/ pc[r] = "check" /\ inputOk[r] /\ ~out.notes - \/ pc[r] = "tlc" - /\ read' = Set(read, r, read[r] \cup {spec}) - /\ Go(r, IF pc[r] = "check" THEN "tlc" ELSE "draw") - /\ UNCHANGED <> - -Draw(r) == - /\ pc[r] = "draw" - /\ Go(r, "publish") - /\ UNCHANGED <> - -Stale(r) == - /\ pc[r] = "publish" /\ spec # snap[r] - /\ Exit(r, "2") - /\ UNCHANGED <> - -Foreign(r) == - /\ pc[r] = "publish" /\ spec = snap[r] /\ out.notes - /\ Exit(r, "2") - /\ UNCHANGED <> - -Swap(r) == - /\ pc[r] = "publish" /\ spec = snap[r] /\ ~out.notes - /\ Go(r, "swap") - /\ UNCHANGED <> - -MoveIn(r) == - /\ pc[r] = "swap" /\ ~out.exists - /\ out' = [exists |-> TRUE, by |-> r, read |-> read[r], notes |-> FALSE] - /\ wrote' = Set(wrote, r, TRUE) - /\ Go(r, "clean") - /\ UNCHANGED <> - -MoveAside(r) == - /\ pc[r] = "swap" /\ out.exists /\ ~aside[r].exists - /\ aside' = Set(aside, r, out) - /\ out' = NoFolder - /\ Go(r, "aside") - /\ UNCHANGED <> - -Checked(r) == - /\ pc[r] = "aside" /\ ~aside[r].notes - /\ Go(r, "swap") - /\ UNCHANGED <> - -Supersede(r) == - /\ pc[r] = "swap" /\ out.exists /\ aside[r].exists /\ ~out.notes - /\ aside' = Set(aside, r, NoFolder) - /\ UNCHANGED <> - -Restore(r) == - /\ pc[r] = "aside" /\ aside[r].notes /\ ~out.exists - /\ out' = aside[r] - /\ aside' = Set(aside, r, NoFolder) - /\ Exit(r, "2") - /\ UNCHANGED <> - -KeepAside(r) == - /\ \/ pc[r] = "aside" /\ aside[r].notes /\ out.exists - \/ pc[r] = "swap" /\ aside[r].exists /\ out.notes - /\ Exit(r, "2") - /\ UNCHANGED <> - -RenameFails(r) == - /\ pc[r] \in {"swap", "aside"} - /\ \/ /\ aside[r].exists /\ ~out.exists - /\ out' = aside[r] - /\ aside' = Set(aside, r, NoFolder) - \/ UNCHANGED <> - /\ broke' = Set(broke, r, TRUE) - /\ Exit(r, "2") - /\ UNCHANGED <> - -Clean(r) == - /\ pc[r] = "clean" - /\ aside' = Set(aside, r, NoFolder) - /\ Exit(r, "0") - /\ UNCHANGED <> - -Stop(r) == - /\ Going(r) - /\ Go(r, "stopped") - /\ UNCHANGED <> - -KillNode(r) == - /\ pc[r] = "tlc" - /\ Go(r, "orphanTlc") - /\ UNCHANGED <> - -OrphanEnd(r) == - /\ pc[r] = "orphanTlc" - /\ Go(r, "stopped") - /\ UNCHANGED <> - -Edit == - /\ spec = 1 - /\ spec' = 2 - /\ UNCHANGED <> - -Note == - /\ ~noted - /\ noted' = TRUE - /\ out' = [out EXCEPT !.exists = TRUE, !.notes = TRUE] - /\ UNCHANGED <> - -Next == - \/ Edit \/ Note - \/ \E r \in Runs : - Start(r) \/ Refuse(r) \/ Tlc(r) \/ Draw(r) \/ Stale(r) \/ Foreign(r) \/ Swap(r) - \/ MoveIn(r) \/ MoveAside(r) \/ Checked(r) \/ Supersede(r) \/ Restore(r) - \/ KeepAside(r) \/ RenameFails(r) \/ Clean(r) \/ Stop(r) \/ KillNode(r) \/ OrphanEnd(r) - -Spec == Init /\ [][Next]_vars - -\* The folder never holds a chart drawn from two versions of the spec. -ChartsOneSpec == out.read \in {{}, {1}, {2}} - -\* A file that you put in the folder is never deleted. -KeepsYourFiles == noted => out.notes \/ \E r \in Runs : aside[r].notes - -\* Once a run writes a chart, the folder or a folder that a run moved aside always holds one. -NeverLosesChart == - (\E r \in Runs : wrote[r]) => out.by # NoRun \/ \E r \in Runs : aside[r].by # NoRun - -\* A run that exits 2 did not write the folder. -ExitTwoLeavesFolder == \A r \in Runs : exitCode[r] = "2" => ~wrote[r] - -\* A run exits 2 only for a wrong input, an edited spec, a file that it did not write, or a failed -\* rename, never because another run overlaps it. -OverlapNeverRefuses == - \A r \in Runs : exitCode[r] = "2" => ~inputOk[r] \/ spec # snap[r] \/ noted \/ broke[r] -==== diff --git a/harness/pstack/claude-code/skills/statechart/scripts/property-loop.mjs b/harness/pstack/claude-code/skills/statechart/scripts/property-loop.mjs new file mode 100644 index 000000000..47bf0372f --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/property-loop.mjs @@ -0,0 +1,60 @@ +import fs from "node:fs"; +import { joinAnd, parseDot, parseState, stateKey } from "./lib.mjs"; +import { mapView } from "./map-view.mjs"; + +const TEMPORAL_BREAK = "Error: Temporal properties were violated."; + +function lasso(tlcOutput) { + const start = tlcOutput.lastIndexOf(TEMPORAL_BREAK); + if (start < 0) return null; + const lines = tlcOutput.slice(start).split("\n"); + const states = []; + for (let i = 0; i < lines.length; i++) { + const back = /^Back to state (\d+): /.exec(lines[i]); + if (back) return { loop: states.slice(Number(back[1]) - 1), stutters: false }; + const head = /^State \d+: (<|Stuttering)/.exec(lines[i]); + if (!head) continue; + if (head[1] === "Stuttering") return { loop: states.slice(-1), stutters: true }; + const body = []; + while (i + 1 < lines.length && lines[i + 1].trim() !== "") body.push(lines[++i]); + states.push(parseState(body.join("\n"))); + } + return null; +} + +const collapseRepeatsAroundLoop = (names) => { + const runs = names.filter((name, i) => name !== names[i - 1]); + return runs.length > 1 && runs.at(-1) === runs[0] ? runs.slice(0, -1) : runs; +}; + +export function repeatsIn(tlcOutput, nameOf) { + const found = lasso(tlcOutput); + return ( + found && { states: collapseRepeatsAroundLoop(found.loop.map(nameOf)), stutters: found.stutters } + ); +} + +function enabledByState({ nodes, edges, loops }) { + const byId = new Map([...nodes.keys()].map((id) => [id, new Set()])); + for (const e of [...edges, ...loops]) byId.get(e.from).add(e.action); + return new Map([...nodes.values()].map((n) => [stateKey(n.vars), byId.get(n.id)])); +} + +export function readerStateNamer(mapping, dotFile) { + const readerActions = new Set(mapping.readerActions ?? []); + let enabledAt; + return (vars) => { + enabledAt ??= enabledByState(parseDot(fs.readFileSync(dotFile, "utf8"))); + const enabled = enabledAt.get(stateKey(vars)) ?? new Set(); + const settled = [...enabled].every((a) => readerActions.has(a)); + return String(mapping.state(mapView(vars), { enabled, settled })); + }; +} + +export function loopSentence(repeats) { + if (!repeats) return null; + const quoted = repeats.states.map((name) => `"${name}"`); + if (repeats.stutters) return `The run stops in ${quoted[0]} forever.`; + if (quoted.length === 1) return `The run stays in ${quoted[0]} forever.`; + return `The run loops through ${joinAnd(quoted)} forever.`; +} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/read-check.mjs b/harness/pstack/claude-code/skills/statechart/scripts/read-check.mjs new file mode 100644 index 000000000..18c52b6d1 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/read-check.mjs @@ -0,0 +1,79 @@ +import { isRecord } from "./lib.mjs"; +import { SEP } from "./chart-parts.mjs"; + +const kindOf = (v) => (isRecord(v) ? "record" : Array.isArray(v) ? "list" : typeof v); +function shapeOf(graph, traces) { + const shape = new Map(); + const walk = (o, path) => { + for (const [k, v] of Object.entries(o)) { + const p = [...path, k]; + const id = p.join(SEP); + if (!shape.has(id)) shape.set(id, { path: p, kinds: new Set() }); + shape.get(id).kinds.add(kindOf(v)); + if (isRecord(v)) walk(v, p); + } + }; + for (const n of graph.nodes.values()) walk(n.vars, []); + for (const t of traces) for (const s of t.steps) walk(s.vars, []); + return shape; +} +const disjoint = (a, b) => ![...b.kinds].some((k) => a.kinds.has(k)); +const foreign = (a, b) => [...b.kinds].some((k) => !a.kinds.has(k)); +function lacking(a, b) { + const out = []; + for (const [id, x] of a) { + const within = out.some((p) => id.startsWith(`${p.join(SEP)}${SEP}`)); + if (!within && (!b.has(id) || disjoint(x, b.get(id)))) out.push(x.path); + } + return out; +} +const unreadOf = (a, b) => + [...a].flatMap(([id, x]) => + !b.has(id) || foreign(x, b.get(id)) ? [{ ...x, somewhere: b.has(id) }] : [], + ); +function guard(vars, unread, label, at = "v") { + if (!unread.length || !isRecord(vars)) return vars; + const here = new Map(); + const deeper = new Map(); + for (const { path, ...x } of unread) { + const [k, ...rest] = path; + if (!rest.length) here.set(k, x); + else deeper.set(k, [...(deeper.get(k) ?? []), { ...x, path: rest }]); + } + const check = (target, k, listing) => { + const x = here.get(k); + const absent = !Object.hasOwn(target, k); + if (!x || (absent ? x.somewhere : x.kinds.has(kindOf(target[k])))) return; + const name = `${at}.${k}`.slice(2); + const has = absent ? `no ${name}` : `${name} as a value of another type`; + const does = listing ? `lists the fields of ${at}` : `reads ${at}.${k}`; + const hint = listing + ? ", and read the fields it needs by name" + : absent + ? `, or test "${k}" in ${at} first` + : ""; + throw new Error( + `${label} has ${has}, and the mapping ${does} on its states. Label states and steps only from what both models have${hint}.`, + ); + }; + return new Proxy(vars, { + get(target, k) { + check(target, k, false); + return deeper.has(k) ? guard(target[k], deeper.get(k), label, `${at}.${k}`) : target[k]; + }, + ownKeys(target) { + for (const k of here.keys()) check(target, k, true); + return Reflect.ownKeys(target); + }, + }); +} + +export function readCheck({ graph, rawTraces, secondGraph, rawSecondTraces, L }) { + const [a, b] = [shapeOf(graph, rawTraces), shapeOf(secondGraph, rawSecondTraces)]; + const unread = unreadOf(a, b); + return { + lacks: lacking(a, b), + extra: lacking(b, a), + guard: (vars) => guard(vars, unread, L), + }; +} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/regions-chart.mjs b/harness/pstack/claude-code/skills/statechart/scripts/regions-chart.mjs new file mode 100644 index 000000000..95d746021 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/regions-chart.mjs @@ -0,0 +1,307 @@ +import fs from "node:fs"; +import { + NO_LOOPS, + compareCharts, + escapeMermaidQuotes, + parseMermaid, + productOf, + wrap, +} from "./lib.mjs"; +import { SEP, addGuard, frontMatter, guarded } from "./chart-parts.mjs"; +import { RANK, diverges, worst } from "./bug-states.mjs"; +import { edgesAlong } from "./chart-edges.mjs"; + +export function regionsChart({ + states, + starts, + edges, + loops, + classes, + proofPath, + allPath, + mapping, + status, + LEGEND, + groupOf, + drawnState, + groupCounts, + head, +}) { + const norm = (s) => s.replaceAll(";", ",").replace(/\s+/g, " ").trim(); + const split = (k) => { + const at = k.indexOf(": "); + return at > 0 && at + 2 < k.length ? [norm(k.slice(0, at)), norm(k.slice(at + 2))] : null; + }; + const nodes = [...states].sort(); + if (!nodes.every(split)) return null; + const steps = []; + for (const e of edges.values()) { + if (e.labels.size === 0) return { why: "a step has no label" }; + for (const l of e.labels) + steps.push({ + from: split(e.from), + to: split(e.to), + label: norm(l), + words: e.guards?.get(l) ?? new Set([""]), + }); + } + const bar = (p) => p.join(" | "); + const flat = { + Main: { + states: new Set(nodes.map((k) => bar(split(k)))), + initial: new Set(starts.map((k) => bar(split(k)))), + transitions: steps.map((s) => ({ from: bar(s.from), to: bar(s.to), label: s.label })), + }, + }; + const ids = new Map([...flat.Main.states].map((s, i) => [s, `p${i}`])); + const proof = [ + ...frontMatter(), + "stateDiagram-v2", + ...(loops ? [] : [` ${NO_LOOPS}`]), + ...[...ids].map(([s, id]) => ` state "${escapeMermaidQuotes(s)}" as ${id}`), + ...[...flat.Main.initial].map((s) => ` [*] --> ${ids.get(s)}`), + ...steps.map((s) => ` ${ids.get(bar(s.from))} --> ${ids.get(bar(s.to))} : ${s.label}`), + ]; + if (proofPath) fs.writeFileSync(proofPath, proof.join("\n") + "\n"); + + const beside = [new Map(), new Map()]; + for (const k of nodes) { + const p = split(k); + for (const i of [0, 1]) { + if (!beside[i].has(p[i])) beside[i].set(p[i], new Set()); + beside[i].get(p[i]).add(p[1 - i]); + } + } + const size = beside[0].size + beside[1].size; + if (size >= nodes.length) + return { why: `${size} region states are not fewer than the ${nodes.length} flat states` }; + + const regionStatus = [new Map(), new Map()]; + for (const k of nodes) { + const p = split(k); + for (const i of [0, 1]) { + const prev = regionStatus[i].get(p[i]); + if (!prev || RANK.indexOf(status(k)) < RANK.indexOf(prev)) + regionStatus[i].set(p[i], status(k)); + } + } + for (const k of nodes) { + const p = split(k); + const shown = worst(...[0, 1].map((i) => regionStatus[i].get(p[i]))); + if (shown !== status(k)) + return { + why: `"${k}" is "${LEGEND[status(k)][1]}", and its two region states would show "${LEGEND[shown][1]}"`, + }; + } + + const moves = [new Map(), new Map()]; + for (const s of steps) + for (const i of [0, 1]) { + const id = [s.from[i], s.label, s.to[i]].join(SEP); + if (!moves[i].has(id)) + moves[i].set(id, { + from: s.from[i], + label: s.label, + to: s.to[i], + beside: new Set(), + guards: new Map(), + }); + moves[i].get(id).beside.add(s.from[1 - i]); + for (const w of s.words) addGuard(moves[i].get(id).guards, s.label, norm(w)); + } + for (const m of moves) + for (const [id, x] of m) + if ( + x.from === x.to && + ![...m.values()].some((y) => y.from === x.from && y.label === x.label && y.to !== x.from) + ) + m.delete(id); + const rid = beside.map((m, i) => new Map([...m.keys()].sort().map((s, j) => [s, `r${i}s${j}`]))); + const regionGroup = [new Map(), new Map()]; + for (const k of nodes) { + const p = split(k); + for (const i of [0, 1]) { + const prev = regionGroup[i].get(p[i]); + regionGroup[i].set(p[i], prev === undefined || prev === groupOf(k) ? groupOf(k) : ""); + } + } + const drawn = rid.map((m, i) => + [...m.keys()].map((s) => drawnState(regionGroup[i].get(s), regionStatus[i].get(s))), + ); + const draw = (only) => { + const out = head(drawn.flat()); + out.push(` state "At the same time" as regions {`); + let count = 0; + for (const i of [0, 1]) { + if (i) out.push(" --"); + const name = mapping.regionNames?.[i]; + const pad = name ? " " : " "; + if (name) out.push(` state "${escapeMermaidQuotes(wrap(name))}" as region${i} {`); + for (const s of new Set(starts.map((k) => split(k)[i]))) + out.push(`${pad}[*] --> ${rid[i].get(s)}`); + for (const [s, id] of rid[i]) + out.push(`${pad}state "${escapeMermaidQuotes(wrap(s))}" as ${id}`); + const shownMoves = [...moves[i]] + .filter(([id]) => !only || only[i].has(id)) + .map(([, m]) => m) + .sort((a, b) => a.from.localeCompare(b.from) || a.label.localeCompare(b.label)); + for (const m of shownMoves) { + const all = beside[i].get(m.from); + const not = [...all].filter((s) => !m.beside.has(s)).sort(); + const yes = [...m.beside].sort(); + const unless = not.length > 0 && not.length < yes.length; + const guard = not.length === 0 ? null : unless ? not : yes; + const worded = guarded(m.guards, m.label); + const text = + worded !== m.label || !guard + ? worded + : `${m.label} [${unless ? "unless" : "when"} ${guard.join(" or ")}]`; + out.push(`${pad}${rid[i].get(m.from)} --> ${rid[i].get(m.to)} : ${wrap(text, 30)}`); + if (guard) + out.push( + `${pad}%% when ${unless ? "not " : ""}${guard.map((s) => rid[1 - i].get(s)).join(", ")}`, + ); + count++; + } + if (name) out.push(" }"); + } + out.push(" }"); + for (const i of [0, 1]) + [...rid[i].values()].forEach((id, j) => out.push(` class ${id} ${drawn[i][j].cls}`)); + return { mermaid: out.join("\n") + "\n", count }; + }; + + const all = draw(null); + const differences = compareCharts(parseMermaid(all.mermaid).regions, flat, { loops }); + if (differences.length) { + const n = differences.reduce((sum, [, xs]) => sum + xs.length, 0); + return { + why: `the regions do not combine into the flat chart, with ${n} difference(s), such as ${differences[0][0]}: ${differences[0][1][0]}`, + }; + } + if (allPath) fs.writeFileSync(allPath, all.mermaid); + + const parts = new Map(nodes.map((k) => [bar(split(k)), split(k)])); + const key = (from, label, to) => `${from} --> ${to} : ${label}`; + const real = new Set(steps.map((s) => key(bar(s.from), s.label, bar(s.to)))); + const next = new Map(); + for (const s of steps) { + if (!next.has(bar(s.from))) next.set(bar(s.from), []); + next.get(bar(s.from)).push(s); + } + const keep = [new Set(), new Set()]; + const movesOf = (s) => + [0, 1] + .filter((i) => s.from[i] !== s.to[i]) + .map((i) => [i, [s.from[i], s.label, s.to[i]].join(SEP)]); + const price = (s) => movesOf(s).filter(([i, id]) => !keep[i].has(id)).length * nodes.length + 1; + const wanted = []; + const want = (s) => { + wanted.push(s); + for (const [i, id] of movesOf(s)) keep[i].add(id); + }; + const stepsOf = (e, said) => { + const labels = [...e.labels].map(norm); + return (labels.includes(said) ? [said] : labels).map((label) => ({ + from: split(e.from), + to: split(e.to), + label, + })); + }; + for (const c of classes) { + for (const step of edgesAlong(c)) { + const e = edges.get(step.id); + if (e) stepsOf(e, norm(step.label)).forEach(want); + } + if (diverges(c)) stepsOf(edges.get(c.second.edge), "").forEach(want); + } + const cheapest = () => { + const cost = new Map(starts.map((k) => [bar(split(k)), 0])); + const parent = new Map(); + const open = new Set(cost.keys()); + while (open.size) { + const k = [...open].reduce((a, b) => (cost.get(b) < cost.get(a) ? b : a)); + open.delete(k); + for (const s of next.get(k) ?? []) { + const c = cost.get(k) + price(s); + if (c < (cost.get(bar(s.to)) ?? Infinity)) { + cost.set(bar(s.to), c); + parent.set(bar(s.to), s); + open.add(bar(s.to)); + } + } + } + return { cost, parent }; + }; + const targets = [ + ...[0, 1].flatMap((i) => + [...beside[i].keys()].sort().map((r) => ({ at: (k) => parts.get(k)[i] === r })), + ), + ...[...new Set(steps.map((s) => s.label))] + .filter((l) => !wanted.some((s) => s.label === l)) + .sort() + .map((l) => ({ ends: (s) => s.label === l })), + ]; + while (targets.length) { + const { cost, parent } = cheapest(); + let best = null; + targets.forEach((t, n) => { + const ways = t.at + ? [...cost.keys()].filter(t.at).map((k) => ({ k, c: cost.get(k), last: null })) + : steps + .filter(t.ends) + .filter((s) => cost.has(bar(s.from))) + .map((s) => ({ k: bar(s.from), c: cost.get(bar(s.from)) + price(s), last: s })); + for (const w of ways) if (!best || w.c < best.c) best = { ...w, n }; + }); + if (!best) break; + targets.splice(best.n, 1); + const path = best.last ? [best.last] : []; + for (let s = parent.get(best.k); s; s = parent.get(bar(s.from))) path.push(s); + path.forEach(want); + } + const takes = (x, label) => + [0, 1].flatMap((i) => + [...moves[i]] + .filter(([, m]) => m.from === x[i] && m.label === label && m.beside.has(x[1 - i])) + .map(([id]) => [i, id]), + ); + let scoped; + let made; + for (;;) { + scoped = draw(keep); + made = productOf(parseMermaid(scoped.mermaid).regions, { loops }); + const madeKeys = new Set(made.transitions.map((t) => key(t.from, t.label, t.to))); + const redo = [ + ...made.transitions + .filter((t) => parts.has(t.from) && !real.has(key(t.from, t.label, t.to))) + .map((t) => [parts.get(t.from), t.label]), + ...wanted + .filter((s) => made.states.has(bar(s.from))) + .filter((s) => !madeKeys.has(key(bar(s.from), s.label, bar(s.to)))) + .map((s) => [s.from, s.label]), + ]; + const before = keep[0].size + keep[1].size; + for (const [x, label] of redo) for (const [i, id] of takes(x, label)) keep[i].add(id); + if (keep[0].size + keep[1].size === before) break; + } + const sound = + [...made.states].every((s) => parts.has(s)) && + made.transitions.every((t) => real.has(key(t.from, t.label, t.to))); + if (!sound) scoped = all; + + return { + mermaid: scoped.mermaid, + states: [beside[0].size, beside[1].size], + transitions: scoped.count, + all: all.count, + flat: { states: nodes.length, transitions: steps.length }, + status: Object.fromEntries( + RANK.filter((s) => LEGEND[s]).map((s) => [ + s, + regionStatus.reduce((n, m) => n + [...m.values()].filter((x) => x === s).length, 0), + ]), + ), + groups: groupCounts(drawn.flat()), + }; +} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/render.mjs b/harness/pstack/claude-code/skills/statechart/scripts/render.mjs new file mode 100644 index 000000000..4c90c5853 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/render.mjs @@ -0,0 +1,531 @@ +import { execFileSync, spawnSync } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import process from "node:process"; +import { pathToFileURL } from "node:url"; +import { GROUP_PALETTE, PLAIN_FILL } from "./lib.mjs"; + +export const PACKAGES = ["pele@1.8.0", "@resvg/resvg-js@2.6.2"]; +export const CACHE_NAME = PACKAGES.map((p) => p.replace(/^@[^/]+\//, "").replace("@", "-")).join( + "-", +); +const ENTRY = 'export { render } from "pele";\nexport { Resvg } from "@resvg/resvg-js";\n'; + +export const cacheDir = () => + path.join( + process.env.XDG_CACHE_HOME || path.join(os.homedir(), ".cache"), + "statechart", + CACHE_NAME, + ); + +function install(dir) { + fs.mkdirSync(path.dirname(dir), { recursive: true }); + const tmp = fs.mkdtempSync(`${dir}-`); + try { + fs.writeFileSync(path.join(tmp, "package.json"), '{ "private": true }\n'); + console.error(`Installing ${PACKAGES.join(" and ")} with npm in ${dir}. This happens once.`); + const r = spawnSync( + "npm", + ["install", "--no-audit", "--no-fund", "--ignore-scripts", "--save-exact", ...PACKAGES], + { + cwd: tmp, + encoding: "utf8", + env: { + ...process.env, + PATH: `${path.dirname(process.execPath)}${path.delimiter}${process.env.PATH ?? ""}`, + }, + shell: process.platform === "win32", + }, + ); + if (r.status !== 0) { + const said = + r.error?.message ?? + (`${r.stderr ?? ""}${r.stdout ?? ""}`.trim().split("\n").slice(-20).join("\n") || + `npm install exited ${r.status}, with no output`); + throw new Error( + `cannot install ${PACKAGES.join(" and ")} with npm:\n${said.replace(/^/gm, " ")}`, + ); + } + fs.writeFileSync(path.join(tmp, "entry.mjs"), ENTRY); + try { + fs.renameSync(tmp, dir); + } catch (e) { + if (!fs.existsSync(path.join(dir, "entry.mjs"))) throw e; + } + } finally { + fs.rmSync(tmp, { recursive: true, force: true }); + } +} + +export async function loadRenderer() { + const dir = cacheDir(); + const entry = path.join(dir, "entry.mjs"); + if (!fs.existsSync(entry)) install(dir); + try { + return await import(pathToFileURL(entry).href); + } catch (e) { + throw new Error( + `cannot load Pele and resvg from ${dir}: ${e.message}\nDelete that folder to install them again.`, + ); + } +} + +const STYLE = { line: 1.5, text: 0.4, arrow: 1.25, zoom: 2 }; +const MAX_PIXELS = 100_000_000; +const MARK = "2.5px"; +const FONT = "system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif"; + +export const THEMES = { + light: { + background: "#ffffff", + stateFill: "#fcfcfc", + stateBorder: "#c9c9c9", + stateText: "#1c1c1c", + edge: "#808080", + edgeLabel: "#656565", + region: { fill: "#f1f1f1", border: "#c9c9c9", text: "#1c1c1c" }, + groups: ["#c3e0ff", "#f0d3bf", "#e3d5ff", "#f0dcbf", "#bfe3df", "#e5d9ce", "#ffcbe6"], + groupBorder: "#c9c9c9", + wrong: { fill: "#ffecee", border: "#e72e3b", text: "#cd2834" }, + fixed: { fill: "#eaf9f2", border: "#10ab61" }, + mixed: { fill: "#fff3e6", border: "#e8730c", text: "#b85a06" }, + after: { fill: "#ffffff", border: "#9a9a9a", text: "#656565" }, + worse: { fill: "#f7edfd", border: "#9b35d6", text: "#8424b8" }, + path: "#1680f3", + }, + dark: { + background: "#1f1f1f", + stateFill: "#121212", + stateBorder: "#4e4e4e", + stateText: "#f5f5f5", + edge: "#939393", + edgeLabel: "#acacac", + region: { fill: "#2c2c2c", border: "#4e4e4e", text: "#f5f5f5" }, + groups: ["#1b3857", "#573117", "#3c2e57", "#573d17", "#174641", "#3d3127", "#57233e"], + groupBorder: "#4e4e4e", + wrong: { fill: "#2e2021", border: "#cf2b35", text: "#ff6463" }, + fixed: { fill: "#1e2a25", border: "#0e9f5a" }, + mixed: { fill: "#2e2619", border: "#d0700f", text: "#ffa257" }, + after: { fill: "#1f1f1f", border: "#6e6e6e", text: "#acacac" }, + worse: { fill: "#29202e", border: "#9a3fcf", text: "#d38cff" }, + path: "#116dd2", + }, + classic: { + background: "#ffffff", + stateFill: PLAIN_FILL, + stateBorder: "#566573", + stateText: "#1b2631", + edge: "#1f1f1f", + edgeLabel: "#1f1f1f", + region: { fill: "#e6e6e6", border: "#8a8a8a", text: "#1f1f1f" }, + groups: null, + groupBorder: null, + wrong: { fill: "#fadbd8", border: "#922b21", text: "#7b241c" }, + fixed: { fill: "#d5f5e3", border: "#196f3d" }, + mixed: { fill: "#fdebd0", border: "#d35400", text: "#7e5109" }, + after: { fill: "#ffffff", border: "#808b96", text: "#566573" }, + worse: { fill: "#ebdef0", border: "#6c3483", text: "#4a235a" }, + path: "#1f3a93", + }, +}; + +const hex6 = (c) => { + const m = /^#([0-9a-f]{3}|[0-9a-f]{6})$/i.exec(c?.trim() ?? ""); + if (!m) return null; + const h = m[1].toLowerCase(); + return `#${h.length === 3 ? [...h].map((x) => x + x).join("") : h}`; +}; +const rgb = (hex) => [1, 3, 5].map((i) => parseInt(hex.slice(i, i + 2), 16)); +const lab = (hex) => { + const [r, g, b] = rgb(hex).map((c) => + (c /= 255) <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4, + ); + const f = (t) => (t > 0.008856 ? Math.cbrt(t) : 7.787 * t + 16 / 116); + const [x, y, z] = [ + (0.4124 * r + 0.3576 * g + 0.1805 * b) / 0.95047, + 0.2126 * r + 0.7152 * g + 0.0722 * b, + (0.0193 * r + 0.1192 * g + 0.9505 * b) / 1.08883, + ].map(f); + return [116 * y - 16, 500 * (x - y), 200 * (y - z)]; +}; +const deltaE = (a, b) => Math.hypot(...lab(a).map((v, i) => v - lab(b)[i])); + +const STATE_CLASS = + /^(plain|group\d+|works|wrong|fixed|mixed|after|worse)(Wrong|Fixed|Mixed|After|Worse|Path)?$/; +const CLASS_DEF = /^(\s*classDef\s+)(\w+)\s+(\S+?)\s*$/gm; +const rawFill = (body) => /(?:^|,)fill:([^,]+)/.exec(body)?.[1]; +const STATUSES = ["wrong", "fixed", "mixed", "after", "worse"]; + +function markOf(name) { + const m = STATE_CLASS.exec(name); + if (!m) return null; + const [, base, mark = ""] = m; + const own = mark.toLowerCase(); + if (STATUSES.includes(own)) return own; + return STATUSES.includes(base) ? base : own; +} + +function paletteSlots(fills) { + const slot = new Map(); + const taken = new Set(); + for (const f of fills) { + const k = GROUP_PALETTE.indexOf(f); + if (k >= 0) { + slot.set(f, k); + taken.add(k); + } + } + const all = GROUP_PALETTE.map((_, k) => k); + for (const f of fills) { + if (f === PLAIN_FILL || slot.has(f)) continue; + const free = all.filter((k) => !taken.has(k)); + const [k] = (free.length ? free : all).sort( + (a, b) => deltaE(f, GROUP_PALETTE[a]) - deltaE(f, GROUP_PALETTE[b]), + ); + slot.set(f, k); + taken.add(k); + } + return slot; +} + +const marked = ({ fill, border, text }) => + `fill:${fill},stroke:${border},stroke-width:${MARK},color:${text}`; +const STYLES = { + wrong: ({ wrong }) => marked(wrong), + mixed: ({ mixed }) => marked(mixed), + after: ({ after }) => marked(after), + worse: ({ worse }) => marked(worse), + fixed: ({ fixed: f, stateText }) => + `fill:${f.fill},stroke:${f.border},stroke-width:${MARK},color:${stateText}`, + path: ({ path, stateText }, fill) => + `${fill ? `fill:${fill},` : ""}stroke:${path},stroke-width:${MARK},color:${stateText}`, + "": ({ groups, stateFill, stateBorder, groupBorder, stateText }, fill) => + groups && + `${fill ? `fill:${fill},` : ""}stroke:${fill === stateFill ? stateBorder : groupBorder},color:${stateText}`, +}; + +function themed(src, theme) { + const own = new Set(); + for (const [, , name, body] of src.matchAll(CLASS_DEF)) { + const fill = hex6(rawFill(body)); + if (fill && (markOf(name) === "" || markOf(name) === "path")) own.add(fill); + } + const slot = paletteSlots([...own]); + const fillFor = (body) => { + const raw = rawFill(body); + const fill = hex6(raw); + if (!theme.groups || !fill) return raw; + if (fill === PLAIN_FILL) return theme.stateFill; + return slot.has(fill) ? theme.groups[slot.get(fill)] : raw; + }; + return src.replace(CLASS_DEF, (line, head, name, body) => { + const mark = markOf(name); + const style = mark === null ? null : STYLES[mark](theme, fillFor(body)); + return style ? `${head}${name} ${style}` : line; + }); +} + +function withRootDirection(src) { + const lines = src.split("\n"); + let at = -1; + let depth = 0; + let regions = false; + let direction = false; + for (const [i, line] of lines.entries()) { + const code = line + .replace(/"[^"]*"/g, '""') + .replace(/:.*$/, "") + .trim(); + if (at < 0) { + if (/^stateDiagram/.test(code)) at = i; + continue; + } + if (code.startsWith("%%")) continue; + if (code === "--") regions = true; + if (depth === 0 && /^direction\s/.test(code)) direction = true; + depth += (code.match(/\{/g) ?? []).length - (code.match(/\}/g) ?? []).length; + } + if (at < 0 || !regions || direction) return src; + lines.splice(at + 1, 0, " direction LR"); + return lines.join("\n"); +} + +const SYSTEM_KEYWORD = /^(system-ui|-apple-system|BlinkMacSystemFont|ui-sans-serif)$/i; + +export function fcMatch(pattern) { + try { + const [file, name] = execFileSync("fc-match", ["-f", "%{file}\n%{family[0]}", pattern], { + encoding: "utf8", + stdio: ["ignore", "pipe", "ignore"], + }).split("\n"); + return file ? { file, name } : null; + } catch { + return null; + } +} + +let systemFont; +export function systemFamily() { + if (systemFont === undefined) systemFont = fcMatch("system-ui")?.name ?? null; + return systemFont; +} + +function pngStack(sys) { + if (!sys) return FONT; + const names = FONT.split(",") + .map((f) => f.trim().replace(/^'|'$/g, "")) + .map((f) => (SYSTEM_KEYWORD.test(f) ? sys : f)); + return [...new Set(names)].map((f) => (/^[\w-]+$/.test(f) ? f : `'${f}'`)).join(", "); +} + +const xml = (s) => s.replace(/&/g, "&").replace(//g, ">"); + +const measurers = new Map(); +export function fontMeasurer(Resvg, family) { + if (measurers.has(family)) return measurers.get(family); + const ink = (face, text, size, style) => { + const svg = + `${xml(text)}`; + const font = face.file + ? { loadSystemFonts: false, fontFiles: [face.file] } + : { loadSystemFonts: true }; + return new Resvg(svg, { font }).getBBox()?.width; + }; + const draws = (face, style) => ink(face, "x", 16, style) > 0; + const matchFace = (style) => { + const name = style & 4 ? "monospace" : family; + return fcMatch(`${name}${style & 1 ? ":bold" : ""}${style & 2 ? ":italic" : ""}`) ?? { name }; + }; + const regular = matchFace(0); + if (!draws(regular, 0)) { + measurers.set(family, null); + return null; + } + const faces = new Map([[0, regular]]); + const faceFor = (style) => { + if (!faces.has(style)) { + const face = matchFace(style); + faces.set(style, draws(face, style) ? face : regular); + } + return faces.get(style); + }; + const widths = new Map(); + const bars = new Map(); + const measurer = { + width(text, size, style = 0) { + const key = `${size}|${style}|${text}`; + if (!widths.has(key)) { + const face = faceFor(style); + const bar = `${size}|${style}`; + if (!bars.has(bar)) bars.set(bar, ink(face, "||", size, style)); + widths.set(key, text ? ink(face, `|${text}|`, size, style) - bars.get(bar) : 0); + } + return widths.get(key); + }, + }; + measurers.set(family, measurer); + return measurer; +} + +const STATE_PAD = 16; +const DEFAULT_WRAP = 200; +const labelLines = (src) => + [...src.matchAll(/^\s*state "([^"]+)" as|^[^%\n]*-->[^:\n]*:\s*([^\n]+)$/gm)] + .flatMap((m) => (m[1] ?? m[2]).split(//)) + .map((l) => l.trim()) + .filter((l) => l && !/["\\]/.test(l)); + +function widthRatio(render, src, measurer) { + const lines = [...new Set(labelLines(src))]; + if (!lines.length) return 1; + const probe = `---\nconfig:\n state:\n wrappingWidth: 2000\n---\nstateDiagram-v2\n${lines.map((l, i) => ` state "${l}" as p${i}`).join("\n")}\n`; + const builtin = new Map( + [...render(probe).svg.matchAll(/data-id="p(\d+)"[^>]*>]*\swidth="([\d.]+)"/g)].map( + ([, i, w]) => [lines[+i], +w - 2 * STATE_PAD], + ), + ); + const sum = (f) => lines.reduce((s, l) => s + f(l), 0); + return ( + sum((l) => measurer.width(l, 16, 0)) / sum((l) => builtin.get(l) ?? measurer.width(l, 16, 0)) + ); +} + +function setWrap(src, width) { + const fm = /^---\n([\s\S]*?\n)---\n/.exec(src); + if (!fm) return `---\nconfig:\n state:\n wrappingWidth: ${width}\n---\n${src}`; + let body = fm[1]; + if (/^ {4}wrappingWidth:.*$/m.test(body)) + body = body.replace(/^ {4}wrappingWidth:.*$/m, ` wrappingWidth: ${width}`); + else if (/^ {2}state:\n/m.test(body)) + body = body.replace(/^( {2}state:\n)/m, `$1 wrappingWidth: ${width}\n`); + else if (/^config:\n/m.test(body)) + body = body.replace(/^(config:\n)/m, `$1 state:\n wrappingWidth: ${width}\n`); + else body += `config:\n state:\n wrappingWidth: ${width}\n`; + return `---\n${body}---\n${src.slice(fm[0].length)}`; +} +const wrapOf = (src) => + +(/^---\n[\s\S]*?^ {4}wrappingWidth:\s*(\d+)[\s\S]*?^---$/m.exec(src)?.[1] ?? DEFAULT_WRAP); + +const toHex = (c) => `#${c.map((v) => Math.round(v).toString(16).padStart(2, "0")).join("")}`; + +function splitTop(s) { + const out = []; + let depth = 0; + let cur = ""; + for (const ch of s) { + if (ch === "(") depth++; + if (ch === ")") depth--; + if (ch === "," && depth === 0) { + out.push(cur); + cur = ""; + } else cur += ch; + } + out.push(cur); + return out.map((x) => x.trim()); +} + +function resolveCss(svg) { + const defs = new Map(); + for (const m of svg.matchAll(/(--[\w-]+)\s*:\s*([^;"}]+)/g)) + if (!defs.has(m[1])) defs.set(m[1], m[2].trim()); + const resolve = (value) => { + let changed = true; + while (changed) { + changed = false; + value = value.replace( + /var\((--[\w-]+)\s*(?:,\s*((?:[^()]|\([^()]*\))*))?\)/g, + (_, name, fallback) => { + changed = true; + const d = defs.get(name); + if (d !== undefined && d !== `var(${name})`) return resolve(d); + return fallback !== undefined ? fallback : "currentColor"; + }, + ); + value = value.replace(/color-mix\(\s*in srgb\s*,((?:[^()]|\([^()]*\))*)\)/g, (_, args) => { + changed = true; + const [a, b] = splitTop(args); + const pa = /(.*?)\s+([\d.]+)%$/.exec(a); + const ca = hex6(pa ? pa[1] : a); + const cb = hex6(b.replace(/\s+[\d.]+%$/, "")); + const p = pa ? Number(pa[2]) / 100 : 0.5; + if (!ca || !cb) return pa ? pa[1] : a; + const [x, y] = [rgb(ca), rgb(cb)]; + return toHex(x.map((v, i) => v * p + y[i] * (1 - p))); + }); + } + return value; + }; + for (const [k, v] of defs) defs.set(k, resolve(v)); + return resolve(svg.replace(/@import[^;]+;/g, "")); +} + +function themeSvg(svg, theme) { + const vars = + `--pele-bg:${theme.background};--pele-fg:${theme.region.text};--pele-muted:${theme.edgeLabel};--pele-line:${theme.edge};` + + `--pele-surface:${theme.stateFill};--pele-surface-alt:${theme.region.fill};--pele-border:${theme.region.border};`; + return svg + .replace(/(]*?\sstyle=")/, `$1${vars}`) + .replace( + /(]*>]*\/>.pele-edges,.pele-dividers{stroke-width:${STYLE.line}px}` + + `.pele-node rect,.pele-node path,.pele-node circle,.pele-cluster rect,.pele-cluster path,.pele-note rect,.pele-note path{stroke-width:${STYLE.line}px}`; +const round = (v) => +v.toFixed(2); + +function heavier(svg) { + svg = resolveCss(svg) + .replace(/style="([^"]*)"/g, (_, css) => `style="${css.replace(/--[\w-]+:[^;]*;?/g, "")}"`) + .replace(/font:\s*([\d.]+px)\s+([^;"]+);/, "font-size:$1;font-family:$2;"); + const rootFill = /]*\sfill="([^"]+)"/.exec(svg)?.[1] ?? "#1f1f1f"; + return svg + .replace(/(]*>)/, `$1${LINES}`) + .replace(/]*)>/g, (_, attrs) => { + const fill = + /style="[^"]*fill:\s*([^;"]+)/.exec(attrs)?.[1] ?? + /\sfill="([^"]+)"/.exec(attrs)?.[1] ?? + rootFill; + return ``; + }) + .replace( + /(class="pele-marker" d=")M([\d.-]+),([\d.-]+)((?:L[\d.-]+,[\d.-]+)+)Z/g, + (_, head, tx, ty, rest) => { + const [x0, y0] = [+tx, +ty]; + const points = [...rest.matchAll(/L([\d.-]+),([\d.-]+)/g)].map( + ([, x, y]) => + `L${round(x0 + (x - x0) * STYLE.arrow)},${round(y0 + (y - y0) * STYLE.arrow)}`, + ); + return `${head}M${tx},${ty}${points.join("")}Z`; + }, + ); +} + +function maskBands(svg) { + const pad = STYLE.line / 2; + // The holes set their own fill because mask content inherits the root fill, which is not black. + const holes = [ + ...svg.matchAll( + /]*>]*\/>`, + ) + .join(""); + if (!holes) return svg; + return svg.replace( + /${holes}$& mask="url(#pele-bands)"`, + ); +} + +export function renderChart(lib, src, { theme = THEMES.light, systemFont = systemFamily() } = {}) { + let text = withRootDirection(themed(src, theme)); + const measurer = systemFont ? fontMeasurer(lib.Resvg, systemFont) : undefined; + let wrap = wrapOf(text); + if (measurer) { + wrap = Math.ceil(wrap * Math.max(1, widthRatio(lib.render, text, measurer))); + text = setWrap(text, wrap); + } + const r = lib.render(text, { fontFamily: FONT, measurer }); + return { + svg: heavier(maskBands(themeSvg(r.svg, theme))), + width: r.width, + height: r.height, + wrap, + }; +} + +export function toPng( + lib, + { svg, width, height }, + { theme = THEMES.light, systemFont = systemFamily() } = {}, +) { + return new lib.Resvg(svg.split(FONT).join(pngStack(systemFont)), { + background: theme.background, + fitTo: { mode: "zoom", value: Math.min(STYLE.zoom, Math.sqrt(MAX_PIXELS / (width * height))) }, + font: systemFont + ? { loadSystemFonts: true, defaultFontFamily: systemFont, sansSerifFamily: systemFont } + : { loadSystemFonts: true }, + }) + .render() + .asPng(); +} + +export function renderPng(lib, src, options = {}) { + try { + return toPng(lib, renderChart(lib, src, options), options); + } catch (e) { + if (e.code !== "limit") throw e; + throw new Error( + `${e.message} The chart is too big for one PNG, and the .mmd file still has all of it. A chart with fewer states fits. Run node statechart.mjs from-spec SPEC.tla --cover --png to draw a smaller overview of the spec.`, + ); + } +} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/rule-book.mjs b/harness/pstack/claude-code/skills/statechart/scripts/rule-book.mjs new file mode 100644 index 000000000..3ea545228 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/rule-book.mjs @@ -0,0 +1,84 @@ +import fs from "node:fs"; +import path from "node:path"; +import { + blankStrings, + definitions, + ruleSentences, + splitCommentsKeepingOffsets, + tlaVariables, +} from "./lib.mjs"; + +const EXTENDS_STATEMENT = /\bEXTENDS\s+(\w+(?:\s*,\s*\w+)*)/g; +const CONSTANT_STATEMENT = /\bCONSTANTS?\s+(\w+(?:\([^)]*\))?(?:\s*,\s*\w+(?:\([^)]*\))?)*)/g; +const DECLARED_NAME = /(\w+)(?:\([^)]*\))?/g; +const LOCAL_DEFINITION = /^LOCAL\s/; +const INSTANCE_STATEMENT = + /^(LOCAL[ \t]+)?(?:([A-Za-z_]\w*)[ \t]*==\s*)?INSTANCE\s+(\w+)((?:[^\n]|\n(?=\s))*)/gm; +const SUBSTITUTION = /(\w+)\s*<-([\s\S]*?)(?=\w+\s*<-|$)/g; +const ALIAS = /^\s*([A-Za-z_]\w*(?:![A-Za-z_]\w*)*)\s*$/; + +export function ruleBook(spec) { + const modules = new Map(); + const moduleOf = (file) => { + if (modules.has(file)) return modules.get(file); + const module = { rules: new Map(), parameters: [] }; + modules.set(file, module); + const text = fs.readFileSync(file, "utf8"); + const code = blankStrings(splitCommentsKeepingOffsets(text).code); + const fileOf = (name) => path.join(path.dirname(file), `${name}.tla`); + const inherited = new Map(); + const instancedReads = new Map(); + const localImports = new Set(); + const take = (imported, { name, local, substitutions = new Map() }) => { + for (const [rule, entry] of moduleOf(imported).rules) { + if (entry.local) continue; + const key = name ? `${name}!${rule}` : rule; + if (local) localImports.add(key); + if (entry.sentence) inherited.set(key, entry.sentence); + instancedReads.set( + key, + [...entry.reads].map((v) => substitutions.get(v) ?? v), + ); + } + }; + const parameters = []; + for (const [, names] of code.matchAll(EXTENDS_STATEMENT)) + for (const extended of names + .split(/\s*,\s*/) + .map(fileOf) + .filter(fs.existsSync)) { + parameters.push(...moduleOf(extended).parameters); + take(extended, {}); + } + for (const [, names] of code.matchAll(CONSTANT_STATEMENT)) + parameters.push(...[...names.matchAll(DECLARED_NAME)].map(([, constant]) => constant)); + for (const [, local, name, instanced, clause] of code.matchAll(INSTANCE_STATEMENT)) + if (fs.existsSync(fileOf(instanced))) + take(fileOf(instanced), { + name, + local, + substitutions: new Map([...clause.matchAll(SUBSTITUTION)].map(([, v, e]) => [v, e])), + }); + const written = ruleSentences(text); + const { bare, defs, own, through } = definitions(text, instancedReads, parameters); + module.parameters.push(...parameters, ...tlaVariables(bare)); + const sentenceOf = (rule, seen = new Set()) => { + if (written.has(rule)) return written.get(rule); + if (seen.has(rule)) return undefined; + const d = defs.get(rule); + if (!d) return inherited.get(rule); + const alias = ALIAS.exec(bare.slice(d.from, d.to))?.[1]; + return alias ? sentenceOf(alias, seen.add(rule)) : undefined; + }; + for (const [rule, entry] of own) { + const d = defs.get(rule); + module.rules.set(rule, { + sentence: sentenceOf(rule), + reads: through(entry).reads, + local: d ? LOCAL_DEFINITION.test(bare.slice(d.at, d.from)) : localImports.has(rule), + }); + } + return module; + }; + return moduleOf(path.resolve(spec)).rules; +} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/second-replay.mjs b/harness/pstack/claude-code/skills/statechart/scripts/second-replay.mjs new file mode 100644 index 000000000..1dba571c2 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/second-replay.mjs @@ -0,0 +1,122 @@ +import { isRecord, stateKey } from "./lib.mjs"; +import { SEP, edgeId } from "./chart-parts.mjs"; +import { badOf } from "./bug-states.mjs"; + +const holder = (o, p) => p.slice(0, -1).reduce((x, k) => (isRecord(x?.[k]) ? x[k] : null), o); +function omit(vars, paths) { + if (!paths.length) return vars; + const out = structuredClone(vars); + for (const p of paths) { + const o = holder(out, p); + if (o) delete o[p.at(-1)]; + } + return out; +} + +export function replaySecond( + trace, + { main, lacks, extra, second, hiddenStep, readerPath, mainActions, secondRules, rawSecondTraces }, +) { + if (!second) return null; + const shared = (vars) => stateKey(omit(vars, lacks)); + const sharedOf = (n) => stateKey(omit(n.vars, extra)); + const out = new Map(); + for (const e of [...second.graph.edges, ...second.graph.loops]) { + if (!out.has(e.from)) out.set(e.from, []); + out.get(e.from).push(e); + } + const actions = new Set([...second.graph.edges, ...second.graph.loops].map((e) => e.action)); + const nodeOf = (id) => second.graph.nodes.get(id); + const screen = new Map( + [...second.graph.nodes.values()].map((n) => [n.id, second.customer(n.vars)]), + ); + const words = (e) => + second.step(e.action, nodeOf(e.from).vars, nodeOf(e.to).vars, "customer") || e.action; + const settle = (ids) => { + const all = new Set(ids); + for (const id of all) + for (const e of out.get(id) ?? []) { + const sameLabel = screen.get(e.to) === screen.get(id); + if ( + !mainActions.has(e.action) && + hiddenStep(trace.invariant, sameLabel, e.action, nodeOf(id).vars, nodeOf(e.to).vars) + ) + all.add(e.to); + } + return all; + }; + const seen = readerPath(trace, main, main.customer, "customer"); + const shownAt = new Map(seen.map((p, j) => [p.modelStep, j])); + let here = settle( + [...second.graph.nodes.values()] + .filter((n) => n.init && screen.get(n.id) === seen[0].key) + .map((n) => n.id), + ); + if (here.size === 0) return { reason: "starts in another state" }; + const passed = new Set(here); + let j = 0; + for (let i = 1; i < trace.steps.length; i++) { + const { action } = trace.steps[i]; + const hidden = !shownAt.has(i); + if (!hidden) j = shownAt.get(i); + const want = seen[j]; + const [before, after] = [trace.steps[i - 1].vars, trace.steps[i].vars]; + const known = actions.has(action); + if (!known && hidden) continue; + const moved = shared(before) !== shared(after); + const steps = (keep) => [...here].flatMap((id) => (out.get(id) ?? []).filter(keep)); + const tries = steps((e) => + known ? e.action === action : moved && sharedOf(nodeOf(e.to)) === shared(after), + ); + if (hidden && tries.length === 0) continue; + const looksSame = (e) => screen.get(e.to) === want.key && (hidden || words(e) === want.label); + const matched = known ? tries.filter(looksSame) : tries; + const same = matched.length || hidden ? matched : steps(looksSame); + if (same.length) { + here = settle(same.map((e) => e.to)); + for (const id of here) passed.add(id); + continue; + } + const outcomes = new Map(); + for (const e of tries) { + const id = `${screen.get(e.to)}${SEP}${words(e)}`; + outcomes.set(id, [...(outcomes.get(id) ?? []), e]); + } + const taken = [...outcomes.values()].sort((a, b) => b.length - a.length)[0]?.[0]; + const from = main.key(before); + const said = hidden ? main.step(action, before, after, "customer") || action : want.label; + const at = { modelStep: i, said, from, hidden }; + if (!taken) + return { + ...at, + key: from, + label: main.step(action, before, after) || action, + disabled: true, + }; + const [b, a] = [nodeOf(taken.from).vars, nodeOf(taken.to).vars]; + return { + ...at, + key: second.key(a), + label: second.step(taken.action, b, a) || taken.action, + own: words(taken), + }; + } + const bad = badOf(rawSecondTraces, trace.invariant); + const checked = secondRules.has(trace.invariant); + const ruleHolds = checked && ![...passed].some((id) => bad.has(stateKey(nodeOf(id).vars))); + return { sameEnd: true, checked, ruleHolds }; +} + +export function divergence(s, path, n, { inSecond, modelEdges }) { + if (s?.key === undefined) return s; + const step = path.findLastIndex((p) => p.modelStep < s.modelStep) + (s.hidden ? 0 : 1); + const plain = edgeId(s.from, s.key); + if (s.disabled) { + const text = inSecond(`${s.label} cannot happen at step ${step}`); + const edge = `${plain}${SEP}cannot${SEP}second`; + return { ...s, step, text, overview: `${text} of bug ${n}`, edge, apart: true }; + } + const text = inSecond(s.label); + const drawn = [s.label, text].some((l) => modelEdges.get(plain)?.labels.has(l)); + return { ...s, step, text, overview: text, edge: drawn ? plain : `${plain}${SEP}second` }; +} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/snapshot.mjs b/harness/pstack/claude-code/skills/statechart/scripts/snapshot.mjs new file mode 100644 index 000000000..ebaf35573 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/snapshot.mjs @@ -0,0 +1,74 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import process from "node:process"; + +const imports = (text, dir) => + [...text.matchAll(/\b(?:from|import)\s*\(?\s*["'](\.{1,2}\/[^"']+)["']/g)].map((m) => + path.resolve(dir, m[1]), + ); + +const DEPENDENCIES = { + ".tla": (text, dir) => + [ + ...text + .replace(/\(\*[^]*?\*\)/g, " ") + .replace(/\\\*.*/g, " ") + .matchAll(/\b(?:EXTENDS|INSTANCE)\s+(\w+(?:\s*,\s*\w+)*)/g), + ].flatMap((m) => m[1].split(/\s*,\s*/).map((name) => path.join(dir, `${name}.tla`))), + ".mjs": imports, + ".js": imports, +}; + +export const copyPath = (root, file, paths = path) => { + const absolute = paths.resolve(file); + const pathRoot = paths.parse(absolute).root; + return paths.join(root, pathRoot.replace(/\W+/g, ""), paths.relative(pathRoot, absolute)); +}; + +const contentOf = (file) => { + try { + return fs.readFileSync(file, "base64"); + } catch { + return null; + } +}; + +export class Inputs { + #root = fs.mkdtempSync(path.join(os.tmpdir(), "statechart-inputs-")); + #files = new Set(); + + constructor() { + process.on("exit", () => fs.rmSync(this.#root, { recursive: true, force: true })); + } + + add(files) { + for (const file of files) if (file) this.#take(path.resolve(file), true); + return this; + } + + copyOf(file) { + return copyPath(this.#root, file); + } + + changed() { + return [...this.#files].find((file) => contentOf(file) !== contentOf(this.copyOf(file))); + } + + #take(file, listed) { + if (this.#files.has(file)) return; + let content; + try { + content = fs.readFileSync(file); + } catch { + if (listed) this.#files.add(file); + return; + } + this.#files.add(file); + const copy = this.copyOf(file); + fs.mkdirSync(path.dirname(copy), { recursive: true }); + fs.writeFileSync(copy, content); + const found = DEPENDENCIES[path.extname(file)]?.(content.toString("utf8"), path.dirname(file)); + for (const dependency of found ?? []) this.#take(dependency, false); + } +} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/spec-text.mjs b/harness/pstack/claude-code/skills/statechart/scripts/spec-text.mjs new file mode 100644 index 000000000..ddfc8d10c --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/spec-text.mjs @@ -0,0 +1,126 @@ +import fs from "node:fs"; +import path from "node:path"; +import { BROKEN_MARK, PRIMES_OR_UNCHANGED, tlaVariables } from "./lib.mjs"; + +export const withoutComments = (spec) => + fs + .readFileSync(spec, "utf8") + .replace(/\(\*[^]*?\*\)/g, " ") + .replace(/\\\*.*/g, " "); + +function specVariables(spec, seen) { + const text = withoutComments(spec); + const extended = /\bEXTENDS\s+(\w+(?:\s*,\s*\w+)*)/.exec(text)?.[1].split(/\s*,\s*/) ?? []; + return [ + ...extended.flatMap((m) => { + const file = path.join(path.dirname(spec), `${m}.tla`); + if (seen.has(m) || !fs.existsSync(file)) return []; + seen.add(m); + return specVariables(file, seen); + }), + ...tlaVariables(text), + ]; +} + +export const RULES_INVARIANT = "StatechartRules"; + +export function writeRulesModule(spec, rules, dir) { + const name = path.basename(spec, ".tla"); + const file = path.join(dir, `${name}_rules.tla`); + const fields = specVariables(spec, new Set([name])).map((v) => `${v} |-> ${v}`); + const broken = rules.map((r) => `(IF ${r} THEN {} ELSE {"${r}"})`).join(" \\cup "); + fs.writeFileSync( + file, + `---- MODULE ${name}_rules ---- +EXTENDS ${name} +StatechartTLC == INSTANCE TLC +StatechartBroken == ${broken} +${RULES_INVARIANT} == StatechartBroken = {} \\/ StatechartTLC!PrintT(<<"${BROKEN_MARK}", [${fields.join(", ")}], StatechartBroken>>) +==== +`, + ); + return file; +} + +export const SCAFFOLD = (name, spec) => { + const text = withoutComments(spec); + const vars = tlaVariables(text); + const primedDefinitions = [ + ...new Set( + text + .split(/\n(?=\S)/) + .map((d) => /^(\w+)\s*(?:\([^)]*\))?\s*==([^]*)/.exec(d)) + .filter((d) => d && PRIMES_OR_UNCHANGED.test(d[2])) + .map((d) => d[1]), + ), + ].sort(); + return `// Chart labels for ${name}, for statechart.mjs. Write labels in the words of the person who reads +// the chart, not in variable or action names. +// v, before, and after map each variable to its value. Strings, numbers, and booleans keep their kind. +// Sequences and tuples, including functions over 1..n, read from 1 as in TLA+, so v.seat[1] is the +// first item. Reading index 0 stops the run. length, array methods, and spread still work. A +// method's index, such as indexOf or the i in map((x, i) => ...), still counts from 0. +// Sets become arrays. Records and other functions become objects. +// Variables: ${vars.join(", ")}. + +// The actions that the reader takes, by action name, when the chart separates what the reader does +// from what happens around the reader. facts.settled is true when only these are enabled. With +// --cover, the overview keeps only the shortest path to each settled state, plus every bug path. +// Leave it empty when every action counts the same. +export const readerActions = []; + +// The label for one model state. facts.enabled is the set of actions the state enables, +// including steps that change no variable. A label "context: progress", such as +// "Primary down: write queued", puts the states of one context in one box, and can draw the +// overview as two parallel regions. Use that form only when the context and the progress change +// independently. A label without ": " draws each state on its own. +// Do not claim a result that a pending step can still change, such as "write lost" while a +// retry is still queued. +export function state(v, facts) { + return JSON.stringify(v); +} + +// The names of the two parallel regions, when the overview draws them: contexts, then progress. +// export const regionNames = ["Primary", "Write"]; + +// The group of one model state, in the reader's words, such as "Order shipped". The chart fills +// the states of one group with one color, and the legend names the groups. TLC still decides +// which states break a rule, and draws them with a thick red border on top of the group color. +// groupColors gives a group a light color. Groups without one take a palette color. +// export function group(v, facts) {} +// export const groupColors = { "Order shipped": "#d1f2eb" }; + +// The label for one step. A step that leaves the reader in the same state is not drawn, +// unless the reader takes it on a bug path. "" draws no label, and a numbered bug step shows the action. +// action is the action's name without its parameters, so compare before and after to tell which +// process or item the step changed. +// STEPS lists every definition that changes a variable. TLC names no step after one that another +// action uses in its /\\ list, such as Log in Save == Log /\\ saved' = TRUE, so delete its entry. +const STEPS = { +${primedDefinitions.map((a) => `\t${a}: ${JSON.stringify(a)},`).join("\n")} +}; +export function step(action, before, after) { + return STEPS[action] ?? action; +} + +// When the step can happen, in plain words, such as "after the lease expires". The chart shows +// "step [words]". "" adds nothing. +// export function guard(action, before, after) {} + +// For a timer step, how many seconds the code waits, from the constant in the code. The chart +// shows "after N s, step". Return nothing for other steps. +// export function timer(action, before, after) {} + +// Why a bad state is bad, in the reader's words, such as "a worker crashed after it took the job, +// and no other worker can take it". No variable, function, or event names. +// Counterexamples that break one rule with one cause are one bug, so two bugs that end in the +// same state stay apart only if their causes differ. Return null to group by the end state. +// export function cause(v, facts, invariant) {} + +// The developer's words for one model state. With it, each bug is also drawn through these states. +// export function detail(v, facts) {} + +// The legend's words for each status, in place of the defaults. Group entries keep their names. +// export const legend = { works: "Works on main", wrong: "Wrong on main", fixed: "Fixed in the open PR" }; +`; +}; diff --git a/harness/pstack/claude-code/skills/statechart/scripts/statechart-help.mjs b/harness/pstack/claude-code/skills/statechart/scripts/statechart-help.mjs new file mode 100644 index 000000000..1a900fc51 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/statechart-help.mjs @@ -0,0 +1,350 @@ +import { CACHE_NAME, PACKAGES } from "./render.mjs"; + +export const USAGE = { + "from-spec": + "node statechart.mjs from-spec SPEC.tla [--config FILE] [--fixed FIXED.tla [--fixed-config FILE] | --against AGREED.tla [--against-config FILE]] [--label TEXT] [--map FILE] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--cover] [--png [--theme NAME]]", + "from-chart": + "node statechart.mjs from-chart CHART.mmd [--module NAME] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png [--theme NAME]]", + render: "node statechart.mjs render CHART.mmd [--theme NAME] [--out FILE]", +}; +export const usage = (lines) => `Usage: ${lines.join("\n ")}`; + +const FROM_SPEC = `from-spec SPEC.tla + Checks each INVARIANT in SPEC.cfg in a separate TLC run. TLC stops at the first rule that a state + breaks, and the separate runs find the rest. Each run also has the rule as a CONSTRAINT, so TLC + stops each path at its first state that breaks the rule, and a rule that stays broken reports + only the step that broke it. SPEC.cfg has SPECIFICATION and one INVARIANT line per rule. Each + PROPERTY, a rule about whole behaviors, gets one TLC run too, and the charts do not draw it. TLC + checks a PROPERTY with no [], <>, or ~> only in the initial states, and the command says so. A + rule listed as both INVARIANT and PROPERTY is checked once, as an INVARIANT. After those runs + print their results, one more run dumps the state graph and checks every INVARIANT on every + state in it, and then the command draws the charts. TLC goes down from the next-state action + through each \\/, \\E, LET, and definition, and names each step after the last definition on that + path. It stops at any other expression, such as /\\, IF, or CASE, so a definition inside one names + no step. Give each kind of step its own definition, and join them with \\/ in the next-state + action. The command warns when steps carry the name of the whole next-state action. The command + prints the config that it checks each model with, and stops when a config does not exist, with + the configs beside the spec. + + --config FILE The config for SPEC.tla. Default: SPEC.cfg next to the spec. + --fixed FIXED.tla The model of the open fix. It is a copy of the spec in a different file, such + as SpecFixed.tla or fix/Spec.tla. The command checks it the same way, with + FIXED.cfg, or with the config of SPEC.tla when there is no FIXED.cfg. The + charts then mark each bad state fixed or still wrong. A rule that the + config of FIXED.tla does not check stays as wrong as it is today. When the + fix shows the same states and steps as a bug of such a rule, the story says + "does not check this rule". + --fixed-config FILE + The config for FIXED.tla. + --against AGREED.tla + The spec that SPEC.tla must follow, when SPEC.tla models the code. It is a + different file from SPEC.tla. The command checks it with AGREED.cfg, and + never with the config of SPEC.tla. SPEC.tla may add variables, record fields, + and actions that AGREED.tla does not have. The comparison leaves them out. + Do not use it with --fixed. + --against-config FILE + The config for AGREED.tla, when it is not AGREED.cfg. Give the config that the + spec's authors say the unchanged spec passes. When the spec shows the same + states and steps as a bug of a rule that this config does not check, the + story says "does not check this rule". + --label TEXT The name of the --fixed or --against model in the charts and the stories. + Default: "the fix", or "the spec". + --map FILE The labels for each model state and step. Default: SPEC.map.mjs next to the + spec. When the file does not exist, the command writes one from the spec's + VARIABLES and actions to fill in, and exits 2 before it runs TLC. Its + comments say what each function gets. The map reads a sequence or tuple + from 1, as in TLA+, so v.seat[1] is its first item. A map that reads from + 0, such as v.pc[r - 1], gets the item before the one it wants, and stops + the run when it reads index 0. + --cover Draw only the shortest path from the start to each settled state, plus + every bug path, in the overview. A state is settled when only the + mapping's readerActions are enabled. The command prints the states and + arrows that the overview leaves out. The counts of states that work or + break a rule still include the states it leaves out. When no step in + TLC's graph is a reader action, the overview draws every state. + + With --fixed or --against, SPEC.tla (and FIXED.tla) models the code, and the command finds each + condition of an action that no comment ties to code with a file and line, as in \\* pay.ts:120. + A condition is a conjunct of the action that changes nothing and reads a variable that the + action does not change. A rule can hold only because the model waits for a condition that the + code does not check, so the command prints these conditions when a rule holds. + With --against, each story says whether a config beside AGREED.tla expects TLC to break the + bug's rule. A config's "\\* expect: RULE" line says what TLC finds, and "\\* module: NAME" names + the module it checks when the folder has more than one .tla file. The command warns when no + config beside AGREED.tla has an expect line for that module.`; + +const FROM_CHART = `from-chart CHART.mmd + Writes NAME.tla, NAME.cfg, and NAME.map.mjs next to the chart. When NAME.tla already exists, it + writes none of them. Otherwise it replaces a NAME.cfg or NAME.map.mjs that is there. Then it + checks the spec as from-spec does, and compares the chart that the spec draws with CHART.mmd. + + --module NAME The TLA+ module name. Default: the chart's file name in CamelCase, such as + NotesApp for notes-app.mmd. + + The chart is a Mermaid stateDiagram-v2. Each transition label is one action. For parallel + regions, split one top-level composite with --, and put each part in a named composite, such as + state Connection { ... }. Each region is one variable. A step moves every region whose current + state has a transition with the step's label, and the other regions stay where they are. Each + rule is one comment line: + %% never NAME: A & B -- the rule in one sentence + Each name after the colon is a state caption or a composite, and each comes from a different + region. A composite stands for every state in it. A "%% when ID, ID" or "%% when not ID, ID" line + right after a transition is its guard, and the transition's label must end with the guard's + words in [ ]. The region takes the transition only while each other region that the line names + is in one of the states that it names, or with not, while none is. A transition with no label is + its own action. NAME.cfg has SPECIFICATION Spec, INVARIANT TypeOK, and one INVARIANT per never + line. The command refuses final, choice, fork, and history states, and regions below the top + level. + + The comparison looks at reachable states, start states, and labeled steps. A chart with regions + compares as its combined machine, with each state named by its region states joined by " | ", so + a guard that blocks a step in only some combinations of regions shows as a difference. The + guard's words in [ ] are not part of the label. A loop compares like any other step. An edge + label "a or b" counts as two labels. An edge with no label counts only when no labeled edge joins + the same two states.`; + +const RENDER = `render CHART.mmd + Draws one chart as a PNG. It does not run TLC, so the chart can come from from-spec, from + from-chart, or from a person. + + --theme NAME The colors of the PNG: light, dark, or classic. Default: light. + --out FILE The PNG to write. Default: the chart's path with .png in place of .mmd. The + command makes the folder when it does not exist. + + Exit codes + 0 The command wrote the PNG. + 2 The command cannot finish: a bad argument, a chart that it cannot read, a renderer that it + cannot install or load, or a PNG that it cannot render or write. It writes no PNG then.`; + +const PNG = `PNG + Pele lays out each chart, and resvg draws it at twice its size. A chart that is more than 100 + million pixels at that size draws smaller, at about 100 million pixels. The first PNG installs + ${PACKAGES.join(" and ")} with npm into a cache folder. The folder is + $XDG_CACHE_HOME/statechart/${CACHE_NAME}, or + ~/.cache/statechart/${CACHE_NAME} when XDG_CACHE_HOME is not set. Delete that folder to + install them again. When npm or the packages fail, the command prints why, exits 2, and writes + no file. + The PNG uses the font that fc-match finds for system-ui. The command measures each label in that + font, so no label goes past its box. When that font is wider than the widths that Pele has built + in, the command makes the wrap width wider by the same amount, so the lines break where the chart + breaks them. Without fc-match, or when resvg cannot read that font, the PNG uses Pele's widths. + Pele puts parallel regions side by side. When a chart with regions has no top-level direction, + the PNG puts the legend above the regions, so the PNG is narrower. + --theme picks light, dark, or classic. light is the default. classic keeps the group fills from + the chart's own classDef lines.`; + +const COMMON = `Both commands + --expect-broken RULE + An INVARIANT or PROPERTY that is known to break. Repeat it, or list rules + with commas, as in A,B. The command then exits 0 when exactly these rules + break, and the --fixed or --against model breaks no other rule. + --title TEXT The chart title. Each bug chart adds ", bug N". + --out DIR The output folder. Default: NAME-statechart next to the input. A run writes its + files in a temporary folder, and then replaces the whole output folder with + it. When runs overlap, the output folder has the files of the run that + finished last. The command does not use an output folder that has files that + it did not write. On a filesystem that cannot rename, the command copies the + files into place, and overlapping runs can mix their files. + --png Also draw each chart as a PNG. The PNG section tells how. + --theme NAME The colors of each PNG: light, dark, or classic. Default: light. It needs + --png. + -h, --help Print this help, and exit 0. + +Output + The command prints whether each rule holds, the chart's counts, and one story per bug. + from-chart also prints the comparison. The command writes these files in the output folder: + chart.mmd The overview. + bug-N.mmd Only the steps of bug N, numbered. Bugs are numbered by fewest reader steps, + then fewest model steps, then rule name, then cause. Bugs that only the + --fixed or --against model has come last. + bug-N-developer.mmd, bug-N-layered.mmd + The path of bug N through the developer's states, when the mapping exports + detail(). The layered chart puts each detail state in a box named by its + state label. A state with a blank detail stays a plain state. + stories.md One entry per bug: the rule, why it breaks, the steps that lead into it, the + shortest path in reader steps, and what the --fixed or --against model does + instead. The rule's sentence is the \\* comment lines above the rule in the + spec, or in a module in its folder that the spec EXTENDS, directly or + through another module. With --fixed or --against, the other spec and its + modules count too, after SPEC.tla and its modules. The entry numbers each + step that changes the state label, each reader action, each step that + changes a variable that the rule reads, and the step into the bug. Between + them, it counts the hidden steps and names each different step once. Above + the steps, a line names up to five different steps that lead into the bug, + most common first, and counts the rest. When more than one step or start + state leads into the bug, the entry says that the path is the shortest that + TLC found. A step leads into the bug when it goes there from a state that + keeps the bug's own rule. That state can already break other rules. "(the + state stays the same)" marks a step on that line that keeps the state's + label and is not on the bug's path. from-spec does not draw such a step. + When TLC names the steps only after the whole next-state action, the entry + cannot count them. Then one line per broken PROPERTY with its sentence, + which comes only from its own spec file, the reader states that repeat + forever, if any, and the TLC output that has its counterexample. + summary.json The counts behind the chart. Each bug has counterexamples, how many model + states end it, each with its own TLC trace; lastSteps, the steps that lead + into it, most common first; and startsHere, whether a start state is in it. + brokenProperties has each broken PROPERTY, its rule sentence, the reader + states that repeat forever as repeats, and its TLC output. mixedStatus + lists the states that break a rule in some cases. chartGroups counts the + drawn states of each group, and mixedGroups lists the states in more than + one group. With --cover, cover lists the states and arrows that the + overview leaves out. With --fixed or --against, uncited has the conditions + with no file and line for each code model. With --against, second.listed + has what each config beside AGREED.tla expects, and each bug has the + configs that list it. + tlc/ The config and the TLC output of each run. When the overview has regions, + from-spec also writes tlc/regions-all.mmd, the regions with every arrow, and + tlc/regions-proof.mmd, every state and step of TLC's graph in one region. + compare-charts.mjs tlc/regions-all.mmd tlc/regions-proof.mmd repeats the + proof. It prints "same" and exits 0, or prints each difference and exits 1. + *.png Each chart as a PNG, with --png. A run without --png deletes them. + +Reading a chart + from-spec draws the overview as two parallel regions when every state reads "context: progress": + the contexts on one side, the progress on the other, both at the same time. It does so only when + that has fewer states, when the regions combine back into exactly every state and step of TLC's + graph, and when each state's color is the worse color of its two region states. Otherwise it puts + the states of one context in a box named context, and prints why. Each state in a box shows only + its progress, and a context with one state stays a plain state. + Regions draw only the arrows of the shortest way to each state, of each bug path, and of the + nearest step with each label. The script checks that these arrows make no step that TLC's graph + does not have. At a step that they make wrongly, it puts back every arrow that the full regions + take there. + An arrow from a box's border leaves from every state in the box. The chart draws it only when + every state in the box takes the step with the same labels to the same state, and no state + outside the box has the box's name. + In regions, a step leaves every region that has it. "[when A or B]" or "[unless A or B]", + whichever is shorter, says which states of the other region the step needs, and a "%% when" or + "%% when not" line under it lists their ids. A loop says the region may also stay. + Other words in [ ] come from the mapping's guard(), and "after N s" from its timer(). An edge + shows guard() words only when every model step behind it has them. In regions, they replace + "when A or B" and "unless A or B". TLC does not check them, and has no clock, so a reviewer + checks them against the code. + Gray states work. Red states break a rule. An orange state breaks a rule in some cases: a model + state under its label breaks a rule, and a path without a bug reaches another one. A white state + is reached only after a bug: every path from the start to its model states passes a state that + breaks a rule, and none of them breaks one. With --fixed, red, orange, and white describe the + fix, and green states are fixed by it or exist only with it. A purple state breaks a rule only + with the fix: today it works, is reached only after a bug, or does not exist. With --against, a + state takes the worse of its two models' colors, red, then orange, then white, and green states + exist only in the spec. + When the mapping exports group(), the fill is the state's group, which the legend names. A chart + state is in a group when all its model states are in it, and a region state when every state + that it is part of is in it. A state in no group, or in more than one, is gray. The legend lists + the groups that the chart draws: first the groups in groupColors, in that order, then the others + from A to Z, and then each status border that the chart uses. groupColors gives a group a light + "#rgb" or "#rrggbb" color. A group with no color takes the next free palette color: light blue, + orange, purple, yellow, teal, tan, then pink. A group cannot have the name of a status, such as + "Breaks a rule". Without group(), the legend shows only the statuses that the chart uses. In the + .mmd file, the status is a thick border of its color, gray for a white state, on top of the group + fill. In a PNG, a red, orange, purple, or white state is a box of that color with a border and + text of its color in place of its group fill, and a fixed state is a green box, so each box shows + one signal. + A thick blue border and numbered labels mark the bug. With group(), and in a PNG, the blue border + marks only the states on the bug's path that work, and the numbered labels show the rest. + Mermaid state diagrams cannot color an edge. + Each bug replays on the other model by its state labels and step words, which are what the + reader sees. The replay takes the bug's own steps. Between them, it also takes the other model's + steps of actions that SPEC.tla's state graph does not have, when they keep the state label, are + not reader actions, and change no variable that the broken rule reads. A bug step that the story + hides must keep the state label on the other model too, unless that model cannot take it. A step + that the reader sees takes a step with the same words between the same two state labels, and + prefers a step of the same action. A hidden step of an action that the other model does not have + is skipped. A step that the reader sees of such an action takes any step to the same shared state + when it changes a shared variable, or else a step with the same words between the same two state + labels. + A label "with the fix, STEP" or "in the spec, STEP" is a step that only the other model takes. On + a bug's path it leaves the state before the first step that the other model shows the reader + differently, and ends where that model goes instead. The story quotes the other model's words + when they differ. "with the fix, STEP cannot happen at step N" or "in the spec, STEP cannot + happen at step N" loops on the state where that model stops the step, on an arrow of its own. On + the overview, the label ends with "of bug M". Outside regions, two states otherwise have at most + one arrow between them. Its lines are the numbered steps, then SPEC.tla's steps, then the other + model's steps, with a blank line between each group. + The other model may have other variables, record fields, and actions. The comparison leaves out a + variable or field that no state of one model has, in its state graph or its traces, and one whose + values never have the same type in the two models. One mapping draws both models, so it may read + only what both have. The command stops when the mapping reads, on a state of the other model, a + variable or field that no state of that model has, or one whose value has a type that SPEC.tla + never has there. It also stops when the mapping lists the fields of such a state, as a spread, + Object.keys, or JSON.stringify does. Test "name" in v before the mapping reads a field that the + other model lacks. The command also stops when a label, guard, group, or cause that the mapping + makes for the other model has the word "undefined" in it. + A chart with another model has the line "%% second model (fix|against): LABEL" right after + stateDiagram-v2. Each chart starts with front matter that keeps Mermaid from shrinking it to the + page or wrapping its labels again. + An edge lists every step that takes it, most common first. Words that the steps share at the + start are written once, as in "you tap Pay or Back". + +Regenerate + List every rule that is known to break with --expect-broken, the goals and the known bugs. Each + run then exits 0 until a rule changes, so one script rebuilds every chart and stops on any change: + set -e + node statechart.mjs from-spec Code.tla --against Agreed.tla --expect-broken NoLostUpload --png + A run that leaves a broken rule out of --expect-broken exits 1, and set -e stops the script there. + +TLC + TLA2TOOLS is the path to tla2tools.jar, and JAVA is the path to the java binary. With only + TLA2TOOLS set, the command runs java from PATH. Otherwise it uses TLC 1.7.4 and Java temurin-25 + from mise, or JAVA when it is set, and installs what is missing with mise. The first install + downloads about 305 MB and needs the network. It checks the checksum of tla2tools.jar. mise runs + in an empty temporary folder, so the mise files around the spec, in a repo, or above TMPDIR do + not apply, do not change, and need no trust. When mise cannot start or fails, the command prints + why, the install command, and how to set TLA2TOOLS and JAVA instead. + When the command starts, it copies its input files, which Exit codes lists. TLC checks the + copies, so an edit during the run does not change what TLC checks. + TLC checks each rule with one worker per CPU. When a rule is broken, TLC checks it again with one + worker, which finds the shortest bug paths, the same on every run. The command then prints how + many times TLC ran and how long that took. + TLC can find that a rule is broken and then stop on an error, such as a Java exception. The + command then keeps the paths that TLC printed, marks the rule broken, and warns that the charts + can miss states that break it. A smaller constant can let TLC finish. + +Exit codes + 0 Every rule holds, or exactly the --expect-broken rules break. from-chart also draws CHART.mmd + back unchanged. + 1 Another rule is broken, an expected rule holds, or the chart that the spec draws is not + CHART.mmd. The output says which. + 2 The command cannot finish: a bad argument, a missing file, a chart or TLC output that it + cannot read, no TLC, a helper script that cannot start, an output folder with files that the + command did not write, an input file whose content changed during the run, a new mapping to + fill in, a mapping that it cannot use, a renderer that --png cannot install or load, or a + chart that --png cannot render. + When a chart does not render, the command keeps the files it wrote and exits 2, whatever the + rules give. Otherwise, exit 2 does not change the output folder, unless the command moved it + aside and cannot put it back. Then the message names the folder's new path. When the command + stops after TLC has run, it keeps the files that it wrote, TLC's output among them, in a + temporary folder, and the message names it. When TLC's state graph is too large to draw, the + command prints whether each rule holds and exits 2, whatever the rules give. It stops TLC as + soon as the graph passes that size. The input files are the spec and config, the --fixed or + --against spec and config, the chart, and the map. Each .tla file in the same folder that an + input .tla file EXTENDS or INSTANCEs is an input file too, and so is each file that an input + .mjs or .js file imports by a relative path. Before --png renders, and again before it writes + the output folder, the command compares the content of each input file with its content at + the start. A save that does not change the content does not stop the run.`; + +export const HELP = { + "": `${usage([...Object.values(USAGE), "node statechart.mjs [from-spec|from-chart|render] --help"])} + +Draws a Mermaid statechart from TLC's state graph of a TLA+ spec. The spec can model any system, +such as a product flow, a protocol, a service, a job queue, or an algorithm. A mapping labels each +state and step in the words of the person who reads the chart, and the help calls that person the +reader. The reader can be a user, an operator, an engineer, or a reviewer. + +${FROM_SPEC} + +${FROM_CHART} + +${COMMON} + +${RENDER} + +${PNG}`, + "from-spec": `${usage([USAGE["from-spec"]])}\n\n${FROM_SPEC}\n\n${COMMON}\n\n${PNG}`, + "from-chart": `${usage([USAGE["from-chart"]])}\n\n${FROM_CHART}\n\n${COMMON}\n\n${PNG}`, + render: `${usage([USAGE.render])}\n\n${RENDER}\n\n${PNG}`, +}; + +export const more = (cmd) => + `Run node statechart.mjs ${cmd ? `${cmd} ` : ""}--help for the flags, the output, and the exit codes.`; diff --git a/harness/pstack/claude-code/skills/statechart/scripts/statechart.mjs b/harness/pstack/claude-code/skills/statechart/scripts/statechart.mjs index 0ff4255c2..555f10b8c 100644 --- a/harness/pstack/claude-code/skills/statechart/scripts/statechart.mjs +++ b/harness/pstack/claude-code/skills/statechart/scripts/statechart.mjs @@ -1,217 +1,29 @@ #!/usr/bin/env node -import { spawnSync } from "node:child_process"; +import { constants } from "node:buffer"; +import { spawn, spawnSync } from "node:child_process"; import fs from "node:fs"; import os from "node:os"; import path from "node:path"; import process from "node:process"; import { fileURLToPath, pathToFileURL } from "node:url"; +import { HELP, more, USAGE, usage } from "./statechart-help.mjs"; import { BROKEN_PROPERTY, - findTlc, + checkedOnlyInInitialStates, joinAnd, parseDot, - PRIMES_OR_UNCHANGED, - tlaVariables, tlcOutputWarning, uncitedConditions, } from "./lib.mjs"; +import { loopSentence, readerStateNamer, repeatsIn } from "./property-loop.mjs"; +import { loadRenderer, renderPng, THEMES } from "./render.mjs"; +import { ruleBook } from "./rule-book.mjs"; +import { Inputs } from "./snapshot.mjs"; +import { RULES_INVARIANT, SCAFFOLD, withoutComments, writeRulesModule } from "./spec-text.mjs"; +import { cannotRunTlc, findTlc } from "./tlc-tools.mjs"; const here = path.dirname(fileURLToPath(import.meta.url)); - -const USAGE = { - "from-spec": - "node statechart.mjs from-spec SPEC.tla [--config FILE] [--fixed FIXED.tla [--fixed-config FILE] | --against AGREED.tla [--against-config FILE]] [--label TEXT] [--map FILE] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png]", - "from-chart": - "node statechart.mjs from-chart CHART.mmd [--module NAME] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png]", -}; -const usage = (lines) => `Usage: ${lines.join("\n ")}`; - -const FROM_SPEC = `from-spec SPEC.tla - Checks each INVARIANT in SPEC.cfg in a separate TLC run. TLC stops at the first rule that a state - breaks, and the separate runs find the rest. Each run also has the rule as a CONSTRAINT, so TLC - stops each path at its first state that breaks the rule, and a rule that stays broken reports - only the step that broke it. SPEC.cfg has SPECIFICATION and one INVARIANT line per rule. Each - PROPERTY, a rule about whole behaviors, gets one TLC run too, and the charts do not draw it. A - rule listed as both INVARIANT and PROPERTY is checked once, as an INVARIANT. One - more run dumps the state graph, and then the command draws the charts. TLC names each step after - the innermost named action that takes it, so give each kind of step its own named action. The - command warns when steps carry the name of the whole next-state action. The command prints the - config that it checks each model with, and stops when a config does not exist, with the configs - beside the spec. - - --config FILE The config for SPEC.tla. Default: SPEC.cfg next to the spec. - --fixed FIXED.tla The model of the open fix. It is a copy of the spec with its own file and - module name, such as SpecFixed.tla with MODULE SpecFixed. The command checks - it the same way, with FIXED.cfg, or with the config of SPEC.tla when there is - no FIXED.cfg. The charts then mark each bad state fixed or still wrong. - --fixed-config FILE - The config for FIXED.tla. - --against AGREED.tla - The spec that SPEC.tla must follow, when SPEC.tla models the code. It needs - its own file and module name. The command checks it with AGREED.cfg, and - never with the config of SPEC.tla. SPEC.tla may add variables, record fields, - and actions that AGREED.tla does not have. The comparison leaves them out. - Do not use it with --fixed. - --against-config FILE - The config for AGREED.tla, when it is not AGREED.cfg. Give the config that the - spec's authors say the unchanged spec passes. A rule that this config does not - check is "not checked" in the stories. - --label TEXT The name of the --fixed or --against model in the charts and the stories. - Default: "the fix", or "the spec". - --map FILE The labels for each model state and step. Default: SPEC.map.mjs next to the - spec. When the file does not exist, the command writes one from the spec's - VARIABLES and actions to fill in, and exits 2 before it runs TLC. Its - comments say what each function gets. - - With --fixed or --against, SPEC.tla (and FIXED.tla) models the code, and the command finds each - condition of an action that no comment ties to code with a file and line, as in \\* pay.ts:120. - A condition is a conjunct of the action that changes nothing and reads a variable that the - action does not change. A rule can hold only because the model waits for a condition that the - code does not check, so the command prints these conditions when a rule holds. - With --against, each story says whether a config beside AGREED.tla expects TLC to break the - bug's rule. A config's "\\* expect: RULE" line says what TLC finds, and "\\* module: NAME" names - the module it checks when the folder has more than one .tla file. The command warns when no - config beside AGREED.tla has an expect line for that module.`; - -const FROM_CHART = `from-chart CHART.mmd - Writes NAME.tla, NAME.cfg, and NAME.map.mjs next to the chart. When NAME.tla already exists, it - writes none of them. Then it checks the spec as from-spec does, and compares the chart that the - spec draws with CHART.mmd. - - --module NAME The TLA+ module name. Default: the chart's file name in CamelCase, such as - NotesApp for notes-app.mmd. - - The chart is a Mermaid stateDiagram-v2. Each transition label is one action. For parallel - regions, split one top-level composite with --, and put each part in a named composite, such as - state Connection { ... }. Each region is one variable. A step moves every region whose current - state has a transition with the step's label, and the other regions stay where they are. Each - rule is one comment line: - %% never NAME: A & B -- the rule in one sentence - Each name after the colon is a state caption or a composite, and each comes from a different - region. A composite stands for every state in it. The command refuses final, choice, fork, and - history states, and regions below the top level.`; - -const COMMON = `Both commands - --expect-broken RULE - An INVARIANT or PROPERTY that is known to break. Repeat it, or list rules - with commas, as in A,B. The command then exits 0 when exactly these rules - break, and the --fixed or --against model breaks no other rule. - --title TEXT The chart title. Each bug chart adds ", bug N". - --out DIR The output folder. Default: NAME-statechart next to the input. A run writes its - files in a temporary folder, and then replaces the whole output folder with - it. When runs overlap, the output folder has the files of the run that - finished last. The command does not use an output folder that has files that - it did not write. On a filesystem that cannot rename, the command copies the - files into place, and overlapping runs can mix their files. - --png Also render each chart to PNG with npx @mermaid-js/mermaid-cli. - -h, --help Print this help, and exit 0. - -Output - The command prints whether each rule holds, the chart's counts, and one story per bug. - from-chart also prints the comparison. The command writes these files in the output folder: - chart.mmd The overview. - bug-N.mmd Only the steps of bug N, numbered. Bugs are numbered by fewest reader steps, - then fewest model steps, then rule name, then cause. Bugs that only the - --fixed or --against model has come last. - bug-N-developer.mmd, bug-N-layered.mmd - The path of bug N through the developer's states, when the mapping exports - detail(). - stories.md One entry per bug: the rule, why it breaks, the shortest path in reader - steps, and what the --fixed or --against model does instead. - summary.json The counts behind the chart. With --fixed or --against, uncited has the - conditions with no file and line for each code model. With --against, - second.listed has what each config beside AGREED.tla expects, and each bug - has the configs that list it. - tlc/ The config and the TLC output of each run. When the overview has regions, - from-spec also writes tlc/regions-all.mmd, the regions with every arrow, and - tlc/regions-proof.mmd, every state and step of TLC's graph in one region. - compare-charts.mjs tlc/regions-all.mmd tlc/regions-proof.mmd repeats the - proof. - *.png Each chart as a PNG, with --png. A run without --png deletes them. - -Reading a chart - from-spec draws the overview as two parallel regions when every state reads "context: progress": - the contexts on one side, the progress on the other, both at the same time. It does so only when - that has fewer states, and when the regions combine back into exactly every state and step of - TLC's graph. Otherwise it puts the states of one context in a box named context, and prints why. - Regions draw only the arrows of the shortest way to each state, of each bug path, and of the - nearest step with each label. The script checks that these arrows make no step that TLC's graph - does not have. - An arrow from a box's border leaves from every state in the box. - In regions, a step leaves every region that has it. "[when A]" or "[unless A]" says which states - of the other region the step needs. A loop says the region may also stay. - Other words in [ ] come from the mapping's guard(), and "after N s" from its timer(). TLC does - not check them, and has no clock, so a reviewer checks them against the code. - Gray states work. Red states break a rule. With --fixed, red states stay wrong with the fix, and - green states are fixed by it or exist only with it. With --against, red states break a rule in - either model, and green states exist only in the spec. - When the mapping exports group(), the fill is the state's group, which the legend names, and the - status is a thick red or green border on top. tlc-to-chart.mjs --help tells more. - A state is bad when it breaks a rule, or when every path from the start to it passes a state - that breaks one. TLC stops each path at the first state that breaks the rule, so a rule that - stays broken colors only the states where it first breaks and the states after them that no - path without a bug reaches. - A thick blue border and numbered labels mark the bug. Mermaid state diagrams cannot color an - edge. - A label "with the fix, STEP" or "in the spec, STEP" is a step that only the other model takes. On - a bug's path it leaves the state before the first step that the other model shows the reader - differently, and ends where that model goes instead. "with the fix, STEP cannot happen" or "in - the spec, STEP cannot happen" loops on the state where that model stops the step. Outside - regions, two states have at most one arrow between them. Its lines are the numbered steps, then - SPEC.tla's steps, then the other model's steps, with a blank line between each group. - An edge lists every step that takes it, most common first. Words that the steps share at the - start are written once, as in "you tap Pay or Back". - -Regenerate - With --expect-broken, one script rebuilds every chart and stops on any change: - set -e - node statechart.mjs from-spec Code.tla --against Agreed.tla --expect-broken NoLostUpload --png - -TLC - TLA2TOOLS is the path to tla2tools.jar, and JAVA is the java binary (default: java, Java 11 or - later). Download tla2tools.jar from https://github.com/tlaplus/tlaplus/releases. When TLA2TOOLS - is not set, the command says where to get TLC and exits 2. - TLC checks each rule with one worker per CPU. When a rule is broken, TLC checks it again with one - worker, which finds the shortest bug paths, the same on every run. The command then prints how - many times TLC ran and how long that took. - TLC can find that a rule is broken and then fail with a Java exception while it reports the rule. - The command then keeps the paths that TLC printed, marks the rule broken, and warns that the - charts can miss states that break it. The failure is in TLC, so do not weaken the rule because of - it. A smaller constant can let TLC finish. - -Exit codes - 0 Every rule holds, or exactly the --expect-broken rules break. from-chart also draws CHART.mmd - back unchanged. - 1 Another rule is broken, an expected rule holds, or the chart that the spec draws is not - CHART.mmd. The output says which. - 2 The command cannot finish: a bad argument, a missing file, a chart or TLC output that it - cannot read, no TLC, a helper script that cannot start, an output folder with files that the - command did not write, an input file that changed during the run, a new mapping to fill in, - or a chart that --png cannot render. When a chart does not render, the command keeps the - files it wrote and exits 2, whatever the rules give. Otherwise, exit 2 does not change the - output folder, unless the command moved it aside and cannot put it back. Then the message - names the folder's new path. When TLC stops on an error, the message names TLC's output, in a - temporary folder that the command keeps. When the command cannot put its files in the output - folder, it keeps them in a temporary folder, and the message names it. The command checks - its input files again before --png renders, so a changed input stops the run before that.`; - -const HELP = { - "": `${usage([...Object.values(USAGE), "node statechart.mjs [from-spec|from-chart] --help"])} - -Draws a Mermaid statechart of what an app's user sees, from TLC's state graph of a TLA+ spec. The -help calls that user the reader. - -${FROM_SPEC} - -${FROM_CHART} - -${COMMON}`, - "from-spec": `${usage([USAGE["from-spec"]])}\n\n${FROM_SPEC}\n\n${COMMON}`, - "from-chart": `${usage([USAGE["from-chart"]])}\n\n${FROM_CHART}\n\n${COMMON}`, -}; - -const more = (cmd) => - `Run node statechart.mjs ${cmd ? `${cmd} ` : ""}--help for the flags, the output, and the exit codes.`; +const { MAX_STRING_LENGTH } = constants; class CannotFinish extends Error {} class TlcStopped extends CannotFinish {} @@ -219,7 +31,7 @@ class NotPublished extends CannotFinish {} const COMMANDS = { "from-spec": { - flags: [ + values: [ "--map", "--config", "--fixed", @@ -230,23 +42,33 @@ const COMMANDS = { "--expect-broken", "--title", "--out", + "--theme", ], + switches: ["--cover", "--png"], + renders: (flags) => flags.png, run: fromSpec, }, - "from-chart": { flags: ["--module", "--expect-broken", "--title", "--out"], run: fromChart }, + "from-chart": { + values: ["--module", "--expect-broken", "--title", "--out", "--theme"], + switches: ["--png"], + renders: (flags) => flags.png, + run: fromChart, + }, + render: { values: ["--theme", "--out"], switches: [], renders: () => true, run: renderFile }, }; function parseArgs([command, ...rest]) { if (command === "-h" || command === "--help") return { help: HELP[""] }; if (!Object.hasOwn(COMMANDS, command)) throw new CannotFinish(`${usage(Object.values(USAGE))}\n${more()}`); + const { values, switches, renders } = COMMANDS[command]; const flags = { "expect-broken": new Set() }; let input; for (let i = 0; i < rest.length; i++) { const a = rest[i]; if (a === "-h" || a === "--help") return { help: HELP[command] }; - if (a === "--png") flags.png = true; - else if (COMMANDS[command].flags.includes(a)) { + if (switches.includes(a)) flags[a.slice(2)] = true; + else if (values.includes(a)) { if (i + 1 === rest.length) throw new CannotFinish(`${a} needs a value. ${more(command)}`); const name = a.slice(2); if (flags[name] instanceof Set) for (const v of rest[++i].split(",")) flags[name].add(v); @@ -255,9 +77,25 @@ function parseArgs([command, ...rest]) { else throw new CannotFinish(`unknown argument ${a}. ${more(command)}`); } if (input === undefined) throw new CannotFinish(`${usage([USAGE[command]])}\n${more(command)}`); + const themes = Object.keys(THEMES); + if (flags.theme !== undefined && !themes.includes(flags.theme)) + throw new CannotFinish( + `--theme takes ${themes.slice(0, -1).join(", ")}, or ${themes.at(-1)}. ${more(command)}`, + ); + if (flags.theme !== undefined && !renders(flags)) + throw new CannotFinish(`--theme needs --png. ${more(command)}`); return { command, input, flags }; } +async function rendererFor(command, flags) { + if (!COMMANDS[command].renders(flags)) return null; + try { + return { lib: await loadRenderer(), theme: THEMES[flags.theme ?? "light"] }; + } catch (e) { + throw new CannotFinish(e.message); + } +} + const rel = (file) => { const r = path.relative(process.cwd(), file) || "."; return r === ".." || r.startsWith(`..${path.sep}`) ? path.resolve(file) : r; @@ -314,19 +152,48 @@ function splitRules(cfg) { }; } -function tlc(tools, spec, cfg, base, { dump, workers = "1" } = {}) { +function runJava(file, args, options, dot) { + return new Promise((resolve) => { + const r = { stdout: "", stderr: "", stopped: false, overflowed: false }; + const child = spawn(file, args, options); + for (const stream of ["stdout", "stderr"]) + child[stream].setEncoding("utf8").on("data", (text) => { + if (r.overflowed) return; + if (r[stream].length + text.length <= MAX_STRING_LENGTH) r[stream] += text; + else { + r.overflowed = true; + child.kill("SIGKILL"); + } + }); + child.on("error", (e) => (r.error = e)); + const watch = + dot && + setInterval(() => { + if ((fs.statSync(dot, { throwIfNoEntry: false })?.size ?? 0) <= MAX_STRING_LENGTH) return; + r.stopped = true; + child.kill("SIGKILL"); + }, 500); + child.on("close", (status, signal) => { + clearInterval(watch); + resolve({ ...r, status, signal }); + }); + }); +} + +async function tlc(tools, spec, cfg, base, { dump, module, workers = "1" } = {}) { fs.writeFileSync(`${base}.cfg`, cfg); const metadir = fs.mkdtempSync(path.join(os.tmpdir(), "tlc-")); // TLC writes the standard modules, such as Naturals.tla, to java.io.tmpdir under fixed names, // so runs that share that folder can read a module while another run rewrites it. const javaTmp = path.join(metadir, "tmp"); fs.mkdirSync(javaTmp); - // -deadlock: a UI at rest has nothing left to do, which TLC would report as a deadlock. - const r = spawnSync( + // -deadlock: a system at rest has nothing left to do, which TLC would report as a deadlock. + const r = await runJava( tools.java, [ "-XX:+UseParallelGC", `-Djava.io.tmpdir=${javaTmp}`, + ...(module ? [`-DTLA-Library=${path.dirname(path.resolve(spec))}`] : []), "-cp", tools.jar, "tlc2.TLC", @@ -339,12 +206,23 @@ function tlc(tools, spec, cfg, base, { dump, workers = "1" } = {}) { ...(dump ? ["-dump", "dot,actionlabels", path.resolve(dump)] : []), "-config", path.resolve(`${base}.cfg`), - path.basename(spec), + module ? path.resolve(module) : path.basename(spec), ], - { cwd: path.dirname(path.resolve(spec)), encoding: "utf8", maxBuffer: 1 << 30 }, + { cwd: path.dirname(path.resolve(spec)) }, + dump, ); fs.rmSync(metadir, { recursive: true, force: true }); - if (r.error) throw new CannotFinish(`cannot run TLC: ${r.error.message}`); + if (r.overflowed) { + fs.writeFileSync(`${base}.out`, r.stdout); + throw new TlcStopped( + `${base}.out: TLC printed more than ${MAX_STRING_LENGTH.toLocaleString("en-US")} characters, the most that the command can read, so the command stopped TLC. Smaller constants make TLC print less.`, + ); + } + if (r.stopped) { + fs.writeFileSync(`${base}.out`, r.stdout); + return { stdout: r.stdout, stopped: true }; + } + if (r.error || !/^TLC2 Version/m.test(r.stdout)) throw new CannotFinish(cannotRunTlc(tools, r)); fs.writeFileSync(`${base}.out`, r.stdout); try { return { stdout: r.stdout, warning: tlcOutputWarning(r.stdout, `${base}.out`) }; @@ -353,14 +231,8 @@ function tlc(tools, spec, cfg, base, { dump, workers = "1" } = {}) { } } -const withoutComments = (spec) => - fs - .readFileSync(spec, "utf8") - .replace(/\(\*[^]*?\*\)/g, " ") - .replace(/\\\*.*/g, " "); - -function warnUnnamedSteps(spec, cfg, dot) { - const text = withoutComments(spec); +function warnUnnamedSteps({ spec, copy }, cfg, dot) { + const text = withoutComments(copy); const specName = /^SPECIFICATION\s+(\w+)/m.exec(cfg)?.[1]; const specBody = specName && new RegExp(`^${specName}\\s*==(.*(\\n[ \\t]+.*)*)`, "m").exec(text); const next = @@ -369,13 +241,15 @@ function warnUnnamedSteps(spec, cfg, dot) { const unnamed = [...graph.edges, ...graph.loops].filter((e) => e.action === next).length; if (unnamed > 0) console.error( - `${rel(spec)}: ${unnamed} ${unnamed === 1 ? "step has" : "steps have"} the name ${next}, the whole next-state action, because TLC names a step after the innermost named action that takes it. If the reader must tell these steps apart, give each part of ${next} its own name, as in Pay == ... and ${next} == Pay \\/ Charge.`, + `${rel(spec)}: ${unnamed} ${unnamed === 1 ? "step has" : "steps have"} the name ${next}, the whole next-state action. TLC goes down from ${next} through each \\/, \\E, LET, and definition, and names each step after the last definition on that path. It stops at any other expression, such as /\\, IF, or CASE. If the reader must tell these steps apart, join a definition for each kind of step with \\/ in ${next}, as in ${next} == Pay \\/ Charge, and move each conjunct of ${next} into those definitions.`, ); + return next; } -function readModel(spec, cfgFile, flag) { - if (!fs.existsSync(spec)) throw new CannotFinish(`${rel(spec)} does not exist`); - if (!fs.existsSync(cfgFile)) { +function readModel(spec, cfgFile, flag, inputs) { + const [copy, cfgCopy] = [spec, cfgFile].map((f) => inputs.copyOf(f)); + if (!fs.existsSync(copy)) throw new CannotFinish(`${rel(spec)} does not exist`); + if (!fs.existsSync(cfgCopy)) { const beside = fs .readdirSync(path.dirname(spec)) .filter((f) => f.endsWith(".cfg")) @@ -388,11 +262,11 @@ function readModel(spec, cfgFile, flag) { }`, ); } - return { spec, cfgFile, ...splitRules(fs.readFileSync(cfgFile, "utf8")) }; + return { spec, copy, cfgFile, ...splitRules(fs.readFileSync(cfgCopy, "utf8")) }; } -function readInput(spec, cfgFile, expected) { - const model = readModel(spec, cfgFile, "--config"); +function readInput(spec, cfgFile, expected, inputs) { + const model = readModel(spec, cfgFile, "--config", inputs); const names = [...model.rules, ...model.properties]; const unknown = [...expected].filter((r) => !names.includes(r)); if (unknown.length) @@ -402,53 +276,79 @@ function readInput(spec, cfgFile, expected) { return model; } -function check(tools, model, { work, out }, expected) { - const { spec, cfgFile, cfgWithoutRules, rules, properties } = model; +async function check(tools, model, { work, out, role }, expected) { + const { spec, copy, cfgFile, cfgWithoutRules, rules, properties } = model; const started = Date.now(); let runs = 0; const run = (cfg, file, options) => { runs++; - return tlc(tools, spec, cfg, file, options); + return tlc(tools, copy, cfg, file, options); }; const name = path.basename(spec, ".tla"); - const tlcDir = path.join(work, "tlc"); - const dot = path.join(tlcDir, `${name}.dot`); - // One worker writes the graph in a fixed order, which breaks ties between equally short paths. - run(cfgWithoutRules, path.join(tlcDir, name), { dump: dot }); - warnUnnamedSteps(spec, cfgWithoutRules, dot); - console.log(`TLC on ${path.basename(spec)} with ${rel(cfgFile)}:`); + const tlcDir = path.join(work, "tlc", ...(role ? [role] : [])); + fs.mkdirSync(tlcDir, { recursive: true }); + console.log(`TLC on ${rel(spec)} with ${rel(cfgFile)}:`); if (rules.length === 0) console.log( ` ${path.basename(cfgFile)} has no INVARIANT, so ${properties.length ? "the chart marks no state broken" : "no rule is checked"}.`, ); const said = (rule, broken) => ` ${rule} ${broken ? (expected.has(rule) ? "is broken (expected)" : "is broken") : "holds"}.`; - const shown = (file) => rel(path.join(out, "tlc", path.basename(file))); - const traces = rules.map((rule) => { + const shown = published(work, out); + const traces = []; + for (const rule of rules) { const file = path.join(tlcDir, `${name}-${rule}`); const cfg = `${cfgWithoutRules}INVARIANT ${rule}\nCONSTRAINT ${rule}\n`; - let result = run(cfg, file, { workers: "auto" }); + let result = await run(cfg, file, { workers: "auto" }); // Several workers can reach a bad state by a longer path first. One worker searches breadth // first in a fixed order, so the chart's bug paths are the shortest, and the same every run. - if (/^Error: Invariant /m.test(result.stdout)) result = run(cfg, file); + if (/^Error: Invariant /m.test(result.stdout)) result = await run(cfg, file); const broken = /^Error: Invariant /m.test(result.stdout); console.log(said(rule, broken)); if (result.warning) console.error(`${shown(`${file}.out`)}: ${result.warning}`); - return { file: `${file}.out`, rule, broken }; - }); - const temporal = properties.map((property) => { + traces.push({ file: `${file}.out`, rule, broken }); + } + const temporal = []; + for (const property of properties) { const file = path.join(tlcDir, `${name}-${property}`); - const { stdout } = run(`${cfgWithoutRules}PROPERTY ${property}\n`, file, { workers: "auto" }); + const { stdout } = await run(`${cfgWithoutRules}PROPERTY ${property}\n`, file, { + workers: "auto", + }); const broken = BROKEN_PROPERTY.test(stdout); - console.log( - `${said(property, broken)}${broken ? ` The chart draws only INVARIANT rules, so read TLC's counterexample in ${shown(`${file}.out`)}.` : ""}`, - ); - return { file: `${file}.out`, rule: property, broken }; + const notes = [ + broken && + `The chart draws only INVARIANT rules, so read TLC's counterexample in ${shown(`${file}.out`)}.`, + checkedOnlyInInitialStates(spec, property) && + `TLC checked ${property} only in the initial states, because it has no [], <>, or ~>. List it as an INVARIANT to check every state.`, + ]; + console.log([said(property, broken), ...notes.filter(Boolean)].join(" ")); + temporal.push({ file: `${file}.out`, rule: property, broken }); + } + const dot = path.join(tlcDir, `${name}.dot`); + const module = rules.length ? writeRulesModule(copy, rules, tlcDir) : undefined; + const dumpCfg = module ? `${cfgWithoutRules}INVARIANT ${RULES_INVARIANT}\n` : cfgWithoutRules; + const { stopped } = await run(dumpCfg, path.join(tlcDir, name), { + dump: dot, + module, + workers: "1", }); + const dotBytes = fs.statSync(dot).size; + const next = dotBytes <= MAX_STRING_LENGTH && warnUnnamedSteps(model, cfgWithoutRules, dot); const seconds = ((Date.now() - started) / 1000).toFixed(1); console.log(` TLC ran ${runs} ${runs === 1 ? "time" : "times"} in ${seconds} s.`); const broken = [...traces, ...temporal].some((t) => t.broken); - return { spec, dot, traces, properties: temporal, broken }; + return { + spec, + copy, + dot, + dotBytes, + stopped, + next, + dumpOut: path.join(tlcDir, `${name}.out`), + traces, + properties: temporal, + broken, + }; } function rulesAsExpected(now, other, expected) { @@ -466,93 +366,28 @@ function rulesAsExpected(now, other, expected) { return !unexpected.length && !holds.length; } -const SCAFFOLD = (name, spec) => { - const text = withoutComments(spec); - const vars = tlaVariables(text); - // A definition that only another action calls is listed too, though TLC names no step after it. - const actions = [ - ...new Set( - text - .split(/\n(?=\S)/) - .map((d) => /^(\w+)\s*(?:\([^)]*\))?\s*==([^]*)/.exec(d)) - .filter((d) => d && PRIMES_OR_UNCHANGED.test(d[2])) - .map((d) => d[1]), - ), - ].sort(); - return `// Chart labels for ${name}, for statechart.mjs. Write labels in plain words, not variable or action names. -// v, before, and after map each variable to its value. Strings, numbers, and booleans keep their kind. -// Sequences, tuples, and sets become arrays, indexed from 0. Records and functions become objects. -// Variables: ${vars.join(", ")}. - -// The actions the reader takes. facts.settled is true when only these are enabled, and the overview -// keeps the shortest path to each settled state, plus every bug path. -export const readerActions = []; - -// The label for one model state. facts.enabled is the set of actions the state enables, -// including steps that change no variable. Write it as "context: progress", such as -// "Offline: upload queued": the chart puts the states of one context in one box. -// Do not claim a result that a pending step can still change, such as "upload failed" while a -// retry is still queued. -export function state(v, facts) { - return JSON.stringify(v); -} - -// The names of the two parallel regions, when the overview draws them: contexts, then progress. -// export const regionNames = ["Network", "What happened to the file"]; - -// The group of one model state, in words the reader sees, such as "Upload done". The chart fills -// the states of one group with one color, and the legend names the groups. TLC still decides -// which states break a rule, and draws them with a thick red border on top of the group color. -// groupColors gives a group a light color. Groups without one take a palette color. -// export function group(v, facts) {} -// export const groupColors = { "Upload done": "#d1f2eb" }; - -// The label for one step. A step that leaves the reader in the same state is not drawn, -// unless the reader takes it on a bug path. "" draws no label, and a numbered bug step shows the action. -const STEPS = { -${actions.map((a) => `\t${a}: ${JSON.stringify(a)},`).join("\n")} -}; -export function step(action, before, after) { - return STEPS[action] ?? action; -} - -// When the step can happen, in plain words, such as "offline or disk full". The chart shows -// "step [words]". "" adds nothing. -// export function guard(action, before, after) {} - -// For a timer step, how many seconds the code waits, from the constant in the code. The chart -// shows "after N s, step". Return nothing for other steps. -// export function timer(action, before, after) {} - -// Why a bad state is bad, in words a product manager reads, such as "the laptop went offline, and -// the app drops an upload it cannot send". No variable, function, or event names. -// Counterexamples that break one rule with one cause are one bug, so two bugs that end on the -// same screen stay apart only if their causes differ. Return null to group by the end state. -// export function cause(v, facts, invariant) {} - -// The developer's words for one model state. With it, each bug is also drawn through these states. -// export function detail(v, facts) {} - -// The legend's words for each status, in place of the defaults. Group entries keep their names. -// export const legend = { works: "Works on main", wrong: "Wrong on main", fixed: "Fixed in the open PR" }; -`; -}; - -async function draw({ now, other, uncited, map, work, out, title, comparing }) { +async function draw({ now, other, uncited, map, work, out, title, comparing, cover }) { + const huge = [now, other].find((m) => m?.dotBytes > MAX_STRING_LENGTH); + const limit = MAX_STRING_LENGTH.toLocaleString("en-US"); + if (huge) + throw new NotPublished( + `${rel(huge.spec)}: TLC's state graph ${huge.stopped ? `passed ${limit} bytes, the most that the command can read, so the command stopped TLC and draws no chart` : `has ${huge.dotBytes.toLocaleString("en-US")} bytes, more than the ${limit} that the command can read, so it draws no chart`}. The rule results above stand. Smaller constants make a smaller graph.`, + ); const shown = published(work, out); const mapping = await import(pathToFileURL(path.resolve(map)).href); - const from = ["--dot", now.dot, "--map", map, "--spec", now.spec]; + const from = ["--dot", now.dot, "--broken", now.dumpOut, "--map", map, "--spec", now.copy]; + for (const m of [now, other]) if (m?.next) from.push("--next", m.next); from.push(...now.traces.flatMap((t) => ["--traces", t.file])); if (other) { - from.push(`--${other.role}-dot`, other.dot, "--spec", other.spec); + from.push(`--${other.role}-dot`, other.dot, `--${other.role}-broken`, other.dumpOut); + from.push("--spec", other.copy); from.push(...other.traces.flatMap((t) => [`--${other.role}-traces`, t.file])); if (other.traces.length) from.push(`--${other.role}-rules`, other.traces.map((t) => t.rule).join(",")); if (other.label !== undefined) from.push("--label", other.label); if (other.listed) from.push("--listed", JSON.stringify(other.listed)); } - if (comparing) from.push("--full-labels", "--flat"); - else if (mapping.readerActions?.length) from.push("--cover"); + if (comparing) from.push("--full-labels", "--flat", "--loops"); const titled = (suffix) => (title ? ["--title", `${title}${suffix}`] : []); const files = { @@ -567,6 +402,7 @@ async function draw({ now, other, uncited, map, work, out, title, comparing }) { [ ...from, ...(comparing ? [] : ["--regions", "--proof", proof, "--regions-all", everyArrow]), + ...(cover ? ["--cover"] : []), ...titled(""), "--highlight", "0", @@ -580,8 +416,46 @@ async function draw({ now, other, uncited, map, work, out, title, comparing }) { map, ); const summary = JSON.parse(fs.readFileSync(files.summary, "utf8")); - if (uncited) - fs.writeFileSync(files.summary, `${JSON.stringify({ ...summary, uncited }, null, 2)}\n`); + const L = summary.second?.label; + const broken = [now, other].filter(Boolean).flatMap((m) => { + const book = ruleBook(m.copy); + const where = m === now ? "" : ` ${other.role === "fixed" ? "with" : "in"} ${L}`; + const nameOf = readerStateNamer(mapping, m.dot); + return m.properties + .filter((p) => p.broken) + .map((p) => ({ + where, + property: { + spec: rel(m.spec), + property: p.rule, + rule: book.get(p.rule)?.sentence ?? null, + repeats: repeatsIn(fs.readFileSync(p.file, "utf8"), nameOf), + counterexample: path.relative(work, p.file), + }, + })); + }); + const brokenProperties = broken.map((b) => b.property); + fs.writeFileSync( + files.summary, + `${JSON.stringify({ ...summary, ...(uncited && { uncited }), brokenProperties }, null, 2)}\n`, + ); + if (brokenProperties.length) { + const lines = broken.map( + ({ where, property: b }) => + `Rule broken${where}: ${[ + (b.rule ?? `\`${b.property}\``).replace(/\.?$/, "."), + loopSentence(b.repeats), + `TLC's counterexample is in ${b.counterexample}.`, + ] + .filter(Boolean) + .join(" ")}`, + ); + const bugs = fs.readFileSync(files.stories, "utf8"); + fs.writeFileSync( + files.stories, + `${bugs}${bugs && "\n"}## Broken rules that the chart does not draw\n\n${lines.join("\n")}\n`, + ); + } const charts = [files.overview]; for (const c of summary.classes) { const views = typeof mapping.detail === "function" ? ["", "-developer", "-layered"] : [""]; @@ -597,8 +471,7 @@ async function draw({ now, other, uncited, map, work, out, title, comparing }) { } } - const { works, wrong, fixed: fixedCount } = summary.chartStatus; - const L = summary.second?.label; + const { works, wrong, fixed: fixedCount, mixed, after, worse } = summary.chartStatus; console.log( `\nChart: ${summary.chartStates} states and ${summary.chartTransitions} transitions from ${summary.modelStates} model states.`, ); @@ -608,58 +481,101 @@ async function draw({ now, other, uncited, map, work, out, title, comparing }) { `It has 2 parallel regions, of ${r.states[0]} and ${r.states[1]} states. Proof: their ${r.all} arrows together make exactly the ${r.flat.states} states and ${r.flat.transitions} labeled steps of ${shown(proof)}, and the ${r.transitions} drawn arrows make only steps among them. ${shown(everyArrow)} has all ${r.all} arrows.`, ); else if (r) console.log(`It has no parallel regions, because ${r.why}.`); - const breaks = `${wrong} ${wrong === 1 ? "breaks" : "break"} a rule`; + const c = summary.cover; + if (c?.drawn === false) console.log(`--cover draws every state, because ${c.why}.`); + else if (c) { + const list = (head, xs) => [ + ...(xs.length ? [head] : []), + ...xs.slice(0, 10).map((x) => ` ${x}`), + ...(xs.length > 10 ? [` and ${xs.length - 10} more in summary.json`] : []), + ]; + const left = [ + ...list("States left out:", c.leftOutStates), + ...list("Transitions left out:", c.leftOutTransitions), + ]; + console.log( + left.length + ? `--cover leaves out ${c.leftOutStates.length} of ${c.states} states and ${c.leftOutTransitions.length} of ${c.transitions} transitions. The counts of states that work or break a rule include the states it leaves out.\n${left.join("\n")}` + : "--cover leaves out no state and no transition.", + ); + } + const verb = (n) => (n === 1 ? "breaks" : "break"); + const breaks = `${wrong} ${verb(wrong)} a rule`; + const withFix = other?.role === "fixed" ? ` with ${L}` : ""; + const partly = + (mixed ? `, ${mixed} ${verb(mixed)} a rule in some cases${withFix}` : "") + + (after ? `, ${after} reached only after a bug${withFix}` : "") + + (worse ? `, ${worse} ${verb(worse)} a rule only${withFix}` : ""); console.log( other?.role === "fixed" - ? `${works} work today, ${wrong} still wrong with ${L}, ${fixedCount} fixed or new with ${L}.\n` + ? `${works} work today, ${wrong} still wrong with ${L}, ${fixedCount} fixed or new with ${L}${partly}.\n` : other?.role === "against" - ? `${works} work, ${breaks}, ${fixedCount} only in ${L}.\n` - : `${works} work, ${breaks}.\n`, + ? `${works} work, ${breaks}, ${fixedCount} only in ${L}${partly}.\n` + : `${works} work, ${breaks}${partly}.\n`, ); + if (summary.mixedStatus.length) + console.log( + `${summary.mixedStatus.length === 1 ? "This state breaks" : "These states break"} a rule in some cases and not in others:\n${summary.mixedStatus.map((t) => ` ${t}`).join("\n")}\n`, + ); if (summary.codeLike.length) console.log( `These labels or causes read like code. Write them in the reader's words in ${rel(map)}:\n${summary.codeLike.map((t) => ` ${t}`).join("\n")}\n`, ); + const stories = fs.readFileSync(files.stories, "utf8").trimEnd(); console.log( summary.classes.length - ? fs.readFileSync(files.stories, "utf8").trimEnd() - : "No state breaks a rule.", + ? stories + : ["No state breaks an INVARIANT rule.", stories].filter(Boolean).join("\n\n"), ); return { files, charts }; } -function render(charts, shown) { +const pngOf = (chart) => chart.replace(/(\.mmd)?$/, ".png"); + +function render(charts, { lib, theme }, shown) { const pngs = []; for (const chart of charts) { - const png = chart.replace(/\.mmd$/, ".png"); - const r = spawnSync( - "npx", - ["-y", "@mermaid-js/mermaid-cli", "-i", chart, "-o", png, "-s", "2"], - { - encoding: "utf8", - }, - ); - if (r.status === 0) pngs.push(png); - else - console.error( - `could not render ${shown(chart)}: ${(r.stderr || r.error?.message || "").trim()}`, - ); + try { + fs.writeFileSync(pngOf(chart), renderPng(lib, fs.readFileSync(chart, "utf8"), { theme })); + pngs.push(pngOf(chart)); + } catch (e) { + console.error(`could not render ${shown(chart)}: ${e.message}`); + } } return pngs; } -function finish({ work, out, inputs }, { files, charts }, ok, png) { +function finish({ work, out, inputs }, { files, charts }, ok, renderer) { unchanged(inputs, out); - const pngs = png ? render(charts, published(work, out)) : []; + const pngs = renderer ? render(charts, renderer, published(work, out)) : []; publish(work, out, inputs); const written = [...charts, ...pngs, files.stories, files.summary] .map((f) => path.basename(f)) .join(", "); console.log(`\nWrote ${written} in ${rel(out)}.`); - if (png && pngs.length < charts.length) return 2; + if (renderer && pngs.length < charts.length) return 2; return ok ? 0 : 1; } +function renderFile(input, flags, { lib, theme }) { + const out = flags.out ?? pngOf(input); + const step = (failed, f) => { + try { + return f(); + } catch (e) { + throw new CannotFinish(`${failed}: ${e.message}`); + } + }; + const src = step(`cannot read ${rel(input)}`, () => fs.readFileSync(input, "utf8")); + const png = step(`could not render ${rel(input)}`, () => renderPng(lib, src, { theme })); + step(`cannot write ${rel(out)}`, () => { + fs.mkdirSync(path.dirname(path.resolve(out)), { recursive: true }); + fs.writeFileSync(out, png); + }); + console.log(`Wrote ${rel(out)}.`); + return 0; +} + const published = (work, out) => (file) => rel(path.join(out, path.relative(work, file))); const OWN = @@ -691,20 +607,6 @@ function outDir(flagged, input, name) { return out; } -const fingerprint = (file) => { - try { - const { ino, ctimeNs } = fs.statSync(file, { bigint: true }); - return `${ino} ${ctimeNs} ${fs.readFileSync(file, "base64")}`; - } catch { - return null; - } -}; - -function snapshot(files, inputs = new Map()) { - for (const f of files) if (f && !inputs.has(f)) inputs.set(f, fingerprint(f)); - return inputs; -} - async function inWorkFolder(body) { const work = fs.mkdtempSync(path.join(os.tmpdir(), "statechart-")); fs.mkdirSync(path.join(work, "tlc")); @@ -712,8 +614,8 @@ async function inWorkFolder(body) { try { return await body(work); } catch (e) { - keep = e instanceof TlcStopped || e instanceof NotPublished; - if (e instanceof NotPublished) + keep = fs.readdirSync(path.join(work, "tlc")).some((f) => f.endsWith(".out")); + if (keep && !(e instanceof TlcStopped)) e.message += ` The files that this run wrote are in ${rel(work)}.`; throw e; } finally { @@ -730,24 +632,61 @@ const moved = (from, to, ...missed) => { if (!["ENOSYS", "EXDEV", "EISDIR"].includes(e.code)) throw e; } // Some filesystems, such as FUSE mounts, cannot rename a folder that has files. They fail with - // ENOSYS, or with EISDIR when the target exists. + // ENOSYS, or with EISDIR when the target exists. They can rename an empty folder. if (!fs.existsSync(from) && missed.includes("ENOENT")) return false; - if (fs.existsSync(to)) { - if (fs.readdirSync(to).length && missed.includes("ENOTEMPTY")) return false; - fs.rmdirSync(to); - } + const claimed = claimedFolder(to, missed); + if (claimed === null) return false; try { - fs.cpSync(from, to, { recursive: true, errorOnExist: true, force: false }); + copyExclusive(from, to); + if (!sameFolder(to, claimed)) + throw new Error("another run replaced the folder during the copy"); } catch (e) { - fs.rmSync(to, { recursive: true, force: true }); + if (sameFolder(to, claimed)) fs.rmSync(to, { recursive: true, force: true }); throw e; + } finally { + fs.closeSync(claimed); } fs.rmSync(from, { recursive: true, force: true }); return true; }; +function claimedFolder(to, missed) { + // Making a folder and opening it take two calls. Another run can replace the folder at `to` + // between them. So the run opens its folder before it moves the folder to `to`. + // A filesystem can reuse a removed folder's inode number. Holding this folder open keeps its + // number, so a folder that another run makes in its place cannot match it. + const mine = fs.mkdtempSync(path.join(path.dirname(to), `.${path.basename(to)}-`)); + const claimed = fs.openSync(mine, "r"); + try { + fs.renameSync(mine, to); + return claimed; + } catch (e) { + fs.closeSync(claimed); + fs.rmdirSync(mine); + if (missed.includes(e.code)) return null; + throw e; + } +} + +function sameFolder(at, claimed) { + const now = fs.statSync(at, { throwIfNoEntry: false }); + const made = fs.fstatSync(claimed); + return now?.dev === made.dev && now?.ino === made.ino; +} + +function copyExclusive(from, to) { + for (const entry of fs.readdirSync(from, { withFileTypes: true })) { + const source = path.join(from, entry.name); + const target = path.join(to, entry.name); + if (entry.isDirectory()) { + fs.mkdirSync(target); + copyExclusive(source, target); + } else fs.copyFileSync(source, target, fs.constants.COPYFILE_EXCL); + } +} + function unchanged(inputs, out) { - const changed = [...inputs].find(([f, was]) => fingerprint(f) !== was)?.[0]; + const changed = inputs.changed(); if (changed) throw new NotPublished( `${rel(changed)} changed while this run read it, so the run left ${rel(out)} as it was. Run the command again.`, @@ -806,7 +745,7 @@ function secondCfg(cfg, flags) { return flags.against && (flags["against-config"] ?? cfgOf(flags.against)); } -function secondModel(input, cfg, { fixed, against, label }) { +function secondModel(input, cfg, { fixed, against, label }, inputs) { if (fixed && against) throw new CannotFinish(`use --fixed or --against, not both. ${more("from-spec")}`); if (!fixed && !against) { @@ -816,19 +755,16 @@ function secondModel(input, cfg, { fixed, against, label }) { ); } const [role, spec] = fixed ? ["fixed", fixed] : ["against", against]; - const name = path.basename(input, ".tla"); - if (path.basename(spec, ".tla") === name) - throw new CannotFinish( - `${rel(spec)} needs its own file and module name, such as ${name}${fixed ? "Fixed" : "Agreed"}.tla.`, - ); - return { role, label, model: readModel(spec, cfg, `--${role}-config`) }; + if (path.resolve(spec) === path.resolve(input)) + throw new CannotFinish(`--${role} needs a file other than the spec, ${rel(input)}.`); + return { role, label, model: readModel(spec, cfg, `--${role}-config`, inputs) }; } function warnUnchecked(now, other) { const unchecked = now.rules.filter((r) => !other.rules.includes(r)); if (unchecked.length) console.error( - `${rel(other.cfgFile)} does not check ${unchecked.join(", ")}, so the stories cannot say whether ${path.basename(other.spec)} keeps ${unchecked.length === 1 ? "it" : "them"}.`, + `${rel(other.cfgFile)} does not check ${unchecked.join(", ")}, so the stories cannot say whether ${rel(other.spec)} keeps ${unchecked.length === 1 ? "it" : "them"}.`, ); } @@ -860,27 +796,28 @@ function warnUnlisted(now, other) { function warnUncited(models) { return Object.fromEntries( - models.map(({ spec, traces, properties }) => { - const found = uncitedConditions(fs.readFileSync(spec, "utf8")); + models.map(({ spec, copy, traces, properties }) => { + const found = uncitedConditions(fs.readFileSync(copy, "utf8")); const held = [...traces, ...properties].filter((t) => !t.broken).map((t) => t.rule); const short = (s) => (s.length > 90 ? `${s.slice(0, 87)}...` : s); if (found.length && held.length) console.error( `${rel(spec)}: ${joinAnd(held)} ${held.length === 1 ? "holds" : "hold"}, but a rule can also hold because the model waits for a condition that the code does not check. No comment gives the file and line of the code that checks ${found.length === 1 ? "this condition" : "these conditions"}:\n${found.map((c) => ` ${rel(spec)}:${c.line} ${c.action}: ${short(c.condition)}`).join("\n")}\nPut that file and line in a comment on each condition, as in \\* pay.ts:120, or remove the condition when no code checks it.`, ); - return [path.basename(spec), found]; + return [rel(spec), found]; }), ); } -async function fromSpec(input, { "expect-broken": expected, ...flags }) { +async function fromSpec(input, { "expect-broken": expected, title, cover, ...flags }, renderer) { const name = path.basename(input, ".tla"); const map = flags.map ?? path.join(path.dirname(input), `${name}.map.mjs`); const cfg = flags.config ?? cfgOf(input); const otherCfg = secondCfg(cfg, flags); - const inputs = snapshot([input, cfg, map, flags.fixed ?? flags.against, otherCfg]); - const second = secondModel(input, otherCfg, flags); - const model = readInput(input, cfg, expected); + const inputs = new Inputs().add([input, cfg, map, flags.fixed ?? flags.against, otherCfg]); + const second = secondModel(input, otherCfg, flags, inputs); + const listed = second?.role === "against" ? listedBreaks(second.model.spec) : undefined; + const model = readInput(input, cfg, expected, inputs); const out = outDir(flags.out, input, name); if (!fs.existsSync(map)) { fs.writeFileSync(map, SCAFFOLD(name, input)); @@ -888,12 +825,12 @@ async function fromSpec(input, { "expect-broken": expected, ...flags }) { } const tools = findTools(); return inWorkFolder(async (work) => { - const now = check(tools, model, { work, out }, expected); + const now = await check(tools, model, { work, out }, expected); const other = second && { role: second.role, label: second.label, - listed: second.role === "against" ? listedBreaks(second.model.spec) : undefined, - ...check(tools, second.model, { work, out }, expected), + listed, + ...(await check(tools, second.model, { work, out, role: second.role }, expected)), }; let uncited; if (second) { @@ -901,12 +838,12 @@ async function fromSpec(input, { "expect-broken": expected, ...flags }) { if (second.role === "against") warnUnlisted(now, other); uncited = warnUncited(second.role === "fixed" ? [now, other] : [now]); } - const drawn = await draw({ now, other, uncited, map, work, out, title: flags.title }); - return finish({ work, out, inputs }, drawn, rulesAsExpected(now, other, expected), flags.png); + const drawn = await draw({ now, other, uncited, map, work, out, title, cover }); + return finish({ work, out, inputs }, drawn, rulesAsExpected(now, other, expected), renderer); }); } -async function fromChart(input, { "expect-broken": expected, ...flags }) { +async function fromChart(input, { "expect-broken": expected, ...flags }, renderer) { const name = flags.module ?? path @@ -916,23 +853,28 @@ async function fromChart(input, { "expect-broken": expected, ...flags }) { throw new CannotFinish(`cannot make a TLA+ module name from ${input}; add --module NAME`); const spec = path.join(path.dirname(input), `${name}.tla`); const map = spec.replace(/\.tla$/, ".map.mjs"); - const inputs = snapshot([input]); + const inputs = new Inputs().add([input]); if (fs.existsSync(spec)) { console.log(`Kept ${rel(spec)}. Delete it to write the spec again from ${rel(input)}.`); } else { script("chart-to-tla.mjs", [input, "--module", name], input); console.log(`Wrote ${name}.tla, ${name}.cfg, and ${name}.map.mjs from ${rel(input)}.`); } - snapshot([spec, cfgOf(spec), map], inputs); + inputs.add([spec, cfgOf(spec), map]); const tools = findTools(); - const model = readInput(spec, cfgOf(spec), expected); + const model = readInput(spec, cfgOf(spec), expected, inputs); const out = outDir(flags.out, input, name); return inWorkFolder(async (work) => { - const now = check(tools, model, { work, out }, expected); + const now = await check(tools, model, { work, out }, expected); const drawn = await draw({ now, map, work, out, title: flags.title, comparing: true }); - const compared = script("compare-charts.mjs", [input, drawn.files.overview], input, { - okExits: [0, 1], - }); + const compared = script( + "compare-charts.mjs", + [inputs.copyOf(input), drawn.files.overview], + input, + { + okExits: [0, 1], + }, + ); if (compared.status === 0) console.log(`\nThe spec draws ${rel(input)} back unchanged.`); else console.log( @@ -942,7 +884,7 @@ async function fromChart(input, { "expect-broken": expected, ...flags }) { { work, out, inputs }, drawn, rulesAsExpected(now, null, expected) && compared.status === 0, - flags.png, + renderer, ); }); } @@ -953,7 +895,7 @@ try { console.log(help); process.exit(0); } - process.exit(await COMMANDS[command].run(input, flags)); + process.exit(await COMMANDS[command].run(input, flags, await rendererFor(command, flags))); } catch (e) { if (!(e instanceof CannotFinish)) throw e; console.error(e.message); diff --git a/harness/pstack/claude-code/skills/statechart/scripts/test/band.mmd b/harness/pstack/claude-code/skills/statechart/scripts/test/band.mmd new file mode 100644 index 000000000..83af58601 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/test/band.mmd @@ -0,0 +1,15 @@ +--- +title: "An edge into a group from above" +--- +stateDiagram-v2 + state "Outside" as a + state "A group" as G { + state "Inside one" as i1 + state "Inside two" as i2 + i1 --> i2 : go + } + state "After" as b + [*] --> a + a --> G : into the group + G --> b : out of the group + a --> i2 : straight to a child diff --git a/harness/pstack/claude-code/skills/statechart/scripts/test/regions.mmd b/harness/pstack/claude-code/skills/statechart/scripts/test/regions.mmd new file mode 100644 index 000000000..d241e24ff --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/test/regions.mmd @@ -0,0 +1,62 @@ +--- +title: "Parcel and payment" +config: + markdownAutoWrap: false + state: + useMaxWidth: false +--- +stateDiagram-v2 + classDef plain fill:#f4f6f7,stroke:#566573,color:#1b2631 + classDef plainWrong fill:#f4f6f7,stroke:#c0392b,stroke-width:4px,color:#7b241c + classDef group0 fill:#d6eaf8,stroke:#6b757c,color:#1b2631 + classDef group0Wrong fill:#d6eaf8,stroke:#c0392b,stroke-width:4px,color:#7b241c + classDef group1 fill:#fdebd0,stroke:#7f7668,color:#1b2631 + state "Legend" as legend { + state "Card on file" as legendGroup0 + state "No card yet" as legendGroup1 + state "Breaks a rule" as legendWrong + } + class legendGroup0 group0 + class legendGroup1 group1 + class legendWrong plainWrong + state "At the same time" as regions { + state "The parcel" as region0 { + [*] --> r0s0 + state "Label printed" as r0s0 + state "In transit" as r0s1 + state "Out for delivery" as r0s2 + state "Left at the front door
without a signature" as r0s3 + state "Held at the depot after
a missed delivery" as r0s4 + state "Sent back to the shop" as r0s5 + r0s0 --> r0s1 : the courier scans the label
at the depot + r0s1 --> r0s2 : the van leaves the depot with
the parcel on board + r0s2 --> r0s3 : the courier leaves the parcel
at the door [when charged
once] + r0s2 --> r0s4 : nobody answers the door
before the van moves on + r0s4 --> r0s2 : the depot books a second
delivery for the next day + r0s4 --> r0s5 : the parcel waits at the depot
for more than fourteen days + } + -- + state "Payment" as region1 { + [*] --> r1s0 + state "No card on file" as r1s0 + state "Card saved, waiting
for the charge" as r1s1 + state "Charged once" as r1s2 + state "Charged twice for
one parcel" as r1s3 + state "Refund sent while the
parcel was on the doorstep" as r1s4 + r1s0 --> r1s1 : you save a card on the
checkout page + r1s1 --> r1s2 : the bank approves the charge
for the parcel + r1s2 --> r1s3 : the payment service retries
a charge that went through + r1s2 --> r1s4 : the shop sends a refund before
the parcel comes back [when
left at the front door] + } + } + class r0s0 group1 + class r0s1 plain + class r0s2 plain + class r0s3 plainWrong + class r0s4 plain + class r0s5 plain + class r1s0 group1 + class r1s1 group0 + class r1s2 group0 + class r1s3 group0Wrong + class r1s4 group0Wrong diff --git a/harness/pstack/claude-code/skills/statechart/scripts/test/statechart.test.mjs b/harness/pstack/claude-code/skills/statechart/scripts/test/statechart.test.mjs index 72782e679..543fe8d67 100644 --- a/harness/pstack/claude-code/skills/statechart/scripts/test/statechart.test.mjs +++ b/harness/pstack/claude-code/skills/statechart/scripts/test/statechart.test.mjs @@ -1,37 +1,58 @@ import assert from "node:assert/strict"; +import { constants } from "node:buffer"; import { spawn, spawnSync } from "node:child_process"; import { + cpSync, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, + renameSync, rmSync, + statSync, + symlinkSync, + utimesSync, writeFileSync, } from "node:fs"; import { tmpdir } from "node:os"; -import { delimiter, dirname, join } from "node:path"; +import { basename, delimiter, dirname, join, posix, win32 } from "node:path"; import { test } from "node:test"; import { setTimeout as sleep } from "node:timers/promises"; -import { fileURLToPath } from "node:url"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { crc32, inflateSync } from "node:zlib"; import { - findTlc, + checkedOnlyInInitialStates, + groupClassDef, parseDot, parseMermaid, parseTraces, parseValue, ruleSentences, stateKey, + STATUS_CLASSES, tlcOutputWarning, uncitedConditions, } from "../lib.mjs"; +import { + CACHE_NAME, + cacheDir, + fcMatch, + fontMeasurer, + loadRenderer, + renderChart, + THEMES, +} from "../render.mjs"; +import { ruleBook } from "../rule-book.mjs"; +import { copyPath } from "../snapshot.mjs"; +import { findTlc, JAVA_TOOL, TLC_TOOL } from "../tlc-tools.mjs"; const here = dirname(fileURLToPath(import.meta.url)); const scripts = dirname(here); const fixture = (name) => readFileSync(join(here, name), "utf8"); const edges = (r) => r.transitions.map((t) => `${t.from} -> ${t.to} : ${t.label}`).sort(); -const node = (script, args, cwd, env = {}) => { - const r = spawnSync(process.execPath, [join(scripts, script), ...args], { +const node = (script, args, cwd, env = {}, at = scripts) => { + const r = spawnSync(process.execPath, [join(at, script), ...args], { cwd, encoding: "utf8", env: { ...process.env, ...env }, @@ -40,6 +61,17 @@ const node = (script, args, cwd, env = {}) => { r.stdout = r.stdout?.replace(/^( {2}TLC ran \d+ times? in )\d+\.\d s\.$/gm, "$1N s."); return r; }; +const statusOf = (chart) => { + const names = new Map( + [...chart.matchAll(/^ {2}state "([^"]+)" as (s\d+)$/gm)].map((m) => [ + m[2], + m[1].replace(//g, " "), + ]), + ); + return Object.fromEntries( + [...chart.matchAll(/^ {2}class (s\d+) (\w+)$/gm)].map((m) => [names.get(m[1]), m[2]]), + ); +}; test("a transition on a composite leaves from every state inside it", () => { const { regions } = parseMermaid(fixture("upload.mmd")); @@ -113,6 +145,283 @@ Bare == TRUE assert.equal(stateKey({ b: 1, a: { d: 2, c: [3] } }), stateKey({ a: { c: [3], d: 2 }, b: 1 })); }); +test("a rule defined in a module that the spec extends, directly or through another module, keeps its sentence and the step that breaks it", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "LampVars.tla"), + `---- MODULE LampVars ---- +VARIABLES light, wire +==== +`, + ); + writeFileSync( + join(dir, "LampRules.tla"), + `---- MODULE LampRules ---- +EXTENDS LampVars +\\* The light never comes on over a shorted wire. +SafeLight == wire = "Short" => light # "On" +==== +`, + ); + writeFileSync( + join(dir, "LampSteps.tla"), + `---- MODULE LampSteps ---- +EXTENDS LampRules +Short == wire = "Ok" /\\ wire' = "Short" /\\ UNCHANGED light +SwitchOn == light = "Off" /\\ light' = "On" /\\ UNCHANGED wire +==== +`, + ); + writeFileSync( + join(dir, "Lamp.tla"), + `---- MODULE Lamp ---- +EXTENDS Naturals, + LampSteps +Init == light = "Off" /\\ wire = "Ok" +Spec == Init /\\ [][Short \\/ SwitchOn]_<> +==== +`, + ); + const dotNode = (id, light, wire, init) => + `${id} [label="/\\\\ light = \\"${light}\\"\\n/\\\\ wire = \\"${wire}\\""${init ? ",style = filled" : ""}]`; + writeFileSync( + join(dir, "Lamp.dot"), + `strict digraph DiskGraph { +${dotNode(1, "Off", "Ok", true)} +${dotNode(2, "Off", "Short")} +${dotNode(3, "On", "Short")} +${dotNode(4, "On", "Ok")} +1 -> 2 [label="Short"]; +2 -> 3 [label="SwitchOn"]; +1 -> 4 [label="SwitchOn"]; +4 -> 3 [label="Short"]; +}`, + ); + writeFileSync( + join(dir, "Lamp.out"), + `Error: Invariant SafeLight is violated. +Error: The behavior up to this point is: +State 1: +/\\ light = "Off" +/\\ wire = "Ok" + +State 2: +/\\ light = "Off" +/\\ wire = "Short" + +State 3: +/\\ light = "On" +/\\ wire = "Short" + +${done}`, + ); + writeFileSync( + join(dir, "lamp.map.mjs"), + 'export const state = (v) => v.light;\nexport const step = (a) => ({ Short: "the wire shorts", SwitchOn: "you switch it on" })[a];\n', + ); + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "Lamp.dot", "--map", "lamp.map.mjs", "--traces", "Lamp.out"], + ...["--spec", "Lamp.tla", "--story", "story.md", "--out", "chart.mmd"], + ], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const story = readFileSync(join(dir, "story.md"), "utf8"); + assert.match(story, /^Rule broken: The light never comes on over a shorted wire\.$/m); + assert.match(story, /^1\. The wire shorts\. Now: Off\n2\. You switch it on\. Now: On$/m); +}); + +const lampRules = `---- MODULE LampRules ---- +VARIABLES light, wire +\\* The light never comes on over a shorted wire. +SafeLight == wire = "Short" => light # "On" +\\* The wire never shorts. +NeverShorts == [](wire # "Short") +==== +`; +const lampSteps = `Init == lamp = "Off" /\\ cable = "Ok" +Short == cable = "Ok" /\\ cable' = "Short" /\\ UNCHANGED lamp +SwitchOn == lamp = "Off" /\\ lamp' = "On" /\\ UNCHANGED cable +Spec == Init /\\ [][Short \\/ SwitchOn]_<> +==== +`; +const lampMap = + 'export const state = (v) => v.lamp;\nexport const step = (a) => ({ Short: "the wire shorts", SwitchOn: "you switch it on" })[a];\n'; + +test("a rule that the spec pulls in through an unnamed INSTANCE with substitutions keeps its sentence and the step that breaks it", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "LampRules.tla"), lampRules); + writeFileSync( + join(dir, "Lamp.tla"), + `---- MODULE Lamp ---- +VARIABLES lamp, cable +INSTANCE LampRules + WITH light <- lamp, + wire <- cable +${lampSteps}`, + ); + const dotNode = (id, lamp, cable, init) => + `${id} [label="/\\\\ cable = \\"${cable}\\"\\n/\\\\ lamp = \\"${lamp}\\""${init ? ",style = filled" : ""}]`; + writeFileSync( + join(dir, "Lamp.dot"), + `strict digraph DiskGraph { +${dotNode(1, "Off", "Ok", true)} +${dotNode(2, "Off", "Short")} +${dotNode(3, "On", "Short")} +${dotNode(4, "On", "Ok")} +1 -> 2 [label="Short"]; +2 -> 3 [label="SwitchOn"]; +1 -> 4 [label="SwitchOn"]; +4 -> 3 [label="Short"]; +}`, + ); + writeFileSync( + join(dir, "Lamp.out"), + `Error: Invariant SafeLight is violated. +Error: The behavior up to this point is: +State 1: +/\\ cable = "Ok" +/\\ lamp = "Off" + +State 2: +/\\ cable = "Short" +/\\ lamp = "Off" + +State 3: +/\\ cable = "Short" +/\\ lamp = "On" + +${done}`, + ); + writeFileSync(join(dir, "lamp.map.mjs"), lampMap); + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "Lamp.dot", "--map", "lamp.map.mjs", "--traces", "Lamp.out"], + ...["--spec", "Lamp.tla", "--story", "story.md", "--out", "chart.mmd"], + ], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const story = readFileSync(join(dir, "story.md"), "utf8"); + assert.match(story, /^Rule broken: The light never comes on over a shorted wire\.$/m); + assert.match(story, /^1\. The wire shorts\. Now: Off\n2\. You switch it on\. Now: On$/m); +}); + +test("rules reached through nested EXTENDS and INSTANCE keep their sentences and read the spec's variables behind each substitution", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const module = (name, body) => + writeFileSync(join(dir, `${name}.tla`), `---- MODULE ${name} ----\n${body}\n====\n`); + module("Wire", `VARIABLE wire\n\\* The wire is never cut.\nIntact == wire # "Cut"`); + module("Socket", `VARIABLE power\n\\* The socket always has power.\nLive == power = "On"`); + module("Count", `VARIABLE v\n\\* The count never goes negative.\nOk == v >= 0`); + module( + "LampRules", + `EXTENDS Naturals, Wire\nVARIABLE light\nPlug == INSTANCE Socket WITH power <- light\nLOCAL INSTANCE Count WITH v <- light\n\\* The light never comes on over a shorted wire.\nSafeLight == wire = "Short" => light # "On"`, + ); + module( + "Lamp", + `EXTENDS TLC +VARIABLES lamp, wire +shown == lamp +INSTANCE LampRules WITH light <- shown +Rules == INSTANCE + LampRules WITH light <- lamp, wire <- lamp +Quoted == INSTANCE Wire WITH wire <- IF lamp = "Cut<-" THEN wire ELSE "Cut" +Plain == Intact +Chain == Plain +Safe == Rules!SafeLight +Ok == wire # "Short" +\\* Its own words. +Worded == Rules!SafeLight`, + ); + const book = ruleBook(join(dir, "Lamp.tla")); + const reads = (rule) => [...book.get(rule).reads].sort(); + assert.deepEqual(reads("SafeLight"), ["lamp", "wire"]); + assert.deepEqual(reads("Intact"), ["wire"]); + assert.deepEqual(reads("Plug!Live"), ["lamp"]); + assert.deepEqual(reads("Rules!SafeLight"), ["lamp"]); + assert.deepEqual(reads("Worded"), ["lamp"]); + assert.deepEqual(reads("Quoted!Intact"), ["lamp", "wire"]); + assert.deepEqual(reads("Ok"), ["wire"]); + const rules = [ + "SafeLight", + "Rules!SafeLight", + "Plug!Live", + "Plain", + "Chain", + "Safe", + "Ok", + "Worded", + ]; + assert.deepEqual( + rules.map((rule) => book.get(rule).sentence), + [ + "The light never comes on over a shorted wire.", + "The light never comes on over a shorted wire.", + "The socket always has power.", + "The wire is never cut.", + "The wire is never cut.", + "The light never comes on over a shorted wire.", + undefined, + "Its own words.", + ], + ); +}); + +test("a LOCAL INSTANCE or LOCAL definition stays inside the module that declares it, and a public name that starts with LOCAL does not", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const module = (name, body) => + writeFileSync(join(dir, `${name}.tla`), `---- MODULE ${name} ----\n${body}\n====\n`); + module("Glow", `VARIABLE g\n\\* The glow is bright.\nOk == g = "Bright"`); + module("Dim", `VARIABLE d\n\\* The glow is dim.\nOk == d = "Dim"`); + module( + "Base", + `VARIABLE b\nLOCAL INSTANCE Dim WITH d <- b\nLOCAL Lit == b = "On"\n\\* The base is lit or dim.\nShown == Lit \\/ Ok\n\\* The base is never off.\nLOCALOn == b # "Off"`, + ); + module("Room", `EXTENDS Base\nVARIABLE r\nINSTANCE Glow WITH g <- r\nHall == INSTANCE Base`); + const book = ruleBook(join(dir, "Room.tla")); + assert.deepEqual(book.get("Ok").sentence, "The glow is bright."); + assert.deepEqual([...book.get("Ok").reads], ["r"]); + assert.deepEqual(book.get("Shown").sentence, "The base is lit or dim."); + assert.deepEqual([...book.get("Shown").reads], ["b"]); + assert.deepEqual([...book.get("Hall!Shown").reads], ["b"]); + assert.deepEqual(book.get("LOCALOn")?.sentence, "The base is never off."); + assert.deepEqual(book.get("Hall!LOCALOn")?.sentence, "The base is never off."); + assert.deepEqual( + ["Lit", "Hall!Lit", "Hall!Ok"].filter((rule) => book.has(rule)), + [], + ); +}); + +test("a rule that reads a CONSTANT reads the spec's variables that WITH puts in its place", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const module = (name, body) => + writeFileSync(join(dir, `${name}.tla`), `---- MODULE ${name} ----\n${body}\n====\n`); + module( + "Limit", + `EXTENDS Naturals\nCONSTANTS Max, Fits(_, _)\nVARIABLE level\n\\* The level stays under the cap.\nUnder == Fits(level, Max)`, + ); + module("Capped", `EXTENDS Limit\nVARIABLE spare\nRoom == spare + level <= Max`); + module( + "Tank", + `EXTENDS Naturals +VARIABLES tank, cap, gap +Gauge == INSTANCE Limit WITH level <- tank, Max <- cap, Fits <- LAMBDA a, b : a <= b +Fixed == INSTANCE Limit WITH level <- tank, Max <- 3, Fits <- LAMBDA a, b : a <= b +Shell == INSTANCE Capped WITH level <- tank, spare <- gap, Max <- cap + 1, Fits <- LAMBDA a, b : a <= b`, + ); + const book = ruleBook(join(dir, "Tank.tla")); + const reads = (rule) => [...book.get(rule).reads].sort(); + assert.deepEqual(reads("Gauge!Under"), ["cap", "tank"]); + assert.deepEqual(reads("Fixed!Under"), ["tank"]); + assert.deepEqual(reads("Shell!Under"), ["cap", "tank"]); + assert.deepEqual(reads("Shell!Room"), ["cap", "gap", "tank"]); + assert.deepEqual(book.get("Gauge!Under").sentence, "The level stays under the cap."); +}); + test("an action's condition on a variable that the action does not change needs a file and line", () => { const found = uncitedConditions(`---- MODULE Pay ---- CONSTANT MaxAge @@ -145,6 +454,78 @@ Next == Tick \\/ Pay \\/ Fill ]); }); +test("a rule reads the variables its helpers read, and not the ones its actions only change or leave unchanged", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Door.tla"), + `---- MODULE Door ---- +VARIABLES door, lock, bell +rest == <> +Locked == lock = "on" +NeverOpenWhileLocked == ~(door = "open" /\\ Locked) +Open == door = "shut" /\\ door' = "open" /\\ bell' = TRUE /\\ UNCHANGED lock +Close == door = "open" /\\ door' = "shut" /\\ UNCHANGED rest +System == Open \\/ Close +Settled == ~ENABLED System +====`, + ); + const book = ruleBook(join(dir, "Door.tla")); + assert.deepEqual([...book.get("NeverOpenWhileLocked").reads].sort(), ["door", "lock"]); + assert.deepEqual([...book.get("Settled").reads], ["door"]); +}); + +test("a property is checked only in the initial states when no definition it reaches has a temporal operator", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "LampBase.tla"), + `---- MODULE LampBase ---- +VARIABLE on +Safe == [](on \\in BOOLEAN) +==== +`, + ); + writeFileSync( + join(dir, "Lamp.tla"), + `---- MODULE Lamp ---- +VARIABLE on +vars == <> +Base == INSTANCE LampBase +StaysOff == on = FALSE +NotEmpty == vars /= <<>> +Always == [](on \\in BOOLEAN) +Eventually == <>on +LeadsTo == on ~> ~on +WhilePlus == StaysOff -+-> StaysOff +Weak == WF_vars(on' = ~on) +Strong == SF_vars(on' = ~on) +Some == \\EE x : StaysOff +Every == \\AA x : StaysOff +ByHelper == Always +FromBase == Base!Safe +==== +`, + ); + const names = [ + "StaysOff", + "NotEmpty", + "Always", + "Eventually", + "LeadsTo", + "WhilePlus", + "Weak", + "Strong", + "Some", + "Every", + "ByHelper", + "FromBase", + "Missing", + ]; + assert.deepEqual( + names.filter((name) => checkedOnlyInInitialStates(join(dir, "Lamp.tla"), name)), + ["StaysOff", "NotEmpty"], + ); +}); + test("the legend is not part of the machine", () => { const { regions } = parseMermaid(`stateDiagram-v2 state "Legend" as legend { @@ -308,10 +689,6 @@ test("TLC output from a run that failed or stopped early is refused", () => { () => tlcOutputWarning("Error: Parsing or semantic analysis failed.\n", "parse.out"), /not a broken rule/, ); - assert.throws( - () => tlcOutputWarning(evaluationError, "eval.out"), - /TLC threw an unexpected exception/, - ); assert.throws( () => tlcOutputWarning( @@ -320,10 +697,6 @@ test("TLC output from a run that failed or stopped early is refused", () => { ), /did not finish/, ); - assert.deepEqual( - parseTraces(evaluationError).map((t) => t.invariant), - ["NoLost", null], - ); }); const tlcThrew = (cls, message) => `Error: Evaluating invariant Small failed. @@ -341,32 +714,50 @@ const smallBroken = "Error: Invariant Small is violated.\nError: The behavior up to this point is:\nState 1: \nx = 0\n\n"; const stoppedEarly = "9 states generated, 9 distinct states found, 4 states left on queue.\n"; -test("a rule that TLC finds broken before TLC throws a Java exception keeps its paths, with a warning", () => { +test("a rule that TLC finds broken keeps its paths after any error that TLC then stops on, with a warning", () => { + const warned = (rule, error) => + `TLC found that ${rule} is broken, and then stopped on an error: ${error}. The paths that TLC printed before it stopped are complete, but TLC did not check every state, so the charts can miss states that break ${rule}. A smaller constant can let TLC finish.`; const aioobe = "java.lang.ArrayIndexOutOfBoundsException: Index 123 out of bounds for length 4"; const oneWorker = `${smallBroken}Error: Invariant Small is violated.\n${tlcThrew( "java.lang.ArrayIndexOutOfBoundsException", "Index 123 out of bounds for length 4", )}\n${stoppedEarly}`; - assert.equal( - tlcOutputWarning(oneWorker, "tlc.out"), - `TLC found that Small is broken, and then failed with ${aioobe}. TLC stopped before it checked every state, so the charts can miss states that break Small. The failure is in TLC, not in the rule. Do not weaken Small because of it. A smaller constant can let TLC finish.`, - ); + assert.equal(tlcOutputWarning(oneWorker, "tlc.out"), warned("Small", aioobe)); const workersInterleaved = `${tlcThrew("java.lang.RuntimeException", aioobe)}${smallBroken}${stoppedEarly}`; - assert.match( - tlcOutputWarning(workersInterleaved, "tlc.out"), - /failed with java\.lang\.ArrayIndexOutOfBoundsException: Index 123/, - ); + assert.equal(tlcOutputWarning(workersInterleaved, "tlc.out"), warned("Small", aioobe)); assert.deepEqual( parseTraces(workersInterleaved).map((t) => t.invariant), ["Small"], ); - assert.throws( - () => - tlcOutputWarning( - `${smallBroken}${tlcThrew("java.lang.RuntimeException", "util.Assert$TLCRuntimeException: Attempted to apply tuple")}${stoppedEarly}`, - "formula.out", - ), - /formula\.out: TLC stopped on an error that is not a broken rule: Evaluating invariant Small failed\./, + const arraycopy = (message) => + `${smallBroken}${tlcThrew("java.lang.ArrayIndexOutOfBoundsException", message)}${stoppedEarly}`; + const negative = "java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -3 is negative"; + assert.equal( + tlcOutputWarning(arraycopy("arraycopy: length -3 is negative"), "tlc.out"), + warned("Small", negative), + ); + assert.equal(tlcOutputWarning(arraycopy(negative), "tlc.out"), warned("Small", negative)); + assert.equal( + tlcOutputWarning( + `${smallBroken}${tlcThrew("java.lang.RuntimeException", "util.Assert$TLCRuntimeException: Attempted to apply tuple")}${stoppedEarly}`, + "formula.out", + ), + warned("Small", "util.Assert$TLCRuntimeException: Attempted to apply tuple"), + ); + assert.equal( + tlcOutputWarning( + `${smallBroken}Error: Invariant Small is violated.\nError: Failed to recover the initial state from its fingerprint.\nThis is probably a TLC bug(4).\n${stoppedEarly}`, + "fingerprint.out", + ), + warned("Small", "Failed to recover the initial state from its fingerprint"), + ); + assert.equal( + tlcOutputWarning(evaluationError, "eval.out"), + warned("NoLost", "TLC threw an unexpected exception"), + ); + assert.deepEqual( + parseTraces(evaluationError).map((t) => t.invariant), + ["NoLost"], ); assert.throws( () => @@ -407,6 +798,220 @@ export const step = (action) => ({ Tap: "you tap Save" })[action] ?? ""; ]); }); +test("a map keys on the bare action name when TLC 1.8.0 writes each step with its parameters", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "seats.dot"), + `strict digraph DiskGraph { +node [shape=box,style=rounded] +nodesep=0.35; +subgraph cluster_graph { +color="white"; +1 [label="seat = <<\\"free\\", \\"free\\">>",style = filled] +1 -> 2 [label="Take(1)",color="black",fontcolor="black"]; +2 [label="seat = <<\\"taken\\", \\"free\\">>",tooltip="seat = <<\\"taken\\", \\"free\\">>"]; +1 -> 3 [label="Take(2)",color="black",fontcolor="black"]; +3 [label="seat = <<\\"free\\", \\"taken\\">>",tooltip="seat = <<\\"free\\", \\"taken\\">>"]; +2 -> 1 [label="Leave(1)",color="black",fontcolor="black"]; +2 -> 4 [label="Take(2)",color="black",fontcolor="black"]; +4 [label="seat = <<\\"taken\\", \\"taken\\">>",tooltip="seat = <<\\"taken\\", \\"taken\\">>"]; +3 -> 4 [label="Take(1)",color="black",fontcolor="black"]; +3 -> 1 [label="Leave(2)",color="black",fontcolor="black"]; +4 -> 3 [label="Leave(1)",color="black",fontcolor="black"]; +4 -> 2 [label="Leave(2)",color="black",fontcolor="black"]; +{rank = same; 1;} +{rank = same; 3;2;} +{rank = same; 4;} +} +}`, + ); + writeFileSync( + join(dir, "seats.map.mjs"), + `export const readerActions = ["Take", "Leave"]; +const taken = (v) => v.seat.filter((s) => s === "taken").length; +export const state = (v) => ["Both free", "One taken", "Both taken"][taken(v)]; +export const step = (action) => ({ Take: "you sit down", Leave: "you get up" })[action] ?? action; +`, + ); + const draw = (...extra) => { + const r = node( + "tlc-to-chart.mjs", + ["--dot", "seats.dot", "--map", "seats.map.mjs", "--out", "chart.mmd", ...extra], + dir, + ); + assert.equal(r.status, 0, r.stderr); + return parseMermaid(readFileSync(join(dir, "chart.mmd"), "utf8")).regions.Main; + }; + assert.deepEqual(edges(draw()), [ + "Both free -> One taken : you sit down", + "Both taken -> One taken : you get up", + "One taken -> Both free : you get up", + "One taken -> Both taken : you sit down", + ]); + assert.deepEqual([...draw("--cover").states].sort(), ["Both free", "Both taken", "One taken"]); +}); + +function layeredHouse({ + extra = [], + edges = "", + state = '(v.x === "attic" ? "Upstairs" : "Downstairs")', + detail = '(v.x === "porch" || v.x === "hall" ? v.x : "")', +} = {}) { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "house.dot"), + `strict digraph DiskGraph { +1 [label="x = \\"start\\"",style = filled] +1 -> 2 [label="Side"]; +2 [label="x = \\"porch\\""]; +1 -> 3 [label="Front"]; +3 [label="x = \\"hall\\""]; +2 -> 4 [label="Arrive"]; +4 [label="x = \\"attic\\""]; +4 -> 1 [label="Leave"]; +${edges} +}`, + ); + writeFileSync( + join(dir, "house.map.mjs"), + `export const readerActions = ["Side", "Front", "Arrive"]; +export const state = (v) => ${state}; +export const detail = (v) => ${detail}; +export const step = (a) => ({ Side: "you take the side door", Front: "you take the front door", Arrive: "you climb the stairs", Leave: "the lights go out" })[a]; +`, + ); + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "house.dot", "--map", "house.map.mjs", "--view", "layered"], + ...["--out", "chart.mmd", ...extra], + ], + dir, + ); + assert.equal(r.status, 0, r.stderr); + return dir; +} + +test("--cover names a layered state with an empty detail by its outer label alone", () => { + const dir = layeredHouse({ extra: ["--cover", "--summary", "summary.json"] }); + const { cover } = JSON.parse(readFileSync(join(dir, "summary.json"), "utf8")); + assert.deepEqual(cover.leftOutStates, ["Upstairs"]); + assert.deepEqual(cover.leftOutTransitions, [ + "Downstairs: porch --> Upstairs: you climb the stairs [Arrive]", + "Upstairs --> Downstairs: the lights go out [Leave]", + ]); +}); + +test("a layered chart draws a state with an empty detail by its outer label alone, outside any box", () => { + const chart = readFileSync(join(layeredHouse(), "chart.mmd"), "utf8"); + assert.doesNotMatch(chart, /state ""/); + assert.match( + chart, + /^ {2}state "Downstairs" as g\d+ \{\n {4}state "hall" as s\d+\n {4}state "porch" as s\d+\n {2}\}$/m, + ); + const plain = /^ {2}state "Downstairs" as (s\d+)$/m.exec(chart)?.[1]; + assert.ok(plain, chart); + assert.match(chart, new RegExp(`^ {2}\\[\\*\\] --> ${plain}$`, "m")); + assert.match(chart, /^ {2}state "Upstairs" as s\d+$/m); +}); + +test("a box's shared step stays on each state when a plain state outside the box has the box's name", () => { + const chart = readFileSync( + join(layeredHouse({ edges: '3 -> 4 [label="Arrive"];' }), "chart.mmd"), + "utf8", + ); + assert.doesNotMatch(chart, /^ {2}g\d+ --> /m); + assert.deepEqual( + flatEdges(chart).map((e) => e.replaceAll("
", " ")), + [ + "Downstairs --> Downstairs: hall : you take the front door [Front]", + "Downstairs --> Downstairs: porch : you take the side door [Side]", + "Downstairs: hall --> Upstairs : you climb the stairs [Arrive]", + "Downstairs: porch --> Upstairs : you climb the stairs [Arrive]", + "Upstairs --> Downstairs : the lights go out [Leave]", + ], + ); +}); + +test("--flat draws a layered chart with no boxes, naming each state by its label and detail", () => { + const chart = readFileSync(join(layeredHouse({ extra: ["--flat"] }), "chart.mmd"), "utf8"); + assert.doesNotMatch(chart, / as g\d+ \{$/m); + assert.deepEqual([...chart.matchAll(/^ {2}state "([^"]+)" as s\d+$/gm)].map((m) => m[1]).sort(), [ + "Downstairs", + "Downstairs: hall", + "Downstairs: porch", + "Upstairs", + ]); +}); + +test("a layered chart and its --cover lists treat a blank state label or detail as absent", () => { + const dir = layeredHouse({ + extra: ["--cover", "--summary", "summary.json"], + state: '({ attic: "Upstairs", hall: " " })[v.x] ?? "Downstairs"', + detail: '(v.x === "porch" || v.x === "hall" ? v.x : " ")', + }); + const chart = readFileSync(join(dir, "chart.mmd"), "utf8"); + assert.doesNotMatch(chart, /state ""/); + assert.match(chart, /^ {2}state "Downstairs" as g\d+ \{\n {4}state "porch" as s\d+\n {2}\}$/m); + assert.deepEqual([...chart.matchAll(/^ {2}state "([^"]+)" as s\d+$/gm)].map((m) => m[1]).sort(), [ + "Downstairs", + "hall", + ]); + const { cover } = JSON.parse(readFileSync(join(dir, "summary.json"), "utf8")); + assert.deepEqual(cover.leftOutStates, ["Upstairs"]); + assert.deepEqual(cover.leftOutTransitions, [ + "Downstairs: porch --> Upstairs: you climb the stairs [Arrive]", + "Upstairs --> Downstairs: the lights go out [Leave]", + ]); +}); + +test("a large graph with a thousand bugs draws its overview and a bug chart in seconds", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const [states, bugs, bound] = [20_000, 1000, 10_000]; + const label = (n, broken) => `/\\\\ n = ${n}\\n/\\\\ broken = ${broken ? "TRUE" : "FALSE"}`; + const dot = []; + for (let i = 1; i <= states; i++) { + dot.push(`${i} [label="${label(i, false)}"${i === 1 ? ",style = filled" : ""}];`); + if (i < states) dot.push(`${i} -> ${i + 1} [label="Tick"];`); + dot.push(`${i} -> ${((i * 7) % states) + 1} [label="Jump"];`); + } + const traces = []; + for (let k = 1; k <= bugs; k++) { + dot.push(`${states + k} [label="${label(k, true)}"];`, `1 -> ${states + k} [label="Break"];`); + traces.push( + `Error: Invariant ${k % 2 ? "Safe" : "Calm"} is violated.\nError: The behavior up to this point is:\nState 1: \n/\\ n = 1\n/\\ broken = FALSE\n\nState 2: \n/\\ n = ${k}\n/\\ broken = TRUE\n`, + ); + } + writeFileSync(join(dir, "big.dot"), `strict digraph DiskGraph {\n${dot.join("\n")}\n}\n`); + writeFileSync(join(dir, "big.out"), `${traces.join("\n")}\n${done}`); + writeFileSync( + join(dir, "big.map.mjs"), + `export const state = (v) => (v.broken ? "Broken" : "Counting"); +export const step = (a) => ({ Tick: "a second passes", Jump: "it jumps", Break: "it breaks" })[a]; +export const cause = (v) => (v.broken ? \`counter \${v.n} broke\` : null); +`, + ); + for (const view of [ + "--highlight 0 --out chart.mmd --story stories.md --summary summary.json", + "--highlight 1 --path-only --out bug-1.mmd", + ]) { + const input = `--dot big.dot --map big.map.mjs --traces big.out ${view}`.split(" "); + const r = spawnSync(process.execPath, [join(scripts, "tlc-to-chart.mjs"), ...input], { + cwd: dir, + encoding: "utf8", + timeout: bound, + }); + assert.equal(r.signal, null, `${view} ran past ${bound / 1000} s`); + assert.equal(r.status, 0, r.stderr); + } + const { classes } = JSON.parse(readFileSync(join(dir, "summary.json"), "utf8")); + assert.equal(classes.length, bugs); + assert.deepEqual( + new Set(classes.map((c) => JSON.stringify([c.lastSteps, c.startsHere]))), + new Set([JSON.stringify([["it breaks"], false])]), + ); +}); + test("chart-to-tla writes one action per label and one invariant per never line", () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); @@ -637,16 +1242,19 @@ FlipTheSwitch == ); }); -test("compare reads a one-region chart as the machine its steps reach, without steps that change nothing", () => { +test("compare reads a one-region chart as the machine its steps reach, with the steps that keep the state", () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); const drawn = "stateDiagram-v2\n [*] --> Idle\n Idle --> Busy : start\n Busy --> Idle : finish\n"; writeFileSync(join(dir, "drawn.mmd"), drawn); writeFileSync(join(dir, "loop.mmd"), `${drawn} Idle --> Idle : refresh\n`); writeFileSync(join(dir, "orphan.mmd"), `${drawn} Lost --> Idle : come back\n`); - const same = node("compare-charts.mjs", ["loop.mmd", "drawn.mmd"], dir); - assert.equal(same.status, 0); - assert.equal(same.stdout, "same\n"); + const loop = node("compare-charts.mjs", ["loop.mmd", "drawn.mmd"], dir); + assert.equal(loop.status, 1); + assert.equal( + loop.stdout, + "transitions only in EXPECTED (a guard in the spec blocks them, or the spec renamed the step):\n Idle --> Idle : refresh\n1 difference(s)\n", + ); const orphan = node("compare-charts.mjs", ["orphan.mmd", "drawn.mmd"], dir); assert.equal(orphan.status, 1); assert.equal( @@ -796,15 +1404,14 @@ export const step = (a) => ({ TapPay: "you tap Pay", TapBack: "you tap Back", Ta assert.equal(node("compare-charts.mjs", ["agreed.mmd", "chart.mmd"], dir).stdout, "same\n"); }); -const NO_TLC = `Cannot find TLC. Download tla2tools.jar from a TLA+ release: - https://github.com/tlaplus/tlaplus/releases -Then set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later). -`; +const NO_TLC = + "Cannot find TLC. It installs with mise, which cannot start: spawnSync mise ENOENT. Install mise (https://mise.jdx.dev), or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later).\n"; const FROM_SPEC_USAGE = - "node statechart.mjs from-spec SPEC.tla [--config FILE] [--fixed FIXED.tla [--fixed-config FILE] | --against AGREED.tla [--against-config FILE]] [--label TEXT] [--map FILE] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png]"; + "node statechart.mjs from-spec SPEC.tla [--config FILE] [--fixed FIXED.tla [--fixed-config FILE] | --against AGREED.tla [--against-config FILE]] [--label TEXT] [--map FILE] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--cover] [--png [--theme NAME]]"; const FROM_CHART_USAGE = - "node statechart.mjs from-chart CHART.mmd [--module NAME] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png]"; -const noTlc = { TLA2TOOLS: "", JAVA: "" }; + "node statechart.mjs from-chart CHART.mmd [--module NAME] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png [--theme NAME]]"; +const RENDER_USAGE = "node statechart.mjs render CHART.mmd [--theme NAME] [--out FILE]"; +const noTlc = { PATH: mkdtempSync(join(tmpdir(), "empty-path-")), TLA2TOOLS: "", JAVA: "" }; test("statechart.mjs writes the spec next to the chart, keeps it on the next run, and stops without TLC", () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); @@ -836,6 +1443,43 @@ test("statechart.mjs writes the spec next to the chart, keeps it on the next run missingJar.stderr, "TLA2TOOLS is /nonexistent/tla2tools.jar, which does not exist.\n", ); + const missingJava = node("statechart.mjs", ["from-spec", "NotesApp.tla"], dir, { + ...noTlc, + JAVA: "/nonexistent/java", + }); + assert.equal(missingJava.status, 2); + assert.equal(missingJava.stderr, "JAVA is /nonexistent/java, which does not exist.\n"); +}); + +const installCommand = (...tools) => + `cd "$(mktemp -d)" && MISE_CEILING_PATHS="$PWD" mise install ${tools.map((t) => `'${t}'`).join(" ")}`; + +test("statechart.mjs shows mise's own error when mise fails before it can look for TLC", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const bin = mkdtempSync(join(tmpdir(), "fake-mise-")); + const said = + "mise ERROR failed to read the global config\nmise ERROR Run with --verbose for more"; + writeFileSync(join(bin, "mise"), `#!/bin/sh\nprintf '%s\\n' "${said}" >&2\nexit 1\n`, { + mode: 0o755, + }); + writeFileSync(join(dir, "Pay.tla"), "---- MODULE Pay ----\n====\n"); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\n"); + writeFileSync(join(dir, "Pay.map.mjs"), "export default {};\n"); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, { + PATH: bin, + TLA2TOOLS: "", + JAVA: "", + }); + assert.equal(r.status, 2); + assert.equal( + r.stderr, + `Cannot find TLC, because mise failed: + mise ERROR failed to read the global config + mise ERROR Run with --verbose for more +The install command: ${installCommand(JAVA_TOOL, TLC_TOOL)} +Fix that, or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later). +`, + ); }); test("statechart.mjs writes no output folder when it refuses the input", () => { @@ -919,17 +1563,11 @@ test("statechart.mjs names the chart it refuses and the argument it does not kno unknown.stderr, "unknown argument --fixed. Run node statechart.mjs from-chart --help for the flags, the output, and the exit codes.\n", ); - const sameName = node("statechart.mjs", ["from-spec", "C.tla", "--fixed", "fix/C.tla"], dir); - assert.equal(sameName.status, 2); - assert.equal( - sameName.stderr, - "fix/C.tla needs its own file and module name, such as CFixed.tla.\n", - ); - const sameAgreed = node("statechart.mjs", ["from-spec", "C.tla", "--against", "a/C.tla"], dir); - assert.equal( - sameAgreed.stderr, - "a/C.tla needs its own file and module name, such as CAgreed.tla.\n", - ); + const sameFile = node("statechart.mjs", ["from-spec", "C.tla", "--fixed", "./C.tla"], dir); + assert.equal(sameFile.status, 2); + assert.equal(sameFile.stderr, "--fixed needs a file other than the spec, C.tla.\n"); + const sameAgreed = node("statechart.mjs", ["from-spec", "C.tla", "--against", "C.tla"], dir); + assert.equal(sameAgreed.stderr, "--against needs a file other than the spec, C.tla.\n"); for (const [extra, stderr] of [ [ ["--fixed", "F.tla", "--against", "A.tla"], @@ -950,25 +1588,45 @@ test("statechart.mjs names the chart it refuses and the argument it does not kno valueless.stderr, "--out needs a value. Run node statechart.mjs from-chart --help for the flags, the output, and the exit codes.\n", ); + for (const [args, stderr] of [ + [ + ["render", "c.mmd", "--theme", "blue"], + "--theme takes light, dark, or classic. Run node statechart.mjs render --help for the flags, the output, and the exit codes.\n", + ], + [ + ["from-chart", "c.mmd", "--theme", "dark"], + "--theme needs --png. Run node statechart.mjs from-chart --help for the flags, the output, and the exit codes.\n", + ], + [ + ["render", "c.mmd", "--png"], + "unknown argument --png. Run node statechart.mjs render --help for the flags, the output, and the exit codes.\n", + ], + [ + ["render"], + `Usage: ${RENDER_USAGE}\nRun node statechart.mjs render --help for the flags, the output, and the exit codes.\n`, + ], + ]) { + const r = node("statechart.mjs", args, dir); + assert.equal(r.status, 2, args.join(" ")); + assert.equal(r.stderr, stderr); + } const bare = node("statechart.mjs", [], dir); assert.equal(bare.status, 2); assert.equal( bare.stderr, `Usage: ${FROM_SPEC_USAGE} ${FROM_CHART_USAGE} + ${RENDER_USAGE} Run node statechart.mjs --help for the flags, the output, and the exit codes. `, ); }); -test("every script prints its help with -h or --help, and points at it for an argument it does not know", () => { +test("statechart.mjs prints its help with -h or --help and points at it for an argument it does not know, and each inner script refuses that argument", () => { const help = { "statechart.mjs": [[], `Usage: ${FROM_SPEC_USAGE}`], "statechart.mjs from-spec": [["from-spec"], `Usage: ${FROM_SPEC_USAGE}`], "statechart.mjs from-chart": [["from-chart", "notes-app.mmd"], `Usage: ${FROM_CHART_USAGE}`], - "chart-to-tla.mjs": [[], "Usage: node chart-to-tla.mjs CHART.mmd --module NAME [--dir OUT]"], - "tlc-to-chart.mjs": [[], "Usage: node tlc-to-chart.mjs --dot FILE --map FILE [options]"], - "compare-charts.mjs": [[], "Usage: node compare-charts.mjs EXPECTED.mmd ACTUAL.mmd"], }; for (const [name, [before, usage]] of Object.entries(help)) { const script = name.split(" ")[0]; @@ -982,7 +1640,8 @@ test("every script prints its help with -h or --help, and points at it for an ar } const top = node("statechart.mjs", ["--help"], here).stdout.split("\n"); assert.equal(top[1], ` ${FROM_CHART_USAGE}`); - assert.equal(top[2], " node statechart.mjs [from-spec|from-chart] --help"); + assert.equal(top[2], ` ${RENDER_USAGE}`); + assert.equal(top[3], " node statechart.mjs [from-spec|from-chart|render] --help"); for (const flag of [ "--fixed FIXED.tla", "--label TEXT", @@ -1003,31 +1662,9 @@ test("every script prints its help with -h or --help, and points at it for an ar ["from-spec", "--nope"], "unknown argument --nope. Run node statechart.mjs from-spec --help for the flags, the output, and the exit codes.\n", ], - [ - "chart-to-tla.mjs", - ["--nope"], - "unknown argument --nope. Run node chart-to-tla.mjs --help for the flags.\n", - ], - [ - "tlc-to-chart.mjs", - ["--nope"], - "unknown argument --nope. Run node tlc-to-chart.mjs --help for the flags.\n", - ], - [ - "tlc-to-chart.mjs", - ["--dot", "a.dot", "--map", "a.map.mjs", "--regions"], - "--regions draws the whole customer chart: use --highlight 0, without --flat or --path-only. Run node tlc-to-chart.mjs --help for the flags.\n", - ], - [ - "tlc-to-chart.mjs", - ["--dot", "a.dot", "--map", "a.map.mjs", "--proof", "p.mmd"], - "--proof writes the regions proof; it needs --regions. Run node tlc-to-chart.mjs --help for the flags.\n", - ], - [ - "compare-charts.mjs", - ["--nope"], - "unknown argument --nope. Run node compare-charts.mjs --help for the flags.\n", - ], + ["chart-to-tla.mjs", ["--nope"], "unknown argument --nope.\n"], + ["tlc-to-chart.mjs", ["--nope"], "unknown argument --nope.\n"], + ["compare-charts.mjs", ["--nope"], "unknown argument --nope.\n"], ]) { const r = node(script, args, here); assert.equal(r.status, 2, script); @@ -1045,6 +1682,124 @@ const tools = (() => { })(); const needsTlc = { skip: tools ? false : "cannot find TLC" }; +const realEntry = pathToFileURL(join(cacheDir(), "entry.mjs")).href; +const lib = await loadRenderer().catch(() => null); +const needsPng = { skip: lib ? false : "cannot install Pele and resvg" }; + +function fakeRenderer() { + const xdg = mkdtempSync(join(tmpdir(), "xdg-")); + const dir = join(xdg, "statechart", CACHE_NAME); + mkdirSync(dir, { recursive: true }); + writeFileSync( + join(dir, "entry.mjs"), + `import { appendFileSync } from "node:fs"; +import { render as real } from ${JSON.stringify(realEntry)}; +export { Resvg } from ${JSON.stringify(realEntry)}; +export function render(text, options) { + const title = JSON.parse(/^title: (.*)$/m.exec(text)?.[1] ?? '""'); + if (process.env.RENDERED) appendFileSync(process.env.RENDERED, \`\${title}\\n\`); + if (title && process.env.UNRENDERED?.split("|").includes(title)) + throw new Error("Pele cannot lay out this chart"); + return real(text, options); +} +`, + ); + return xdg; +} + +const paeth = (a, b, c) => { + const [pa, pb, pc] = [a, b, c].map((v) => Math.abs(a + b - c - v)); + return pa <= pb && pa <= pc ? a : pb <= pc ? b : c; +}; +const FILTERS = [() => 0, (a) => a, (a, b) => b, (a, b) => (a + b) >> 1, paeth]; + +function decodePng(png) { + assert.deepEqual([...png.subarray(0, 8)], [137, 80, 78, 71, 13, 10, 26, 10], "PNG signature"); + const chunks = {}; + for (let at = 8; at < png.length;) { + const size = png.readUInt32BE(at); + const type = png.toString("latin1", at + 4, at + 8); + const crc = crc32(png.subarray(at + 4, at + 8 + size)); + assert.equal(png.readUInt32BE(at + 8 + size), crc, `${type} chunk CRC`); + (chunks[type] ??= []).push(png.subarray(at + 8, at + 8 + size)); + at += 12 + size; + } + const [ihdr] = chunks.IHDR; + const [width, height] = [ihdr.readUInt32BE(0), ihdr.readUInt32BE(4)]; + assert.deepEqual([ihdr[8], ihdr[9], ihdr[12]], [8, 6, 0], "8-bit RGBA, not interlaced"); + const raw = inflateSync(Buffer.concat(chunks.IDAT)); + const stride = width * 4; + const px = Buffer.alloc(stride * height); + for (let y = 0; y < height; y++) { + const row = y * (stride + 1); + const filter = FILTERS[raw[row]]; + assert.ok(filter, `row ${y} has PNG filter ${raw[row]}`); + for (let x = 0; x < stride; x++) { + const a = x < 4 ? 0 : px[y * stride + x - 4]; + const b = y ? px[(y - 1) * stride + x] : 0; + const c = x < 4 || !y ? 0 : px[(y - 1) * stride + x - 4]; + px[y * stride + x] = raw[row + 1 + x] + filter(a, b, c); + } + } + const at = (x, y) => (Math.floor(y) * width + Math.floor(x)) * 4; + return { + width, + height, + pixel: (x, y) => `#${px.subarray(at(x, y), at(x, y) + 3).toString("hex")}`, + }; +} + +const xmlText = (s) => + s + .replace(/</g, "<") + .replace(/>/g, ">") + .replace(/"/g, '"') + .replace(/'/g, "'") + .replace(/&/g, "&"); +const textLines = (text) => { + const spans = [...text.matchAll(/]*>([^<]*)<\/tspan>/g)].map((m) => m[1]); + return (spans.length ? spans : [text.replace(/<[^>]+>/g, "")]) + .map(xmlText) + .filter((l) => l.trim()); +}; +const width = (rect) => +/\swidth="([\d.]+)"/.exec(rect)[1]; + +function labeledBoxes(svg) { + const states = [ + ...svg.matchAll( + /]*>(]*\/>)([\s\S]*?)<\/g>/g, + ), + ].flatMap(([, id, rect, inside]) => { + const text = /]*>[\s\S]*?<\/text>/.exec(inside); + return text + ? [{ kind: "state", id, box: width(rect), size: 16, lines: textLines(text[0]) }] + : []; + }); + const edges = [ + ...svg.matchAll( + /(]*\/>)()/g, + ), + ].map(([, id, rect, text]) => ({ + kind: "edge label", + id, + box: width(rect), + size: 14, + lines: textLines(text), + })); + const regions = [ + ...svg.matchAll( + /(]*\/>)[\s\S]*?()/g, + ), + ].map(([, id, rect, text]) => ({ + kind: "region title", + id, + box: width(rect), + size: 16, + lines: textLines(text), + })); + return [...states, ...edges, ...regions]; +} + function tlc(dir, name) { const r = spawnSync( tools.java, @@ -1069,6 +1824,246 @@ function tlc(dir, name) { return r; } +const FAKE_MISE = `#!/bin/sh +echo "$PWD $*" >> "$MISE_LOG" +d=$PWD +while :; do + grep -qs '^locked = true' "$d/mise.toml" && locked=1 + case ":$MISE_CEILING_PATHS:" in *":$d:"*) break;; esac + [ "$d" = / ] && break + d=$(dirname "$d") +done +case $1 in +install) + if [ -n "$FAIL" ]; then echo "mise ERROR cannot download tla2tools.jar" >&2; exit 1; fi + if [ -n "$locked" ]; then echo "mise ERROR No lockfile URL found for $3 (--locked mode)" >&2; exit 1; fi + touch "$MISE_STATE/installed" + if [ -n "$FAIL_AFTER_INSTALL" ]; then echo "mise ERROR failed to rebuild runtime symlinks: File exists (os error 17)" >&2; exit 1; fi + exit 0;; +where) if [ -e "$MISE_STATE/installed" ]; then echo "$TLAPLUS_HOME"; exit 0; fi;; +which) if [ -e "$MISE_STATE/installed" ]; then echo "$TLC_JAVA"; exit 0; fi;; +esac +echo "mise ERROR $2 is not installed" >&2 +exit 1 +`; + +function outsideRepoWithoutTlc() { + const root = mkdtempSync(join(tmpdir(), "statechart-outside-")); + const at = join(root, "skill", "scripts"); + cpSync(scripts, at, { recursive: true, filter: (f) => !f.startsWith(here) }); + const bin = join(root, "bin"); + const state = join(root, "state"); + const dir = join(root, "work"); + for (const d of [bin, state, dir]) mkdirSync(d); + writeFileSync(join(bin, "mise"), FAKE_MISE, { mode: 0o755 }); + writeFileSync(join(dir, "mise.toml"), '[tools]\njava = "17"\n'); + const log = join(root, "mise.log"); + const env = { + PATH: `${bin}${delimiter}${process.env.PATH}`, + TLA2TOOLS: "", + JAVA: "", + MISE_STATE: state, + MISE_LOG: log, + }; + const calls = () => + existsSync(log) + ? readFileSync(log, "utf8") + .trim() + .split("\n") + .map((line) => { + const [folder, ...args] = line.split(" "); + return { folder, args }; + }) + : []; + return { at, dir, env, calls }; +} + +function assertMiseRanInARemovedEmptyFolder(dir, calls) { + const folders = [...new Set(calls.map((c) => c.folder))]; + assert.equal(folders.length > 0, true); + for (const folder of folders) { + assert.equal(folder.startsWith(dir), false, folder); + assert.equal(existsSync(folder), false, folder); + } +} + +const INSTALLING = "Installing TLC and Java with mise. This happens once.\n"; + +test("statechart.mjs outside the repo installs TLC with mise in an empty folder once and shows mise's error", () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + writeFileSync(join(dir, "Pay.tla"), "---- MODULE Pay ----\n====\n"); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\n"); + writeFileSync(join(dir, "Pay.map.mjs"), "export default {};\n"); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, { ...env, FAIL: "1" }, at); + assert.equal(r.status, 2); + assert.equal( + r.stderr, + `${INSTALLING}Cannot find TLC, and mise could not install it: + mise ERROR cannot download tla2tools.jar +The install command: ${installCommand(JAVA_TOOL, TLC_TOOL)} +Or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later). +`, + ); + const install = ["install", "--quiet", JAVA_TOOL, TLC_TOOL]; + assert.deepEqual( + calls().map((c) => c.args), + [ + ["where", TLC_TOOL], + ["which", "java", "--tool", JAVA_TOOL], + install, + ["where", TLC_TOOL], + ["which", "java", "--tool", JAVA_TOOL], + ], + ); + assertMiseRanInARemovedEmptyFolder(dir, calls()); +}); + +test("statechart.mjs with JAVA set installs only TLC with mise", () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + writeFileSync(join(dir, "Pay.tla"), "---- MODULE Pay ----\n====\n"); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\n"); + writeFileSync(join(dir, "Pay.map.mjs"), "export default {};\n"); + const javaEnv = { ...env, JAVA: process.execPath, FAIL: "1" }; + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, javaEnv, at); + assert.equal(r.status, 2); + assert.equal( + r.stderr, + `Installing TLC with mise. This happens once. +Cannot find TLC, and mise could not install it: + mise ERROR cannot download tla2tools.jar +The install command: ${installCommand(TLC_TOOL)} +Or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later). +`, + ); + assert.deepEqual( + calls().map((c) => c.args[0]), + ["where", "install", "where"], + ); + assertMiseRanInARemovedEmptyFolder(dir, calls()); +}); + +test( + "statechart.mjs outside the repo installs a missing TLC with mise once, and leaves the mise files next to the spec alone", + needsTlc, + () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + const tlcEnv = { ...env, TLAPLUS_HOME: dirname(tools.jar), TLC_JAVA: tools.java }; + writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); + const first = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, tlcEnv, at); + assert.equal(first.status, 0, first.stderr); + assert.equal(first.stderr, INSTALLING); + assert.match(first.stdout, /\nThe spec draws upload\.mmd back unchanged\.\n/); + const again = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, tlcEnv, at); + assert.equal(again.status, 0, again.stderr); + assert.equal(again.stderr, ""); + assert.equal(calls().filter((c) => c.args[0] === "install").length, 1); + assertMiseRanInARemovedEmptyFolder(dir, calls()); + }, +); + +test( + "statechart.mjs uses TLC that mise installed even when mise install then exits non-zero", + needsTlc, + () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + const tlcEnv = { + ...env, + TLAPLUS_HOME: dirname(tools.jar), + TLC_JAVA: tools.java, + FAIL_AFTER_INSTALL: "1", + }; + writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); + const r = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, tlcEnv, at); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, INSTALLING); + assert.match(r.stdout, /\nThe spec draws upload\.mmd back unchanged\.\n/); + assert.equal(calls().filter((c) => c.args[0] === "install").length, 1); + }, +); + +test( + "statechart.mjs installs TLC with mise inside a repo whose mise.toml sets locked = true", + needsTlc, + () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + writeFileSync(join(dir, "mise.toml"), "[settings]\nlocked = true\n"); + const specs = join(dir, "specs"); + mkdirSync(specs); + writeFileSync(join(specs, "upload.mmd"), fixture("upload.mmd")); + const tlcEnv = { ...env, TLAPLUS_HOME: dirname(tools.jar), TLC_JAVA: tools.java }; + const r = node("statechart.mjs", ["from-chart", "upload.mmd"], specs, tlcEnv, at); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, INSTALLING); + assert.match(r.stdout, /\nThe spec draws upload\.mmd back unchanged\.\n/); + assertMiseRanInARemovedEmptyFolder(dir, calls()); + }, +); + +test( + "statechart.mjs finds TLC with the real mise when an untrusted mise.toml is next to the spec and above TMPDIR", + { + skip: + !tools || process.env.TLA2TOOLS + ? "needs TLC from mise, not from TLA2TOOLS" + : spawnSync("mise", ["--version"]).error + ? "needs mise" + : false, + }, + () => { + const root = mkdtempSync(join(tmpdir(), "statechart-outside-")); + const at = join(root, "scripts"); + cpSync(scripts, at, { recursive: true, filter: (f) => !f.startsWith(here) }); + const dir = join(root, "work"); + const tmp = join(dir, "tmp"); + mkdirSync(tmp, { recursive: true }); + writeFileSync(join(dir, "mise.toml"), '[tools]\njava = "17"\n\n[env]\nSTATECHART = "1"\n'); + writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); + const env = { JAVA: "", TMPDIR: tmp }; + const r = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, env, at); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, ""); + assert.match(r.stdout, /\nThe spec draws upload\.mmd back unchanged\.\n/); + assert.deepEqual(readdirSync(dir).sort(), [ + "Upload-statechart", + "Upload.cfg", + "Upload.map.mjs", + "Upload.tla", + "mise.toml", + "tmp", + "upload.mmd", + ]); + }, +); + +test("statechart.mjs shows why java cannot start TLC", needsTlc, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); + const setVars = + "Fix that, or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later).\n"; + const folder = dirname(tools.jar); + const notJar = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, { + TLA2TOOLS: folder, + JAVA: tools.java, + }); + assert.equal(notJar.status, 2); + assert.equal( + notJar.stderr, + `cannot run TLC with ${tools.java} -cp ${folder}: java exited 1: + Error: Could not find or load main class tlc2.TLC + Caused by: java.lang.ClassNotFoundException: tlc2.TLC +${setVars}`, + ); + const noJava = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, { + ...noTlc, + TLA2TOOLS: tools.jar, + }); + assert.equal(noJava.status, 2); + assert.equal( + noJava.stderr, + `cannot run TLC with java -cp ${tools.jar}: spawn java ENOENT\n${setVars}`, + ); +}); + test("from-chart checks the agreed chart and draws it back unchanged", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); @@ -1084,7 +2079,7 @@ TLC on Upload.tla with Upload.cfg: Chart: 5 states and 8 transitions from 5 model states. 5 work, 0 break a rule. -No state breaks a rule. +No state breaks an INVARIANT rule. The spec draws upload.mmd back unchanged. @@ -1094,6 +2089,45 @@ Wrote chart.mmd, stories.md, summary.json in Upload-statechart. assert.equal(r.stderr, ""); }); +const ringing = `stateDiagram-v2 + [*] --> Idle + Idle --> Ringing : a call comes in + Ringing --> Ringing : it rings again + Ringing --> Idle : the caller hangs up +`; + +test( + "from-chart counts a self-loop, so a spec that drops it does not draw the chart back", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "call.mmd"), ringing); + const drawn = node("statechart.mjs", ["from-chart", "call.mmd"], dir); + assert.equal(drawn.status, 0, drawn.stdout + drawn.stderr); + assert.match(drawn.stdout, /\nThe spec draws call\.mmd back unchanged\.\n/); + + const spec = readFileSync(join(dir, "Call.tla"), "utf8"); + const dropped = spec.replace(" \\/ ItRingsAgain", ""); + assert.notEqual(dropped, spec); + writeFileSync(join(dir, "Call.tla"), dropped); + const r = node("statechart.mjs", ["from-chart", "call.mmd"], dir); + assert.equal(r.status, 1, r.stdout + r.stderr); + assert.match( + r.stdout, + /\nThe spec does not draw call\.mmd:\n {2}transitions only in EXPECTED [^\n]*\n {4}Ringing --> Ringing : it rings again\n {2}1 difference\(s\)\n/, + ); + }, +); + +test("from-chart draws a chart with no transitions back unchanged", needsTlc, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "lone.mmd"), "stateDiagram-v2\n [*] --> Idle\n"); + const r = node("statechart.mjs", ["from-chart", "lone.mmd"], dir); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match(r.stdout, /\nThe spec draws lone\.mmd back unchanged\.\n/); + assert.equal(r.stderr, ""); +}); + test("TLC reads a label with a guarded and an unguarded move in one region", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeFileSync(join(dir, "switch.mmd"), switchLamp); @@ -1105,7 +2139,7 @@ test("TLC reads a label with a guarded and an unguarded move in one region", nee const syncStory = `## 1. Offline | Synced Rule broken: You never see "Synced" while you are offline. -This is the only way to get here in the model. +In the model, one step leads into this bug: "the server confirms". 0. Start: Online | Synced 1. The network drops. Now: Offline | Waiting to sync @@ -1168,7 +2202,14 @@ Wrote chart.mmd, bug-1.mmd, stories.md, summary.json in Sync-statechart. `, ); const summary = JSON.parse(readFileSync(join(dir, "Sync-statechart", "summary.json"), "utf8")); - assert.deepEqual(summary.chartStatus, { works: 3, wrong: 0, fixed: 1 }); + assert.deepEqual(summary.chartStatus, { + works: 3, + wrong: 0, + fixed: 1, + mixed: 0, + after: 0, + worse: 0, + }); assert.equal(summary.classes[0].fix, "Offline | Waiting to sync"); const map = readFileSync(join(dir, "Sync.map.mjs"), "utf8"); @@ -1213,7 +2254,7 @@ TLC on Sync.tla with Sync.cfg: Chart: 3 states and 5 transitions from 3 model states. 3 work, 0 break a rule. -No state breaks a rule. +No state breaks an INVARIANT rule. The spec does not draw sync.mmd: states only in EXPECTED (no path from the start reaches them in the spec, or the spec renamed them): @@ -1237,155 +2278,845 @@ Wrote chart.mmd, stories.md, summary.json in Sync-statechart. }); test( - "--png exits 2 unless every chart renders, even with --expect-broken, and keeps the files it wrote", + "a second model with the module name of the spec, in its own folder, keeps its own TLC files", needsTlc, () => { - const args = ["from-chart", "sync.mmd", "--expect-broken", "NeverSyncedWhileOffline", "--png"]; - const bin = mkdtempSync(join(tmpdir(), "fake-npx-")); - const launch = "Error: Failed to launch the browser process!"; - writeFileSync( - join(bin, "npx"), - `#!/bin/sh -while [ $# -gt 0 ]; do - case "$1" in -i) chart=$2 ;; -o) png=$2 ;; esac - shift -done -for c in $UNRENDERED; do - case "$chart" in */$c.mmd) echo "${launch}" >&2; exit 1 ;; esac -done -echo png > "$png" -`, - { mode: 0o755 }, + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); + assert.equal(node("statechart.mjs", ["from-chart", "sync.mmd"], dir).status, 1); + const guarded = readFileSync(join(dir, "Sync.tla"), "utf8").replace( + "TheServerConfirms ==\n", + 'TheServerConfirms ==\n /\\ connection = "Online"\n', ); - for (const [unrendered, status, written] of [ - ["chart bug-1", 2, ["chart.mmd", "bug-1.mmd"]], - ["bug-1", 2, ["chart.mmd", "bug-1.mmd", "chart.png"]], - ["", 0, ["chart.mmd", "bug-1.mmd", "chart.png", "bug-1.png"]], - ]) { - const dir = mkdtempSync(join(tmpdir(), "statechart-")); - writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); - const r = node("statechart.mjs", args, dir, { - PATH: `${bin}${delimiter}${process.env.PATH}`, - UNRENDERED: unrendered, - }); - assert.equal(r.status, status, `${unrendered}: ${r.stdout}${r.stderr}`); - assert.equal( - r.stderr, - unrendered - .split(" ") - .filter(Boolean) - .map((c) => `could not render Sync-statechart/${c}.mmd: ${launch}\n`) - .join(""), - ); - assert.match(r.stdout, /\nBroken as expected: NeverSyncedWhileOffline\.\n/); + mkdirSync(join(dir, "fix")); + writeFileSync(join(dir, "fix", "Sync.tla"), guarded); + writeFileSync(join(dir, "SyncFixed.tla"), guarded.replace("MODULE Sync", "MODULE SyncFixed")); + const run = (...args) => node("statechart.mjs", ["from-spec", "Sync.tla", ...args], dir); + const broken = /^Error: Invariant NeverSyncedWhileOffline is violated/m; + + const named = run("--fixed", "SyncFixed.tla", "--out", "named"); + const same = run("--fixed", "fix/Sync.tla", "--out", "same"); + assert.equal(same.status, 1, same.stderr); + assert.equal(same.stderr, named.stderr.replaceAll("SyncFixed.tla", "fix/Sync.tla")); + assert.equal( + same.stdout, + named.stdout + .replace("TLC on SyncFixed.tla with", "TLC on fix/Sync.tla with") + .replace(" in named.\n", " in same.\n"), + ); + for (const file of ["chart.mmd", "bug-1.mmd", "stories.md"]) assert.equal( - r.stdout.split("\n").at(-2), - `Wrote ${[...written, "stories.md", "summary.json"].join(", ")} in Sync-statechart.`, + readFileSync(join(dir, "same", file), "utf8"), + readFileSync(join(dir, "named", file), "utf8"), ); - assert.deepEqual( - readdirSync(join(dir, "Sync-statechart")).sort(), - [...written, "stories.md", "summary.json", "tlc"].sort(), - ); - } + const tlcOut = (...parts) => readFileSync(join(dir, "same", "tlc", ...parts), "utf8"); + assert.match(tlcOut("Sync-NeverSyncedWhileOffline.out"), broken); + assert.doesNotMatch(tlcOut("fixed", "Sync-NeverSyncedWhileOffline.out"), broken); + + cpSync(join(dir, "Sync.cfg"), join(dir, "fix", "Sync.cfg")); + const agreed = run("--against", "fix/Sync.tla", "--out", "agreed"); + assert.equal(agreed.status, 1, agreed.stderr); + assert.match(agreed.stdout, /^TLC on fix\/Sync\.tla with fix\/Sync\.cfg:\n/m); + const agreedOut = (...parts) => readFileSync(join(dir, "agreed", "tlc", ...parts), "utf8"); + assert.match(agreedOut("Sync-NeverSyncedWhileOffline.out"), broken); + assert.doesNotMatch(agreedOut("against", "Sync-NeverSyncedWhileOffline.out"), broken); }, ); -const pay = (name, charge) => `---- MODULE ${name} ---- -EXTENDS Naturals -VARIABLES screen, charges -vars == <> -Init == screen = "Cart" /\\ charges = 0 -Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED charges -Charge == ${charge} -Refund == charges = 2 /\\ charges' = 1 /\\ screen' = "Refunded" -Next == Pay \\/ Charge \\/ Refund -Spec == Init /\\ [][Next]_vars -NoDoubleCharge == charges <= 1 +const lockSpec = `---- MODULE Lock ---- +VARIABLES a, b, locked +vars == <> +Init == a = "idle" /\\ b = "idle" /\\ locked = FALSE +CheckA == a = "idle" /\\ ~locked /\\ a' = "checked" /\\ UNCHANGED <> +TakeA == a = "checked" /\\ a' = "critical" /\\ locked' = TRUE /\\ UNCHANGED b +ReleaseA == a = "critical" /\\ a' = "idle" /\\ locked' = FALSE /\\ UNCHANGED b +CheckB == b = "idle" /\\ ~locked /\\ b' = "checked" /\\ UNCHANGED <> +TakeB == b = "checked" /\\ b' = "critical" /\\ locked' = TRUE /\\ UNCHANGED a +ReleaseB == b = "critical" /\\ b' = "idle" /\\ locked' = FALSE /\\ UNCHANGED a +Spec == Init /\\ [][CheckA \\/ TakeA \\/ ReleaseA \\/ CheckB \\/ TakeB \\/ ReleaseB]_vars +\\* Two processes are never in the critical section at once. +MutualExclusion == ~(a = "critical" /\\ b = "critical") ==== `; -const payMap = `export const readerActions = ["Pay"]; -export const state = (v) => v.screen + ["", ", charged", ", charged twice"][v.charges]; -export const step = (action) => ({ Pay: "you pay", Charge: "the bank charges you", Refund: "the bank refunds you" })[action] ?? ""; -export const cause = () => "each charge request charges again"; +const lockMap = `const S = { idle: "outside", checked: "saw the lock free", critical: "in the critical section" }; +export const state = (v) => \`A \${S[v.a]}, B \${S[v.b]}\`; +export const step = (a) => ({ + CheckA: "A checks the lock", TakeA: "A takes the lock", ReleaseA: "A releases the lock", + CheckB: "B checks the lock", TakeB: "B takes the lock", ReleaseB: "B releases the lock", +})[a]; `; test( - "from-spec checks each PROPERTY in its own TLC run, and names the counterexample of a broken one", + "from-spec on a lock protocol names every step that leads into the bug, not only the shortest trace", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); - writeFileSync( - join(dir, "Pay.tla"), - pay("Pay", `screen = "Paying" /\\ charges = 0 /\\ charges' = 1 /\\ UNCHANGED screen`).replace( - "====", - 'ChargedAtMostOnce == [](charges <= 1)\nEventuallyPaying == <>(screen = "Paying")\n====', - ), - ); - writeFileSync( - join(dir, "Pay.cfg"), - "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\nPROPERTIES ChargedAtMostOnce EventuallyPaying\n", + writeFileSync(join(dir, "Lock.tla"), lockSpec); + writeFileSync(join(dir, "Lock.cfg"), "SPECIFICATION Spec\nINVARIANT MutualExclusion\n"); + writeFileSync(join(dir, "Lock.map.mjs"), lockMap); + const r = node( + "statechart.mjs", + ["from-spec", "Lock.tla", "--expect-broken", "MutualExclusion"], + dir, ); - writeFileSync(join(dir, "Pay.map.mjs"), payMap); - const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); - assert.equal(r.status, 1, r.stderr); + assert.equal(r.status, 0, r.stdout + r.stderr); + const story = `## 1. A in the critical section, B in the critical section + +Rule broken: Two processes are never in the critical section at once. +In the model, 2 different steps lead into this bug: "A takes the lock" and "B takes the lock". The steps below are the shortest path that TLC found. + +0. Start: A outside, B outside +1. A checks the lock. Now: A saw the lock free, B outside +2. B checks the lock. Now: A saw the lock free, B saw the lock free +3. A takes the lock. Now: A in the critical section, B saw the lock free +4. B takes the lock. Now: A in the critical section, B in the critical section +`; assert.equal( - r.stdout.split("\n\n")[0], - `TLC on Pay.tla with Pay.cfg: - NoDoubleCharge holds. - ChargedAtMostOnce holds. - EventuallyPaying is broken. The chart draws only INVARIANT rules, so read TLC's counterexample in ${join("Pay-statechart", "tlc", "Pay-EventuallyPaying.out")}. - TLC ran 4 times in N s.`, + r.stdout, + `TLC on Lock.tla with Lock.cfg: + MutualExclusion is broken (expected). + TLC ran 3 times in N s. + +Chart: 9 states and 18 transitions from 13 model states. +8 work, 1 breaks a rule. + +${story} +Broken as expected: MutualExclusion. + +Wrote chart.mmd, bug-1.mmd, stories.md, summary.json in Lock-statechart. +`, ); - assert.match(r.stdout, /\nNo state breaks a rule\.\n/); - const tlcDir = join(dir, "Pay-statechart", "tlc"); - assert.match( - readFileSync(join(tlcDir, "Pay-EventuallyPaying.out"), "utf8"), - /^Error: Temporal properties were violated\./m, + const out = join(dir, "Lock-statechart"); + assert.equal(readFileSync(join(out, "stories.md"), "utf8"), story); + const [bug] = JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).classes; + assert.equal(bug.counterexamples, 1); + assert.deepEqual(bug.lastSteps, ["A takes the lock", "B takes the lock"]); + assert.equal(bug.startsHere, false); + }, +); + +const counterMap = (steps) => + `export const state = (v) => \`The counter shows \${v.x}\`;\nexport const step = (a) => (${JSON.stringify(steps)})[a] ?? "the counter moves";\n`; + +test( + "a story does not count the steps into a bug when TLC names them only after the whole next-state action", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Jump.tla"), + `---- MODULE Jump ---- +VARIABLE x +Init == x = 0 +Next == + \\/ x = 0 /\\ x' = 1 + \\/ x = 1 /\\ x' = 3 + \\/ x = 0 /\\ x' = 3 +Spec == Init /\\ [][Next]_x +\\* The counter never shows 3. +NotThree == x # 3 +==== +`, ); - assert.deepEqual( - readdirSync(tlcDir) - .filter((f) => f.endsWith(".cfg") && /PROPERT/.test(readFileSync(join(tlcDir, f), "utf8"))) - .sort(), - ["Pay-ChargedAtMostOnce.cfg", "Pay-EventuallyPaying.cfg"], + writeFileSync(join(dir, "Jump.cfg"), "SPECIFICATION Spec\nINVARIANT NotThree\n"); + writeFileSync(join(dir, "Jump.map.mjs"), counterMap({})); + const r = node("statechart.mjs", ["from-spec", "Jump.tla", "--expect-broken", "NotThree"], dir); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match( + r.stderr, + /^Jump\.tla: 3 steps have the name Next, the whole next-state action\./m, ); - const expected = node( - "statechart.mjs", - ["from-spec", "Pay.tla", "--expect-broken", "EventuallyPaying"], - dir, + assert.equal( + readFileSync(join(dir, "Jump-statechart", "stories.md"), "utf8"), + `## 1. The counter shows 3 + +Rule broken: The counter never shows 3. +In the model, steps labeled "the counter moves" lead into this bug. TLC names them Next, the whole next-state action, so the story cannot count them. The steps below are the shortest path that TLC found. + +0. Start: The counter shows 0 +1. The counter moves. Now: The counter shows 3 +`, ); - assert.equal(expected.status, 0, expected.stderr); - assert.match(expected.stdout, /\n {2}EventuallyPaying is broken \(expected\)\. /); }, ); test( - "from-spec checks a rule that the config lists as both PROPERTY and INVARIANT once, as the INVARIANT, and charts its bug", + "a story says its path is the shortest only when more than one step leads into the bug", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeFileSync( - join(dir, "Pay.tla"), - pay( - "Pay", - `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, - ), + join(dir, "Skip.tla"), + `---- MODULE Skip ---- +EXTENDS Naturals +VARIABLE x +Init == x = 0 +Up == x < 2 /\\ x' = x + 1 +Skip == x = 0 /\\ x' = 2 +Spec == Init /\\ [][Up \\/ Skip]_x +\\* The counter never shows 1. +NotOne == x # 1 +\\* The counter never shows 2. +NotTwo == x # 2 +==== +`, ); writeFileSync( - join(dir, "Pay.cfg"), - "SPECIFICATION Spec\nPROPERTY NoDoubleCharge\nINVARIANT NoDoubleCharge\n", + join(dir, "Skip.cfg"), + "SPECIFICATION Spec\nINVARIANT NotOne\nINVARIANT NotTwo\n", ); - writeFileSync(join(dir, "Pay.map.mjs"), payMap); - const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); - assert.equal(r.status, 1, r.stderr); - assert.equal( - r.stdout.split("\n\n")[0], - `TLC on Pay.tla with Pay.cfg: - NoDoubleCharge is broken. - TLC ran 3 times in N s.`, + writeFileSync(join(dir, "Skip.map.mjs"), counterMap({})); + const r = node( + "statechart.mjs", + ["from-spec", "Skip.tla", "--expect-broken", "NotOne,NotTwo"], + dir, ); - assert.match(r.stdout, /\nRule broken: `NoDoubleCharge`\n/); + assert.equal(r.status, 0, r.stdout + r.stderr); assert.equal( - r.stdout.split("\n").at(-2), + readFileSync(join(dir, "Skip-statechart", "stories.md"), "utf8"), + `## 1. The counter shows 1 + +Rule broken: The counter never shows 1. +In the model, one step leads into this bug: "the counter moves". + +0. Start: The counter shows 0 +1. The counter moves. Now: The counter shows 1 + +## 2. The counter shows 2 + +Rule broken: The counter never shows 2. +In the model, one step leads into this bug: "the counter moves". The steps below are the shortest path that TLC found. + +0. Start: The counter shows 0 +1. The counter moves. Now: The counter shows 2 +`, + ); + }, +); + +test( + "a story says which step into the bug keeps the state, so that no chart draws it", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Panel.tla"), + `---- MODULE Panel ---- +VARIABLES panel, saved +vars == <> +Init == panel = "closed" /\\ saved = TRUE +Open == panel = "closed" /\\ panel' = "open" /\\ UNCHANGED saved +Close == panel = "open" /\\ panel' = "closed" /\\ UNCHANGED saved +Lose == panel = "open" /\\ saved /\\ saved' = FALSE /\\ UNCHANGED panel +Pop == panel = "closed" /\\ saved /\\ panel' = "open" /\\ saved' = FALSE +Spec == Init /\\ [][Open \\/ Close \\/ Lose \\/ Pop]_vars +\\* The note is never lost. +NeverLost == saved +==== +`, + ); + writeFileSync(join(dir, "Panel.cfg"), "SPECIFICATION Spec\nINVARIANT NeverLost\n"); + writeFileSync( + join(dir, "Panel.map.mjs"), + 'export const state = (v) => `The panel is ${v.panel}`;\nexport const step = (a) => ({ Open: "you open the panel", Close: "you close the panel", Lose: "the note is lost", Pop: "the panel pops open" })[a];\n', + ); + const r = node( + "statechart.mjs", + ["from-spec", "Panel.tla", "--expect-broken", "NeverLost"], + dir, + ); + assert.equal(r.status, 0, r.stdout + r.stderr); + const out = join(dir, "Panel-statechart"); + assert.match( + readFileSync(join(out, "stories.md"), "utf8"), + /^In the model, 2 different steps lead into this bug: "the note is lost" \(the state stays the same\) and "the panel pops open"\. The steps below are the shortest path that TLC found\.\n\n0\. Start: The panel is closed\n1\. The panel pops open\. Now: The panel is open\n$/m, + ); + for (const chart of readdirSync(out).filter((f) => f.endsWith(".mmd"))) + assert.doesNotMatch(readFileSync(join(out, chart), "utf8"), /the note is lost/, chart); + }, +); + +test( + "--png exits 2 unless every chart renders, even with --expect-broken, and keeps the files it wrote", + { skip: needsTlc.skip || needsPng.skip }, + () => { + const args = [ + "from-chart", + "sync.mmd", + "--expect-broken", + "NeverSyncedWhileOffline", + "--title", + "Sync", + "--png", + ]; + const titles = { chart: "Sync", "bug-1": "Sync, bug 1" }; + const xdg = fakeRenderer(); + for (const [unrendered, status, written] of [ + [["chart", "bug-1"], 2, ["chart.mmd", "bug-1.mmd"]], + [["bug-1"], 2, ["chart.mmd", "bug-1.mmd", "chart.png"]], + [[], 0, ["chart.mmd", "bug-1.mmd", "chart.png", "bug-1.png"]], + ]) { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); + const rendered = join(dir, "rendered"); + const r = node("statechart.mjs", args, dir, { + XDG_CACHE_HOME: xdg, + TLA2TOOLS: tools.jar, + JAVA: tools.java, + RENDERED: rendered, + UNRENDERED: unrendered.map((c) => titles[c]).join("|"), + }); + assert.equal(r.status, status, `${unrendered}: ${r.stdout}${r.stderr}`); + assert.equal( + r.stderr, + unrendered + .map((c) => `could not render Sync-statechart/${c}.mmd: Pele cannot lay out this chart\n`) + .join(""), + ); + assert.deepEqual( + readFileSync(rendered, "utf8").split("\n").filter(Boolean), + ["Sync", "Sync, bug 1"], + "tries every chart", + ); + assert.match(r.stdout, /\nBroken as expected: NeverSyncedWhileOffline\.\n/); + assert.equal( + r.stdout.split("\n").at(-2), + `Wrote ${[...written, "stories.md", "summary.json"].join(", ")} in Sync-statechart.`, + ); + const out = join(dir, "Sync-statechart"); + assert.deepEqual( + readdirSync(out).sort(), + [...written, "stories.md", "summary.json", "tlc"].sort(), + ); + for (const png of written.filter((f) => f.endsWith(".png"))) + assert.ok(decodePng(readFileSync(join(out, png))).width > 0, png); + } + }, +); + +const SOURCE = fixture("regions.mmd"); +const SOURCE_LINES = [...SOURCE.matchAll(/state "([^"]+)" as|--> \w+ : (.+)$/gm)] + .flatMap((m) => (m[1] ?? m[2]).split("
")) + .sort(); +const drawnLines = (svg) => + labeledBoxes(svg) + .flatMap((b) => b.lines) + .sort(); + +function rectOf(svg, id) { + const [ox, oy] = // + .exec(svg) + .slice(1) + .map(Number); + const [tx, ty, x, y, w, h] = new RegExp( + `data-id="${id}"(?: transform="translate\\(([\\d.-]+),([\\d.-]+)\\)")?> +(v ?? 0)); + const [left, top] = [ox + tx + x, oy + ty + y]; + return { left, top, right: left + w, bottom: top + h }; +} + +test( + "render writes a PNG at twice the chart's size, next to the chart or at --out", + needsPng, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "regions.mmd"), SOURCE); + writeFileSync(join(dir, "regions.txt"), SOURCE); + for (const [args, file, theme] of [ + [["regions.mmd"], "regions.png", "light"], + [["regions.txt"], "regions.txt.png", "light"], + ...Object.keys(THEMES).map((t) => [ + ["regions.mmd", "--theme", t, "--out", join("new", t, "chart.png")], + join("new", t, "chart.png"), + t, + ]), + ]) { + const r = node("statechart.mjs", ["render", ...args], dir); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, ""); + assert.equal(r.stdout, `Wrote ${file}.\n`); + const chart = renderChart(lib, SOURCE, { theme: THEMES[theme] }); + const png = decodePng(readFileSync(join(dir, file))); + assert.deepEqual([png.width, png.height], [2 * chart.width, 2 * chart.height], file); + assert.equal(png.pixel(0, 0), THEMES[theme].background, file); + } + }, +); + +test( + "a chart too big to draw at twice its size draws at about 100 million pixels", + needsPng, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const steps = Array.from({ length: 60 }, (_, i) => + [0, 1, 2].map((k) => ` s${i} --> s${(i * 7 + k * 13 + 1) % 60} : e${i}x${k}`), + ).flat(); + const src = `stateDiagram-v2\n [*] --> s0\n${steps.join("\n")}\n`; + writeFileSync(join(dir, "c.mmd"), src); + const chart = renderChart(lib, src); + assert.ok(4 * chart.width * chart.height > 1.2e8, `${chart.width}x${chart.height}`); + const r = node("statechart.mjs", ["render", "c.mmd"], dir); + assert.equal(r.status, 0, r.stderr); + const png = readFileSync(join(dir, "c.png")); + const pixels = png.readUInt32BE(16) * png.readUInt32BE(20); + assert.ok(pixels > 0.99e8 && pixels < 1.001e8, `${pixels} pixels`); + }, +); + +test( + "a PNG fills a state with its theme's group color, and a state that breaks a rule with the theme's wrong fill and border", + needsPng, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "regions.mmd"), SOURCE); + const chartFill = /classDef group0 fill:(#\w{6})/.exec(SOURCE)[1]; + for (const name of [undefined, ...Object.keys(THEMES)]) { + const theme = THEMES[name ?? "light"]; + const r = node( + "statechart.mjs", + ["render", "regions.mmd", ...(name ? ["--theme", name] : [])], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const png = decodePng(readFileSync(join(dir, "regions.png"))); + const { svg } = renderChart(lib, SOURCE, { theme }); + const inside = (box) => png.pixel(2 * (box.left + 8), box.top + box.bottom); + const borderWidth = (box) => { + const [x, y] = [2 * box.left, box.top + box.bottom]; + const ink = png.pixel(x, y); + let n = 0; + for (let i = x - 6; i <= x + 6; i++) if (png.pixel(i, y) === ink) n++; + return n; + }; + const wrong = rectOf(svg, "r1s3"); + const group = rectOf(svg, "r1s2"); + assert.equal(inside(wrong), theme.wrong.fill, `${name} wrong fill`); + assert.equal( + png.pixel(2 * wrong.left, wrong.top + wrong.bottom), + theme.wrong.border, + `${name} wrong border`, + ); + assert.ok( + borderWidth(wrong) >= 4 && borderWidth(group) <= 3, + `${name} wrong border is ${borderWidth(wrong)}px and a group border ${borderWidth(group)}px at twice the chart's size`, + ); + assert.equal(inside(group), theme.groups?.[0] ?? chartFill, `${name} group fill`); + } + }, +); + +test( + "a PNG draws a state that breaks a rule in some cases, one reached only after a bug, and one that breaks a rule only with the fix, in its theme's colors", + needsPng, + () => { + const source = [ + "stateDiagram-v2", + ...STATUS_CLASSES.map((c) => ` ${c}`), + ...["Mixed", "After", "Worse"].map((b) => ` ${groupClassDef("group0", "#d6eaf8", b)}`), + ' state "Works" as w', + ' state "Wrong in some cases" as m', + ' state "Reached after a bug" as a', + ' state "Wrong only with the fix" as o', + ' state "Wrong in some cases, in a group" as gm', + ' state "Reached after a bug, in a group" as ga', + ' state "Wrong only with the fix, in a group" as go', + " [*] --> w", + " w --> m : one", + " m --> a : two", + " a --> o : five", + " w --> gm : three", + " gm --> ga : four", + " ga --> go : six", + " class w works", + " class m mixed", + " class a after", + " class o worse", + " class gm group0Mixed", + " class ga group0After", + " class go group0Worse", + "", + ].join("\n"); + for (const [name, theme] of Object.entries(THEMES)) { + const { svg } = renderChart(lib, source, { theme }); + for (const [id, status] of [ + ["m", "mixed"], + ["a", "after"], + ["o", "worse"], + ["gm", "mixed"], + ["ga", "after"], + ["go", "worse"], + ]) { + const [box, label] = new RegExp( + `data-id="${id}"[^>]*>]*style="([^"]*)"/>]*style="([^"]*)"`, + ) + .exec(svg) + .slice(1) + .map((s) => + Object.fromEntries( + s + .split(";") + .filter(Boolean) + .map((d) => d.split(":")), + ), + ); + assert.deepEqual( + [box.fill, box.stroke, label.fill], + [theme[status]?.fill, theme[status]?.border, theme[status]?.text], + `${name} ${id}`, + ); + } + } + }, +); + +test("a PNG draws no edge over a group's title band", needsPng, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const source = fixture("band.mmd"); + writeFileSync(join(dir, "band.mmd"), source); + for (const [name, theme] of Object.entries(THEMES)) { + const r = node("statechart.mjs", ["render", "band.mmd", "--theme", name], dir); + assert.equal(r.status, 0, r.stderr); + const png = decodePng(readFileSync(join(dir, "band.png"))); + const { svg } = renderChart(lib, source, { theme }); + const group = rectOf(svg, "G"); + const [rectY, dividerY] = + /data-id="G">]*\/> fcMatch(f)?.name === f, +); +const type1 = ["Nimbus Sans Narrow", "Nimbus Sans", "URW Gothic"].find((f) => + /\.(pfb|pfa|t1)$/.test(fcMatch(f)?.file ?? ""), +); +const fcFile = (pattern) => + spawnSync("fc-list", [pattern, "file"], { encoding: "utf8" }).stdout?.match(/^(.+): $/m)?.[1]; +const type1Styles = [ + "Nimbus Sans:style=Regular:fontformat=CFF", + "Nimbus Sans:style=Bold:fontformat=Type 1", + "Nimbus Sans:style=Italic:fontformat=Type 1", +].map(fcFile); +test( + "each label fits its box in the font it is drawn in, and a wider font gets a wider wrap, so lines break where the chart breaks them", + { + skip: + needsPng.skip || + (fonts.length < 2 && "needs two of DejaVu Sans, Liberation Sans, Noto Sans, and Inter"), + }, + () => { + const runs = fonts.map((font) => { + const measure = fontMeasurer(lib.Resvg, font); + const chart = renderChart(lib, SOURCE, { systemFont: font }); + for (const { kind, id, box, size, lines } of labeledBoxes(chart.svg)) + for (const line of lines) + assert.ok( + measure.width(line, size) <= box, + `${font}: ${kind} ${id} "${line}" fits ${box}`, + ); + assert.deepEqual( + drawnLines(chart.svg), + SOURCE_LINES, + `${font} breaks lines as the chart does`, + ); + return { + font, + wrap: chart.wrap, + ink: SOURCE_LINES.reduce((s, l) => s + measure.width(l, 16), 0), + }; + }); + runs.sort((a, b) => a.ink - b.ink); + const [narrow, wide] = [runs[0], runs.at(-1)]; + assert.ok( + wide.wrap > narrow.wrap, + `${wide.font} wraps at ${wide.wrap}, ${narrow.font} at ${narrow.wrap}`, + ); + }, +); + +test( + "a font that resvg cannot read keeps Pele's widths, so the chart keeps every label", + { skip: needsPng.skip || (!type1 && "cannot find a Type 1 font") }, + () => { + assert.deepEqual(drawnLines(renderChart(lib, SOURCE, { systemFont: type1 }).svg), SOURCE_LINES); + }, +); + +test( + "a bold or italic face that resvg cannot read is measured in its regular face, so the chart keeps every label", + { + skip: + needsPng.skip || + (!type1Styles.every(Boolean) && + "needs Nimbus Sans with an OpenType regular face and Type 1 bold and italic faces"), + }, + () => { + const dir = mkdtempSync(join(tmpdir(), "fontconfig-")); + for (const file of type1Styles) symlinkSync(file, join(dir, basename(file))); + writeFileSync( + join(dir, "fonts.conf"), + `${dir}${dir}`, + ); + const styled = SOURCE.replace(/(--> \w+ : )(.+)$/m, "$1$2").replace( + /(--> \w+ : )([^<].*)$/m, + "$1$2", + ); + const run = spawnSync( + process.execPath, + [ + "--input-type=module", + "-e", + `import { loadRenderer, renderChart } from ${JSON.stringify(pathToFileURL(join(scripts, "render.mjs")).href)}; +process.stdout.write(renderChart(await loadRenderer(), ${JSON.stringify(styled)}, { systemFont: "Nimbus Sans" }).svg);`, + ], + { encoding: "utf8", env: { ...process.env, FONTCONFIG_FILE: join(dir, "fonts.conf") } }, + ); + assert.equal(run.status, 0, run.stderr); + assert.deepEqual(drawnLines(run.stdout), SOURCE_LINES); + }, +); + +test("a PNG puts parallel regions side by side and the legend above them", needsPng, () => { + const { svg } = renderChart(lib, SOURCE); + const [parcel, payment, legend, regions] = ["region0", "region1", "legend", "regions"].map((id) => + rectOf(svg, id), + ); + assert.ok(parcel.top < payment.bottom && payment.top < parcel.bottom, "the regions share rows"); + assert.ok(parcel.right <= payment.left, "the regions do not share columns"); + assert.ok(legend.bottom <= regions.top, "the legend is above the regions"); +}); + +test("a renderer that npm cannot install exits 2 and writes nothing", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "c.mmd"), SOURCE); + writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); + const xdg = mkdtempSync(join(tmpdir(), "xdg-")); + const env = { + XDG_CACHE_HOME: xdg, + npm_config_cache: mkdtempSync(join(tmpdir(), "npm-")), + npm_config_offline: "true", + }; + for (const args of [ + ["render", "c.mmd"], + ["from-chart", "sync.mmd", "--png"], + ]) { + const r = node("statechart.mjs", args, dir, env); + assert.equal(r.status, 2, `${args}: ${r.stdout}${r.stderr}`); + assert.equal(r.stdout, ""); + assert.ok( + r.stderr.startsWith( + `Installing pele@1.8.0 and @resvg/resvg-js@2.6.2 with npm in ${join(xdg, "statechart", CACHE_NAME)}. This happens once.\n`, + ), + r.stderr, + ); + assert.match( + r.stderr, + /\ncannot install pele@1\.8\.0 and @resvg\/resvg-js@2\.6\.2 with npm:\n/, + ); + assert.deepEqual(readdirSync(dir).sort(), ["c.mmd", "sync.mmd"]); + assert.deepEqual(readdirSync(join(xdg, "statechart")), []); + } +}); + +test("a renderer that cannot load exits 2, names its folder, and says to delete it", () => { + const xdg = mkdtempSync(join(tmpdir(), "xdg-")); + const cache = join(xdg, "statechart", CACHE_NAME); + mkdirSync(cache, { recursive: true }); + writeFileSync(join(cache, "entry.mjs"), 'export { render } from "no-such-package";\n'); + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "c.mmd"), SOURCE); + const r = node("statechart.mjs", ["render", "c.mmd"], dir, { XDG_CACHE_HOME: xdg }); + assert.equal(r.status, 2); + assert.equal(r.stdout, ""); + assert.ok(r.stderr.startsWith(`cannot load Pele and resvg from ${cache}: `), r.stderr); + assert.ok(r.stderr.endsWith("\nDelete that folder to install them again.\n"), r.stderr); + assert.deepEqual(readdirSync(dir), ["c.mmd"]); +}); + +test("a chart too long for Pele exits 2, writes no PNG, and says how to get one", needsPng, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const lines = Array.from({ length: 3000 }, (_, i) => ` s${i} --> s${i + 1} : step ${i}`); + writeFileSync(join(dir, "c.mmd"), `stateDiagram-v2\n${lines.join("\n")}\n`); + const r = node("statechart.mjs", ["render", "c.mmd"], dir); + assert.equal(r.status, 2); + assert.equal(r.stdout, ""); + assert.ok(r.stderr.startsWith("could not render c.mmd: "), r.stderr); + assert.ok( + r.stderr.endsWith( + " The chart is too big for one PNG, and the .mmd file still has all of it. A chart with fewer states fits. Run node statechart.mjs from-spec SPEC.tla --cover --png to draw a smaller overview of the spec.\n", + ), + r.stderr, + ); + assert.deepEqual(readdirSync(dir), ["c.mmd"]); +}); + +const pay = (name, charge) => `---- MODULE ${name} ---- +EXTENDS Naturals +VARIABLES screen, charges +vars == <> +Init == screen = "Cart" /\\ charges = 0 +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED charges +Charge == ${charge} +Refund == charges = 2 /\\ charges' = 1 /\\ screen' = "Refunded" +Next == Pay \\/ Charge \\/ Refund +Spec == Init /\\ [][Next]_vars +NoDoubleCharge == charges <= 1 +==== +`; +const payMap = `export const readerActions = ["Pay"]; +export const state = (v) => v.screen + ["", ", charged", ", charged twice"][v.charges]; +export const step = (action) => ({ Pay: "you pay", Charge: "the bank charges you", Refund: "the bank refunds you" })[action] ?? ""; +export const cause = () => "each charge request charges again"; +`; + +test( + "from-spec checks each PROPERTY in its own TLC run, and names the counterexample of a broken one", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay("Pay", `screen = "Paying" /\\ charges = 0 /\\ charges' = 1 /\\ UNCHANGED screen`).replace( + "====", + 'ChargedAtMostOnce == [](charges <= 1)\nEventuallyPaying == <>(screen = "Paying")\n====', + ), + ); + writeFileSync( + join(dir, "Pay.cfg"), + "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\nPROPERTIES ChargedAtMostOnce EventuallyPaying\n", + ); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); + assert.equal(r.status, 1, r.stderr); + assert.equal( + r.stdout.split("\n\n")[0], + `TLC on Pay.tla with Pay.cfg: + NoDoubleCharge holds. + ChargedAtMostOnce holds. + EventuallyPaying is broken. The chart draws only INVARIANT rules, so read TLC's counterexample in ${join("Pay-statechart", "tlc", "Pay-EventuallyPaying.out")}. + TLC ran 4 times in N s.`, + ); + const stories = `## Broken rules that the chart does not draw + +Rule broken: \`EventuallyPaying\`. The run stops in "Cart" forever. TLC's counterexample is in ${join("tlc", "Pay-EventuallyPaying.out")}. +`; + assert.ok(r.stdout.includes(`\nNo state breaks an INVARIANT rule.\n\n${stories}`), r.stdout); + const tlcDir = join(dir, "Pay-statechart", "tlc"); + assert.match( + readFileSync(join(tlcDir, "Pay-EventuallyPaying.out"), "utf8"), + /^Error: Temporal properties were violated\./m, + ); + assert.deepEqual( + readdirSync(tlcDir) + .filter((f) => f.endsWith(".cfg") && /PROPERT/.test(readFileSync(join(tlcDir, f), "utf8"))) + .sort(), + ["Pay-ChargedAtMostOnce.cfg", "Pay-EventuallyPaying.cfg"], + ); + const expected = node( + "statechart.mjs", + ["from-spec", "Pay.tla", "--expect-broken", "EventuallyPaying"], + dir, + ); + assert.equal(expected.status, 0, expected.stderr); + assert.match(expected.stdout, /\n {2}EventuallyPaying is broken \(expected\)\. /); + }, +); + +test( + "from-spec names the reader states that a broken PROPERTY's counterexample loops through, without a state twice in a row and without the states before the loop", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Bell.tla"), + `---- MODULE Bell ---- +VARIABLES door, bell, knocked +vars == <> +Init == door = "closed" /\\ bell = "off" /\\ knocked = FALSE +SwitchOn == bell = "off" /\\ bell' = "quiet" /\\ UNCHANGED <> +Knock == bell = "quiet" /\\ ~knocked /\\ knocked' = TRUE /\\ UNCHANGED <> +Ring == knocked /\\ bell' = "ringing" /\\ knocked' = FALSE /\\ UNCHANGED door +Hush == bell = "ringing" /\\ bell' = "quiet" /\\ UNCHANGED <> +Open == bell = "ringing" /\\ door' = "open" /\\ UNCHANGED <> +Spec == Init /\\ [][SwitchOn \\/ Knock \\/ Ring \\/ Hush \\/ Open]_vars /\\ WF_vars(SwitchOn \\/ Knock \\/ Ring \\/ Hush) +EventuallyOpen == <>(door = "open") +==== +`, + ); + writeFileSync(join(dir, "Bell.cfg"), "SPECIFICATION Spec\nPROPERTY EventuallyOpen\n"); + writeFileSync( + join(dir, "Bell.map.mjs"), + `export const readerActions = ["Open"]; +export const state = (v, facts) => v.door === "open" ? "Door open" : v.bell === "off" ? "Bell off" : facts.enabled.has("Hush") ? "Bell ringing" : "Waiting"; +export const step = (action) => action; +`, + ); + const r = node( + "statechart.mjs", + ["from-spec", "Bell.tla", "--expect-broken", "EventuallyOpen"], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const out = join(dir, "Bell-statechart"); + assert.match( + readFileSync(join(out, "tlc", "Bell-EventuallyOpen.out"), "utf8"), + /^Back to state 2: /m, + ); + const stories = `## Broken rules that the chart does not draw + +Rule broken: \`EventuallyOpen\`. The run loops through "Waiting" and "Bell ringing" forever. TLC's counterexample is in ${join("tlc", "Bell-EventuallyOpen.out")}. +`; + assert.ok(r.stdout.includes(`\nNo state breaks an INVARIANT rule.\n\n${stories}`), r.stdout); + assert.deepEqual( + JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).brokenProperties[0].repeats, + { states: ["Waiting", "Bell ringing"], stutters: false }, + ); + }, +); + +test( + "from-spec checks a rule that the config lists as both PROPERTY and INVARIANT once, as the INVARIANT, and charts its bug", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync( + join(dir, "Pay.cfg"), + "SPECIFICATION Spec\nPROPERTY NoDoubleCharge\nINVARIANT NoDoubleCharge\n", + ); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); + assert.equal(r.status, 1, r.stderr); + assert.equal( + r.stdout.split("\n\n")[0], + `TLC on Pay.tla with Pay.cfg: + NoDoubleCharge is broken. + TLC ran 3 times in N s.`, + ); + assert.match(r.stdout, /\nRule broken: `NoDoubleCharge`\n/); + assert.equal( + r.stdout.split("\n").at(-2), "Wrote chart.mmd, bug-1.mmd, stories.md, summary.json in Pay-statechart.", ); const unknown = node( @@ -1406,31 +3137,165 @@ test("from-spec reports a broken PROPERTY in each form that TLC reports it in", `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, ).replace( "====", - `ChargedAtMostOnce == [](charges <= 1)\nNeverChargedAgain == [][charges' <= 1]_charges\nStartsPaying == screen = "Paying"\n====`, + `ChargedAtMostOnce == [](charges <= 1)\nNeverChargedAgain == [][charges' <= 1]_charges\nStartsPaying == screen = "Paying"\nNeverTwiceNorRefunded == [](charges <= 1) /\\ [](screen # "Refunded")\nStartsPayingUncharged == screen = "Paying" /\\ charges = 0\n====`, ), ); writeFileSync( join(dir, "Pay.cfg"), - "SPECIFICATION Spec\nPROPERTIES ChargedAtMostOnce NeverChargedAgain StartsPaying\n", + "SPECIFICATION Spec\nPROPERTIES ChargedAtMostOnce NeverChargedAgain StartsPaying NeverTwiceNorRefunded StartsPayingUncharged\n", ); writeFileSync(join(dir, "Pay.map.mjs"), payMap); const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); assert.equal(r.status, 1, r.stderr); const broken = (p) => ` ${p} is broken. The chart draws only INVARIANT rules, so read TLC's counterexample in ${join("Pay-statechart", "tlc", `Pay-${p}.out`)}.`; + const initialOnly = (p) => + ` TLC checked ${p} only in the initial states, because it has no [], <>, or ~>. List it as an INVARIANT to check every state.`; assert.equal( r.stdout.split("\n\n")[0], `TLC on Pay.tla with Pay.cfg: Pay.cfg has no INVARIANT, so the chart marks no state broken. ${broken("ChargedAtMostOnce")} ${broken("NeverChargedAgain")} -${broken("StartsPaying")} - TLC ran 4 times in N s.`, +${broken("StartsPaying")}${initialOnly("StartsPaying")} +${broken("NeverTwiceNorRefunded")} +${broken("StartsPayingUncharged")}${initialOnly("StartsPayingUncharged")} + TLC ran 6 times in N s.`, + ); + const tlcDir = join(dir, "Pay-statechart", "tlc"); + assert.match( + readFileSync(join(tlcDir, "Pay-NeverTwiceNorRefunded.out"), "utf8"), + /^Error: Invariant line \d+, col \d+ to line \d+, col \d+ of module Pay is violated\.$/m, + ); + assert.match( + readFileSync(join(tlcDir, "Pay-StartsPayingUncharged.out"), "utf8"), + /^Error: Property line \d+, col \d+ to line \d+, col \d+ of module Pay is violated by the initial state:$/m, ); }); test( - "a state reached only after a bug takes the bug's color, and the fix edge leaves the state before the changed step", + "from-spec reports a broken refinement of an instanced module in its output, the stories, and the summary", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "TicketApi.tla"), + `---- MODULE TicketApi ---- +VARIABLE status +Init == status = "open" +Take == status = "open" /\\ status' = "taken" +Close == status = "taken" /\\ status' = "closed" +Spec == Init /\\ [][Take \\/ Close]_status +==== +`, + ); + writeFileSync( + join(dir, "Desk.tla"), + `---- MODULE Desk ---- +VARIABLES phase, agent +vars == <> +Init == phase = "open" /\\ agent = "none" +Assign == phase = "open" /\\ phase' = "taken" /\\ agent' = "ana" +Finish == phase = "taken" /\\ phase' = "closed" /\\ UNCHANGED agent +Drop == phase = "open" /\\ phase' = "closed" /\\ UNCHANGED agent +Spec == Init /\\ [][Assign \\/ Finish \\/ Drop]_vars +Known == phase \\in {"open", "taken", "closed"} +Api == INSTANCE TicketApi WITH status <- phase +\\* A ticket closes only after someone takes it. +ApiRefines == Api!Spec +StartsOpen == phase = "open" +==== +`, + ); + writeFileSync( + join(dir, "Desk.cfg"), + "SPECIFICATION Spec\nINVARIANT Known\nPROPERTIES ApiRefines StartsOpen\n", + ); + writeFileSync( + join(dir, "Desk.map.mjs"), + `export const readerActions = []; +export const state = (v) => "Ticket " + v.phase; +export const step = (action) => ({ Assign: "an agent takes it", Finish: "the agent closes it", Drop: "it closes unassigned" })[action]; +`, + ); + const r = node( + "statechart.mjs", + ["from-spec", "Desk.tla", "--expect-broken", "ApiRefines"], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const out = join(dir, "Desk-statechart"); + assert.equal( + r.stdout.split("\n\n")[0], + `TLC on Desk.tla with Desk.cfg: + Known holds. + ApiRefines is broken (expected). The chart draws only INVARIANT rules, so read TLC's counterexample in ${join("Desk-statechart", "tlc", "Desk-ApiRefines.out")}. + StartsOpen holds. TLC checked StartsOpen only in the initial states, because it has no [], <>, or ~>. List it as an INVARIANT to check every state. + TLC ran 4 times in N s.`, + ); + const stories = `## Broken rules that the chart does not draw + +Rule broken: A ticket closes only after someone takes it. TLC's counterexample is in ${join("tlc", "Desk-ApiRefines.out")}. +`; + assert.ok(r.stdout.includes(`\nNo state breaks an INVARIANT rule.\n\n${stories}`), r.stdout); + assert.equal(readFileSync(join(out, "stories.md"), "utf8"), stories); + assert.deepEqual(JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).brokenProperties, [ + { + spec: "Desk.tla", + property: "ApiRefines", + rule: "A ticket closes only after someone takes it.", + repeats: null, + counterexample: join("tlc", "Desk-ApiRefines.out"), + }, + ]); + }, +); + +test( + "from-spec explains an invariant and a property that name rules of a named INSTANCE with substitutions in the instanced module's words", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "LampRules.tla"), lampRules); + writeFileSync( + join(dir, "Lamp.tla"), + `---- MODULE Lamp ---- +VARIABLES lamp, cable +Rules == INSTANCE LampRules WITH light <- lamp, wire <- cable +Safe == Rules!SafeLight +Sound == + Rules!NeverShorts +${lampSteps}`, + ); + writeFileSync(join(dir, "Lamp.cfg"), "SPECIFICATION Spec\nINVARIANT Safe\nPROPERTIES Sound\n"); + writeFileSync(join(dir, "Lamp.map.mjs"), lampMap); + const r = node( + "statechart.mjs", + ["from-spec", "Lamp.tla", "--expect-broken", "Safe", "--expect-broken", "Sound"], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const out = join(dir, "Lamp-statechart"); + const stories = readFileSync(join(out, "stories.md"), "utf8"); + assert.match(stories, /^Rule broken: The light never comes on over a shorted wire\.$/m); + assert.match(stories, /^1\. The wire shorts\. Now: Off\n2\. You switch it on\. Now: On$/m); + assert.ok( + stories.includes( + `\nRule broken: The wire never shorts. TLC's counterexample is in ${join("tlc", "Lamp-Sound.out")}.\n`, + ), + stories, + ); + assert.deepEqual( + JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).brokenProperties.map( + (b) => b.rule, + ), + ["The wire never shorts."], + ); + }, +); + +test( + "a state reached only after a bug gets its own color, and the fix edge leaves the state before the changed step", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); @@ -1455,17 +3320,6 @@ test( assert.match(tlc(dir, "Pay").stdout, /Invariant NoDoubleCharge is violated/); assert.doesNotMatch(tlc(dir, "PayFixed").stdout, /is violated/); - const statusOf = (chart) => { - const names = new Map( - [...chart.matchAll(/^ {2}state "([^"]+)" as (s\d+)$/gm)].map((m) => [ - m[2], - m[1].replace(//g, " "), - ]), - ); - return Object.fromEntries( - [...chart.matchAll(/^ {2}class (s\d+) (\w+)$/gm)].map((m) => [names.get(m[1]), m[2]]), - ); - }; const args = [ "--dot", "Pay.dot", @@ -1484,7 +3338,7 @@ test( Paying: "works", "Paying, charged": "works", "Paying, charged twice": "wrong", - "Refunded, charged": "wrong", + "Refunded, charged": "after", }); const withFix = [ @@ -1550,28 +3404,12 @@ test( }; assert.match( story(), - /^In the spec you see the same screens and steps, but the spec does not check this rule\.$/m, + /^In the spec you see the same states and steps, but the spec does not check this rule\.$/m, ); assert.match( story("--against-traces", "Pay.out", "--against-rules", "NoDoubleCharge"), - /^In the spec you see the same screens and steps, so the spec does not change this path\.$/m, + /^In the spec you see the same states and steps, so the spec does not change this path\.$/m, ); - const alone = node( - "tlc-to-chart.mjs", - [ - "--dot", - "Pay.dot", - "--map", - "pay.map.mjs", - "--against-dot", - "Pay.dot", - "--against-traces", - "Pay.out", - ], - dir, - ); - assert.equal(alone.status, 2); - assert.match(alone.stderr, /^--against-traces and --against-rules go together\./); }, ); @@ -1621,13 +3459,13 @@ test( TLC ran 3 times in N s. Chart: 5 states and 4 transitions from 5 model states. -3 work, 2 break a rule. +3 work, 1 breaks a rule, 1 reached only after a bug. ## 1. Paying, charged twice Rule broken: \`NoDoubleCharge\` Why: each charge request charges again. -This is the only way to get here in the model. +In the model, one step leads into this bug: "the bank charges you". 0. Start: Cart 1. You pay. Now: Paying @@ -1652,7 +3490,7 @@ Wrote chart.mmd, bug-1.mmd, bug-1-developer.mmd, bug-1-layered.mmd, stories.md, ]); assert.equal( readFileSync(join(dir, "Pay-statechart", "tlc", "Pay.cfg"), "utf8"), - "SPECIFICATION Spec\n", + "SPECIFICATION Spec\nINVARIANT StatechartRules\n", ); assert.equal( readFileSync(join(dir, "Pay-statechart", "tlc", "Pay-NoDoubleCharge.cfg"), "utf8"), @@ -1701,9 +3539,205 @@ NeverSyncedWhileOffline == ~(conn = "Offline" /\\ badge = "Synced") summary.classes.map((c) => [c.end, c.readerSteps]), [["Offline | Synced", 0]], ); - assert.deepEqual(summary.chartStatus, { works: 2, wrong: 1, fixed: 0 }); + assert.deepEqual(summary.chartStatus, { works: 2, wrong: 1, fixed: 0, mixed: 0, after: 0 }); }); +test( + "a screen that a clean path reaches is marked wrong in some cases when a state under it breaks a rule after a bug", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Fork.tla"), + `---- MODULE Fork ---- +VARIABLE screen +Init == screen = "start" +Slip == screen = "start" /\\ screen' = "bad" +Go == screen = "start" /\\ screen' = "good" +Fall == screen = "bad" /\\ screen' = "worse" +Spec == Init /\\ [][Slip \\/ Go \\/ Fall]_screen +NeverBroken == screen \\notin {"bad", "worse"} +==== +`, + ); + writeFileSync(join(dir, "Fork.cfg"), "SPECIFICATION Spec\nINVARIANT NeverBroken\n"); + writeFileSync( + join(dir, "Fork.map.mjs"), + 'export const state = (v) => ({ start: "Home", bad: "Error", good: "Done", worse: "Done" })[v.screen];\nexport const step = (a) => ({ Slip: "you slip", Go: "you go", Fall: "you fall" })[a];\n', + ); + const r = node("statechart.mjs", ["from-spec", "Fork.tla"], dir); + assert.equal(r.status, 1, r.stderr); + const out = join(dir, "Fork-statechart"); + assert.deepEqual(statusOf(readFileSync(join(out, "chart.mmd"), "utf8")), { + Done: "mixed", + Error: "wrong", + Home: "works", + }); + const summary = JSON.parse(readFileSync(join(out, "summary.json"), "utf8")); + assert.deepEqual(summary.chartStatus, { works: 1, wrong: 1, fixed: 0, mixed: 1, after: 0 }); + assert.deepEqual(summary.mixedStatus, ["Done"]); + assert.match( + r.stdout, + /\n1 work, 1 breaks a rule, 1 breaks a rule in some cases\.\n\nThis state breaks a rule in some cases and not in others:\n {2}Done\n/, + ); + }, +); + +test( + "a bug that ends on the start screen's label marks that screen wrong in some cases, and a screen reached only after the bug gets its own color", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Echo.tla"), + `---- MODULE Echo ---- +VARIABLE screen +Init == screen = "idle" +Lose == screen = "idle" /\\ screen' = "lost" +Leave == screen = "lost" /\\ screen' = "gone" +Spec == Init /\\ [][Lose \\/ Leave]_screen +NeverLost == screen # "lost" +==== +`, + ); + writeFileSync(join(dir, "Echo.cfg"), "SPECIFICATION Spec\nINVARIANT NeverLost\n"); + writeFileSync( + join(dir, "Echo.map.mjs"), + 'export const state = (v) => (v.screen === "gone" ? "Away" : "Home");\nexport const step = (a) => ({ Lose: "the note is lost", Leave: "you leave" })[a];\n', + ); + const r = node("statechart.mjs", ["from-spec", "Echo.tla"], dir); + assert.equal(r.status, 1, r.stderr); + const out = join(dir, "Echo-statechart"); + const chart = readFileSync(join(out, "chart.mmd"), "utf8"); + assert.deepEqual(statusOf(chart), { Away: "after", Home: "mixed" }); + assert.deepEqual( + [...chart.matchAll(/^ {4}state "([^"]+)" as legend\w+$/gm)].map((m) => m[1]), + ["Breaks a rule in some cases", "Reached only after a bug"], + ); + const summary = JSON.parse(readFileSync(join(out, "summary.json"), "utf8")); + assert.deepEqual(summary.chartStatus, { works: 0, wrong: 0, fixed: 0, mixed: 1, after: 1 }); + assert.deepEqual(summary.mixedStatus, ["Home"]); + }, +); + +test( + "a bug story numbers the step into the bug and each step that changes what the rule reads, and names the steps it hides", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Reads.tla"), + `---- MODULE Reads ---- +EXTENDS Naturals +VARIABLES screen, reads, ticks +vars == <> +Init == screen = "Idle" /\\ reads = 0 /\\ ticks = 0 +Open == screen = "Idle" /\\ screen' = "Open" /\\ UNCHANGED <> +Read == screen = "Open" /\\ reads < 2 /\\ ticks = 2 * reads /\\ reads' = reads + 1 /\\ UNCHANGED <> +Tick == screen = "Open" /\\ ticks < 2 * reads /\\ ticks' = ticks + 1 /\\ UNCHANGED <> +Spec == Init /\\ [][Open \\/ Read \\/ Tick]_vars +ReadOnce == reads <= 1 +==== +`, + ); + writeFileSync(join(dir, "Reads.cfg"), "SPECIFICATION Spec\nINVARIANT ReadOnce\n"); + writeFileSync( + join(dir, "reads.map.mjs"), + 'export const state = (v) => v.screen;\nexport const step = (a) => ({ Open: "you open the page", Read: "the page is read" })[a] ?? "";\n', + ); + assert.match(tlc(dir, "Reads").stdout, /Invariant ReadOnce is violated/); + const run = (...flags) => { + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "Reads.dot", "--map", "reads.map.mjs", "--traces", "Reads.out", ...flags], + ...["--story", "story.md", "--summary", "summary.json", "--out", "chart.mmd"], + ], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const story = readFileSync(join(dir, "story.md"), "utf8"); + return { + steps: story.slice(story.indexOf("0. Start")), + readerSteps: JSON.parse(readFileSync(join(dir, "summary.json"), "utf8")).classes[0] + .readerSteps, + numbered: edges(parseMermaid(readFileSync(join(dir, "chart.mmd"), "utf8")).regions.Main), + }; + }; + assert.deepEqual(run("--spec", "Reads.tla"), { + steps: `0. Start: Idle +1. You open the page. Now: Open +2. The page is read. Now: Open + (2 hidden steps: Tick) +3. The page is read. Now: Open +`, + readerSteps: 3, + numbered: ["Idle -> Open : 1. you open the page", "Open -> Open : 2, 3. the page is read"], + }); + assert.equal( + run().steps, + `0. Start: Idle +1. You open the page. Now: Open + (3 hidden steps: the page is read, Tick) +2. The page is read. Now: Open +`, + ); + }, +); + +test( + "a story of a bug that only the spec has numbers a step that changes what the spec's rule reads, skips the variables that the spec lacks, and a step that the spec lacks cannot happen in the spec", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Code.tla"), + `---- MODULE Code ---- +VARIABLES s, cache +vars == <> +Init == s = "Idle" /\\ cache = "fresh" +Go == s = "Idle" /\\ s' = "Busy" /\\ UNCHANGED cache +Rot == s = "Busy" /\\ cache = "fresh" /\\ cache' = "stale" /\\ UNCHANGED s +Spec == Init /\\ [][Go \\/ Rot]_vars +NoStale == ~(s = "Busy" /\\ cache = "stale") +==== +`, + ); + writeFileSync( + join(dir, "Agreed.tla"), + `---- MODULE Agreed ---- +VARIABLES s, n +vars == <> +Init == s = "Idle" /\\ n = 0 +Go == s = "Idle" /\\ s' = "Busy" /\\ UNCHANGED n +Tick == s = "Busy" /\\ n = 0 /\\ n' = 1 /\\ UNCHANGED s +Fail == s = "Busy" /\\ n = 1 /\\ s' = "Broken" /\\ UNCHANGED n +Spec == Init /\\ [][Go \\/ Tick \\/ Fail]_vars +NoStale == ~(s = "Broken" /\\ n = 1) +==== +`, + ); + for (const name of ["Code", "Agreed"]) + writeFileSync(join(dir, `${name}.cfg`), "SPECIFICATION Spec\nINVARIANT NoStale\n"); + writeFileSync(join(dir, "Code.map.mjs"), "export const state = (v) => v.s;\n"); + const r = node( + "statechart.mjs", + ["from-spec", "Code.tla", "--against", "Agreed.tla", "--expect-broken", "NoStale"], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const stories = readFileSync(join(dir, "Code-statechart", "stories.md"), "utf8"); + assert.match( + stories, + /^## 2\. Only in the spec: Broken\n[^#]*\n0\. Start: Idle\n1\. Go\. Now: Busy\n2\. Tick\. Now: Busy\n3\. Fail\. Now: Broken\n/m, + ); + assert.match( + stories, + /^1\. Go\. Now: Busy\n2\. Rot\. Now: Busy\n\nIn the spec, step 2 \(Rot\) cannot happen, so you stay at: Busy\.\n/m, + ); + }, +); + const edit = (name, fixed) => `---- MODULE ${name} ---- VARIABLES s, flashed vars == <> @@ -1719,7 +3753,7 @@ NoLostEdit == s # "Lost edit" /\\ ~(s = "Flash" /\\ flashed) `; test( - "a bug that only the fixed model has is numbered after the current bugs and drawn wrong", + "a bug that only the fixed model has is numbered after the current bugs and drawn as breaking a rule only with the fix", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); @@ -1776,15 +3810,165 @@ test( [2, "Flash", "new with the fix"], ], ); - assert.deepEqual(summary.chartStatus, { works: 2, wrong: 1, fixed: 1 }); + assert.deepEqual(summary.chartStatus, { + works: 2, + wrong: 0, + fixed: 1, + mixed: 0, + after: 0, + worse: 1, + }); + assert.equal(statusOf(readFileSync(join(dir, "chart.mmd"), "utf8")).Flash, "worse"); assert.match(readFileSync(join(dir, "story.md"), "utf8"), /^## 2\. New with the fix: Flash$/m); const two = node( "tlc-to-chart.mjs", [...args, "--highlight", "2", "--path-only", "--out", "bug2.mmd"], dir, ); - assert.equal(two.status, 0, two.stderr); - assert.match(readFileSync(join(dir, "bug2.mmd"), "utf8"), /: 2\. the screen flashes\n/); + assert.equal(two.status, 0, two.stderr); + assert.match(readFileSync(join(dir, "bug2.mmd"), "utf8"), /: 2\. the screen flashes\n/); + }, +); + +test( + "with --fixed, a state that breaks a rule only with the fix has its own status, legend line, and count", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const counter = (name, leap) => `---- MODULE ${name} ---- +EXTENDS Naturals +VARIABLE x +Init == x = 0 +CountUp == x < 2 /\\ x' = x + 1 +Leap == x \\in {1, 2} /\\ x' = 5 +Spec == Init /\\ [][CountUp${leap ? " \\/ Leap" : ""}]_x +\\* The counter never shows five. +NotFive == x # 5 +==== +`; + writeFileSync(join(dir, "Counter.tla"), counter("Counter", false)); + writeFileSync(join(dir, "CounterFixed.tla"), counter("CounterFixed", true)); + writeFileSync(join(dir, "Counter.cfg"), "SPECIFICATION Spec\nINVARIANT NotFive\n"); + writeFileSync( + join(dir, "Counter.map.mjs"), + `export const state = (v) => \`The counter shows \${v.x}\`; +export const step = (a) => ({ CountUp: "the counter goes up", Leap: "the counter leaps" })[a]; +`, + ); + const r = node( + "statechart.mjs", + ["from-spec", "Counter.tla", "--fixed", "CounterFixed.tla"], + dir, + ); + assert.equal(r.status, 1, r.stdout + r.stderr); + assert.match( + r.stdout, + /\n3 work today, 0 still wrong with the fix, 0 fixed or new with the fix, 1 breaks a rule only with the fix\.\n\n## 1\. New with the fix: The counter shows 5\n/, + ); + const out = join(dir, "Counter-statechart"); + const chart = readFileSync(join(out, "chart.mmd"), "utf8"); + assert.equal(statusOf(chart)["The counter shows 5"], "worse"); + assert.deepEqual( + [...chart.matchAll(/^ {4}state "([^"]+)" as legend\w+$/gm)].map((m) => m[1]), + ["Works today", "Breaks a rule only with the fix"], + ); + assert.deepEqual(JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).chartStatus, { + works: 3, + wrong: 0, + fixed: 0, + mixed: 0, + after: 0, + worse: 1, + }); + }, +); + +test( + "only --cover trims the overview, and it lists what it leaves out and counts every state", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Route.tla"), + `---- MODULE Route ---- +VARIABLE x +Init == x = "start" +Side == x = "start" /\\ x' = "porch" +Front == x = "start" /\\ x' = "hall" +Arrive == x = "porch" /\\ x' = "attic" +Leave == x = "attic" /\\ x' = "start" +Spec == Init /\\ [][Side \\/ Front \\/ Arrive \\/ Leave]_x +NoPorch == x # "porch" +==== +`, + ); + writeFileSync(join(dir, "Route.cfg"), "SPECIFICATION Spec\nINVARIANT NoPorch\n"); + const mapping = (readerActions) => + `export const readerActions = ${JSON.stringify(readerActions)}; +export const state = (v) => v.x; +export const step = (a) => ({ Side: "you take the side door", Front: "you take the front door", Arrive: "you climb the stairs", Leave: "the lights go out" })[a]; +`; + writeFileSync(join(dir, "Route.map.mjs"), mapping(["Side", "Front", "Arrive"])); + writeFileSync(join(dir, "nobody.map.mjs"), mapping(["Knock"])); + const run = (out, ...flags) => { + const r = node( + "statechart.mjs", + ["from-spec", "Route.tla", "--expect-broken", "NoPorch", "--out", out, ...flags], + dir, + ); + assert.equal(r.status, 0, r.stdout + r.stderr); + return { r, summary: JSON.parse(readFileSync(join(dir, out, "summary.json"), "utf8")) }; + }; + + const whole = run("whole"); + assert.equal(whole.summary.chartStates, 4); + assert.equal(whole.summary.chartTransitions, 4); + assert.deepEqual(whole.summary.chartStatus, { + works: 2, + wrong: 1, + fixed: 0, + mixed: 0, + after: 1, + }); + assert.equal(whole.summary.cover, undefined); + assert.doesNotMatch(whole.r.stdout, /--cover/); + + const covered = run("covered", "--cover"); + assert.equal(covered.summary.chartStates, 3); + assert.equal(covered.summary.chartTransitions, 2); + assert.deepEqual(covered.summary.chartStatus, { + works: 2, + wrong: 1, + fixed: 0, + mixed: 0, + after: 1, + }); + assert.deepEqual(covered.summary.cover, { + drawn: true, + states: 4, + transitions: 4, + leftOutStates: ["attic"], + leftOutTransitions: [ + "attic --> start: the lights go out", + "porch --> attic: you climb the stairs", + ], + }); + assert.match( + covered.r.stdout, + /^--cover leaves out 1 of 4 states and 2 of 4 transitions\. The counts of states that work or break a rule include the states it leaves out\.\nStates left out:\n {2}attic\nTransitions left out:\n {2}attic --> start: the lights go out\n {2}porch --> attic: you climb the stairs\n2 work, 1 breaks a rule, 1 reached only after a bug\.$/m, + ); + + const none = run("none", "--cover", "--map", "nobody.map.mjs"); + assert.equal(none.summary.chartStates, 4); + assert.equal(none.summary.chartTransitions, 4); + assert.deepEqual(none.summary.cover, { + drawn: false, + why: "no step in TLC's graph is a reader action", + }); + assert.match( + none.r.stdout, + /^--cover draws every state, because no step in TLC's graph is a reader action\.$/m, + ); }, ); @@ -1946,17 +4130,18 @@ TLC on SaveAgreed.tla with SaveCode.cfg: NoFalseSaved holds. TLC ran 3 times in N s. -Chart: 4 states and 4 transitions from 10 model states. +Chart: 4 states and 6 transitions from 10 model states. 3 work, 1 breaks a rule, 0 only in the agreed chart. ## 1. Saved, draft Rule broken: You never see "Saved" while the note is still a draft. Why: the app stops after one retry and shows Saved anyway. -There are 2 ways to get here in the model. This is the shortest. +In the model, one step leads into this bug: "the save times out". The steps below are the shortest path that TLC found. 0. Start: Editing, draft 1. You press Save. Now: Saving, draft + (1 hidden step: the app tries again) 2. The save times out. Now: Saved, draft In the agreed chart, step 2 (the save times out) cannot happen, so you stay at: Saving, draft. @@ -1983,7 +4168,7 @@ ${uncitedWarning}`; ); assert.match( chart, - /^ {2}s3 --> s3 : in the agreed chart, the savetimes out cannot happen$/m, + /^ {2}s3 --> s3 : in the agreed chart, the savetimes out cannot happen atstep 2 of bug 1$/m, ); const bug = readFileSync(join(out, "bug-1.mmd"), "utf8"); assert.deepEqual([...bug.matchAll(/^ {2}state "([^"]+)" as s\d+$/gm)].map((m) => m[1]).sort(), [ @@ -2038,11 +4223,33 @@ ${uncitedWarning}`; "${v.note.text} r${v.note.rev}`", ), ); - const lacks = run("--map", "rev.map.mjs", "--expect-broken", "NoFalseSaved"); + const tmp = join(dir, "tmp"); + mkdirSync(tmp); + const lacks = node( + "statechart.mjs", + [ + ...base, + "--against-config", + "SaveCode.cfg", + "--map", + "rev.map.mjs", + "--expect-broken", + "NoFalseSaved", + ], + dir, + { TMPDIR: tmp }, + ); assert.equal(lacks.status, 2); + const [, kept] = lacks.stderr.match( + / The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/, + ); assert.equal( lacks.stderr, - `${warnings}rev.map.mjs: the agreed chart has no note.rev, and the mapping reads v.note.rev on its states. Label states and steps only from what both models have, or test "rev" in v.note first.\n`, + `${warnings}rev.map.mjs: the agreed chart has no note.rev, and the mapping reads v.note.rev on its states. Label states and steps only from what both models have, or test "rev" in v.note first. The files that this run wrote are in ${kept}.\n`, + ); + assert.match( + readFileSync(join(dir, kept, "tlc", "SaveCode-NoFalseSaved.out"), "utf8"), + /^Error: Invariant NoFalseSaved is violated/m, ); writeFileSync( join(dir, "rev.map.mjs"), @@ -2113,8 +4320,603 @@ ${uncitedWarning}`; const fresh = run("--expect-broken", "NoFalseSaved"); assert.equal(fresh.status, 0, fresh.stderr); assert.match( - fresh.stdout, - /\nNot listed: no config of SaveAgreed\.tla expects TLC to break this rule\. Its configs expect TLC to break `TypeOK`\.\n/, + fresh.stdout, + /\nNot listed: no config of SaveAgreed\.tla expects TLC to break this rule\. Its configs expect TLC to break `TypeOK`\.\n/, + ); + }, +); + +const cartCode = `---- MODULE CartCode ---- +VARIABLES phase, note, tag, mark, extra +vars == <> +Init == phase = "cart" /\\ note = [text |-> "hi"] /\\ tag = [id |-> "a"] /\\ mark = [on |-> TRUE] /\\ extra = "kept" +Pay == phase = "cart" /\\ phase' = "paid" /\\ note' = [text |-> note.text, rev |-> 1] /\\ UNCHANGED <> +Back == phase = "paid" /\\ phase' = "cart" /\\ UNCHANGED <> +Spec == Init /\\ [][Pay \\/ Back]_vars +Ok == phase \\in {"cart", "paid"} +==== +`; +const cartAgreed = `---- MODULE CartAgreed ---- +VARIABLES phase, note, tag, mark +vars == <> +Init == phase = "cart" /\\ note = [text |-> "hi"] /\\ tag = "a" /\\ mark = [on |-> TRUE] +Pay == phase = "cart" /\\ phase' = "paid" /\\ mark' = [off |-> TRUE] /\\ UNCHANGED <> +Back == phase = "paid" /\\ phase' = "cart" /\\ UNCHANGED <> +Spec == Init /\\ [][Pay \\/ Back]_vars +Ok == phase \\in {"cart", "paid"} +==== +`; + +test( + "from-spec --against stops when the mapping reads what the spec lacks through a spread, a later state, a field of another type, or a field some states lack", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "CartCode.tla"), cartCode); + writeFileSync(join(dir, "CartAgreed.tla"), cartAgreed); + writeFileSync(join(dir, "CartCode.cfg"), "SPECIFICATION Spec\nINVARIANT Ok\n"); + const map = (name, state, more = "") => + writeFileSync( + join(dir, name), + `export const readerActions = ["Pay", "Back"]; +export const state = ${state}; +export const step = (a) => ({ Pay: "you pay", Back: "you go back" })[a]; +${more}`, + ); + const tmp = join(dir, "tmp"); + mkdirSync(tmp); + const stops = (name, message) => { + const r = node( + "statechart.mjs", + [ + "from-spec", + "CartCode.tla", + "--against", + "CartAgreed.tla", + "--label", + "the agreed chart", + "--against-config", + "CartCode.cfg", + "--map", + name, + ], + dir, + { TMPDIR: tmp }, + ); + assert.equal(r.status, 2, r.stdout); + const [, kept] = + r.stderr.match(/ The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/) ?? []; + assert.equal(r.stderr, `${message} The files that this run wrote are in ${kept}.\n`); + }; + + map("spread.map.mjs", "(v) => { const c = { ...v }; return `${c.phase} ${c.extra}`; }"); + stops( + "spread.map.mjs", + "spread.map.mjs: the agreed chart has no extra, and the mapping lists the fields of v on its states. Label states and steps only from what both models have, and read the fields it needs by name.", + ); + + map("later.map.mjs", "(v) => `${v.phase} ${v.note.rev}`"); + stops( + "later.map.mjs", + 'later.map.mjs: the agreed chart has no note.rev, and the mapping reads v.note.rev on its states. Label states and steps only from what both models have, or test "rev" in v.note first.', + ); + + map("typed.map.mjs", "(v) => `${v.phase} ${v.tag.id}`"); + stops( + "typed.map.mjs", + "typed.map.mjs: the agreed chart has tag as a value of another type, and the mapping reads v.tag on its states. Label states and steps only from what both models have.", + ); + + map("partial.map.mjs", "(v) => `${v.phase} ${v.mark.on}`"); + stops( + "partial.map.mjs", + 'partial.map.mjs: the mapping labels a state of the agreed chart "paid undefined", so it reads a field the agreed chart does not have there. Label states and steps only from what both models have.', + ); + + map( + "guarded.map.mjs", + "(v) => v.phase", + "export const guard = (a, b) => `tag ${b.mark.on}`;\n", + ); + stops( + "guarded.map.mjs", + 'guarded.map.mjs: the mapping labels a step of the agreed chart "tag undefined", so it reads a field the agreed chart does not have there. Label states and steps only from what both models have.', + ); + }, +); + +const seatsSpec = `---- MODULE Seats ---- +VARIABLES seat, grid +vars == <> +Init == seat = <<"empty", "empty">> /\\ grid = << <<"a", "b">>, <<"c", "d">> >> +Fill == seat[1] = "empty" /\\ seat' = [seat EXCEPT ![1] = "full"] /\\ UNCHANGED grid +Next == Fill +Spec == Init /\\ [][Next]_vars +==== +`; + +function seats() { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "Seats.tla"), seatsSpec); + writeFileSync(join(dir, "Seats.cfg"), "SPECIFICATION Spec\n"); + const draw = (name, map) => { + writeFileSync(join(dir, `${name}.map.mjs`), map); + return node( + "statechart.mjs", + ["from-spec", "Seats.tla", "--map", `${name}.map.mjs`, "--out", name], + dir, + ); + }; + const chartOf = (name) => readFileSync(join(dir, name, "chart.mmd"), "utf8"); + const states = (name, map) => { + const r = draw(name, map); + assert.equal(r.status, 0, r.stderr); + return Object.keys(statusOf(chartOf(name))).sort(); + }; + return { draw, chartOf, states }; +} + +test( + "a map reads a tuple from 1, as in TLA+, stops when it reads index 0, and keeps length, array methods, spread, and JSON", + needsTlc, + () => { + const { draw, chartOf } = seats(); + + const first = draw( + "first", + 'export const state = (v) => `first seat ${v.seat[1]}`;\nexport const step = () => "you take the first seat";\n', + ); + assert.equal(first.status, 0, first.stderr); + assert.deepEqual(edges(parseMermaid(chartOf("first")).regions.Main), [ + "first seat empty -> first seat full : you take the first seat", + ]); + + const zero = draw("zero", "export const state = (v) => `first seat ${v.seat[0]}`;\n"); + assert.equal(zero.status, 2, zero.stdout); + assert.match( + zero.stderr, + /^zero\.map\.mjs: the mapping reads v\.seat\[0\], .*reads from 1.*\bv\.seat\[1\]/m, + ); + + const shapes = draw( + "shapes", + 'export const state = (v) =>\n `${v.seat.filter((s) => s === "full").length} of ${v.seat.length} full, ${[...v.seat].join(" and ")}, corner ${v.grid[1][2]}`;\nexport const step = (action, before, after) => JSON.stringify(after);\n', + ); + assert.equal(shapes.status, 0, shapes.stderr); + const chart = chartOf("shapes"); + assert.deepEqual(Object.keys(statusOf(chart)).sort(), [ + "0 of 2 full, empty and empty, corner b", + "1 of 2 full, full and empty, corner b", + ]); + assert.deepEqual( + [...chart.matchAll(/^ {2}s\d+ --> s\d+ : (.*)$/gm)].map((m) => m[1]), + [ + JSON.stringify({ + grid: [ + ["a", "b"], + ["c", "d"], + ], + seat: ["full", "empty"], + }), + ], + ); + }, +); + +const zeroIndex = (at) => + new RegExp( + String.raw`the mapping looks for index 0 in ${at}, .*reads from 1, as in TLA\+\. Use \[\.\.\.${at}\] for an array that reads from 0\.`, + ); + +test( + "a map's in and hasOwnProperty see a tuple's indexes from 1, and asking for index 0 stops the run", + needsTlc, + () => { + const { draw, states } = seats(); + assert.deepEqual( + states( + "has", + "export const state = (v) =>\n `${v.seat[1]}: ${[1, 2, 3].map((i) => i in v.seat)} ${v.seat.hasOwnProperty(2)}`;\n", + ), + ["empty: true,true,false true", "full: true,true,false true"], + ); + + const zero = draw("zero", "export const state = (v) => `${0 in v.seat}`;\n"); + assert.equal(zero.status, 2, zero.stdout); + assert.match( + zero.stderr, + new RegExp(`^zero\\.map\\.mjs: ${zeroIndex("v\\.seat").source}`, "m"), + ); + }, +); + +test("a map's Object.keys and Object.entries list a tuple's indexes from 1", needsTlc, () => { + const { states } = seats(); + assert.deepEqual( + states( + "keys", + 'export const state = (v) =>\n `${v.seat[1]}: ${Object.keys(v.seat)} ${Object.entries(v.seat).map(([i, s]) => `${i}=${s}`).join(" ")}`;\n', + ), + ["empty: 1,2 1=empty 2=empty", "full: 1,2 1=full 2=empty"], + ); +}); + +test("a map that spreads a tuple into an object gets its items at keys from 1", needsTlc, () => { + const { states } = seats(); + assert.deepEqual( + states( + "spread", + "export const state = (v) => {\n const s = { ...v.seat };\n return `${s[1]} ${s[2]} ${0 in s}`;\n};\n", + ), + ["empty empty false", "full empty false"], + ); +}); + +test( + "a map that hands a tuple to concat stops the run and names the tuple it wrote", + needsTlc, + () => { + const { draw } = seats(); + const r = draw("concat", 'export const state = (v) => [].concat(v.seat).join(" ");\n'); + assert.equal(r.status, 2, r.stdout); + assert.match(r.stderr, new RegExp(`^concat\\.map\\.mjs: ${zeroIndex("v\\.seat").source}`, "m")); + }, +); + +test( + "a map flattens nested tuples with flat and flatMap, and flattening an array that holds a tuple stops the run", + needsTlc, + () => { + const { draw, states } = seats(); + assert.deepEqual( + states( + "flat", + 'export const state = (v) =>\n `${v.seat[1]} ${v.grid.flat().join("")} ${v.grid.flatMap((row) => row).join("")} ${v.grid.flat(Infinity).join("")}`;\n', + ), + ["empty abcd abcd abcd", "full abcd abcd abcd"], + ); + + const inArray = draw("inarray", 'export const state = (v) => [v.seat].flat().join(" ");\n'); + assert.equal(inArray.status, 2, inArray.stdout); + assert.match( + inArray.stderr, + new RegExp(`^inarray\\.map\\.mjs: ${zeroIndex("v\\.seat").source}`, "m"), + ); + }, +); + +test("a map flattens a set of tuples with flat and flatMap", needsTlc, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pairs.tla"), + `---- MODULE Pairs ---- +VARIABLES sent +Init == sent = {<<1, "a">>} +Send == sent = {<<1, "a">>} /\\ sent' = sent \\cup {<<2, "b">>} +Spec == Init /\\ [][Send]_sent +==== +`, + ); + writeFileSync(join(dir, "Pairs.cfg"), "SPECIFICATION Spec\n"); + writeFileSync( + join(dir, "Pairs.map.mjs"), + 'export const state = (v) =>\n `${v.sent.flat().join("")} ${v.sent.flatMap((m) => m).join("")} ${structuredClone(v.sent).flat().join("")}`;\n', + ); + const r = node("statechart.mjs", ["from-spec", "Pairs.tla", "--out", "out"], dir); + assert.equal(r.status, 0, r.stderr); + assert.deepEqual( + Object.keys(statusOf(readFileSync(join(dir, "out", "chart.mmd"), "utf8"))).sort(), + ["1a 1a 1a", "1a2b 1a2b 1a2b"], + ); +}); + +test("a map's structuredClone copies tuples that still read and write from 1", needsTlc, () => { + const { draw, states } = seats(); + assert.deepEqual( + states( + "clone", + 'export const state = (v) => {\n const c = structuredClone(v);\n c.seat[2] = "held";\n return `${c.seat[1]} ${c.seat[2]} ${v.seat[2]} ${c.grid[2][1]} ${Object.keys(c.seat)} ${structuredClone(v.grid)[2][2]}`;\n};\n', + ), + ["empty held empty c 1,2 d", "full held empty c 1,2 d"], + ); + + const zero = draw( + "clonezero", + 'export const state = (v) => {\n const c = structuredClone(v);\n c.seat[0] = "held";\n return c.seat[1];\n};\n', + ); + assert.equal(zero.status, 2, zero.stdout); + assert.match( + zero.stderr, + /^clonezero\.map\.mjs: the mapping writes v\.seat\[0\], but a tuple reads from 1, as in TLA\+\. Write v\.seat\[1\] for its first item\./m, + ); +}); + +test( + "from-spec --against draws a step that step() does not name by its action in the developer and layered views", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const packing = (name, more = "") => `---- MODULE ${name} ---- +VARIABLES phase +Init == phase = "cart" +Pay == phase = "cart" /\\ phase' = "paid" +Pack == phase = "paid" /\\ phase' = "packed" +Ship == phase = "packed" /\\ phase' = "shipped" +${more}Spec == Init /\\ [][Pay \\/ Pack \\/ Ship${more && " \\/ Refund"}]_phase +NotShipped == phase # "shipped" +==== +`; + writeFileSync(join(dir, "PackCode.tla"), packing("PackCode")); + writeFileSync( + join(dir, "PackAgreed.tla"), + packing("PackAgreed", `Refund == phase = "paid" /\\ phase' = "cart"\n`), + ); + writeFileSync(join(dir, "PackCode.cfg"), "SPECIFICATION Spec\nINVARIANT NotShipped\n"); + writeFileSync( + join(dir, "pack.map.mjs"), + `export const readerActions = ["Pay"]; +export const state = (v) => v.phase; +export const detail = (v) => v.phase; +export const step = (a) => ({ Pay: "you pay", Ship: "we ship" })[a]; +`, + ); + const r = node( + "statechart.mjs", + [ + ...["from-spec", "PackCode.tla", "--against", "PackAgreed.tla"], + ...["--label", "the agreed chart", "--against-config", "PackCode.cfg"], + ...["--map", "pack.map.mjs", "--out", "out"], + ], + dir, + ); + assert.equal(r.status, 1, r.stderr); + assert.doesNotMatch(r.stderr, /the mapping labels/); + for (const view of ["developer", "layered"]) { + const chart = readFileSync(join(dir, "out", `bug-1-${view}.mmd`), "utf8"); + assert.doesNotMatch(chart, /undefined/, view); + assert.match(chart, /--> s\d+ : 2\. Pack$/m, view); + } + }, +); + +test( + "from-spec --against compares a rule's variable of another type without blaming the mapping", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "ShopCode.tla"), + `---- MODULE ShopCode ---- +VARIABLES phase, tag +vars == <> +Init == phase = "cart" /\\ tag = [id |-> "a"] +Pay == phase = "cart" /\\ phase' = "paid" /\\ UNCHANGED tag +Spec == Init /\\ [][Pay]_vars +Ok == ~(phase = "paid" /\\ tag = tag) +==== +`, + ); + writeFileSync( + join(dir, "ShopAgreed.tla"), + `---- MODULE ShopAgreed ---- +VARIABLES phase, tag, seen +vars == <> +Init == phase = "cart" /\\ tag = "a" /\\ seen = FALSE +Peek == ~seen /\\ seen' = TRUE /\\ tag' = "b" /\\ UNCHANGED phase +Pay == phase = "cart" /\\ phase' = "paid" /\\ UNCHANGED <> +Spec == Init /\\ [][Peek \\/ Pay]_vars +Ok == ~(phase = "paid" /\\ tag = tag) +==== +`, + ); + writeFileSync(join(dir, "ShopCode.cfg"), "SPECIFICATION Spec\nINVARIANT Ok\n"); + writeFileSync( + join(dir, "shop.map.mjs"), + `export const readerActions = ["Pay"]; +export const state = (v) => v.phase; +export const step = (a) => ({ Pay: "you pay" })[a]; +`, + ); + const r = node( + "statechart.mjs", + [ + "from-spec", + "ShopCode.tla", + "--against", + "ShopAgreed.tla", + "--label", + "the agreed chart", + "--against-config", + "ShopCode.cfg", + "--map", + "shop.map.mjs", + ], + dir, + ); + assert.equal(r.status, 1, r.stderr); + assert.doesNotMatch(r.stderr, /the mapping reads/); + }, +); + +const shipCode = `---- MODULE ShipCode ---- +EXTENDS Naturals +VARIABLES phase, n, mark +vars == <> +Init == phase = "cart" /\\ n = 0 /\\ mark = [on |-> TRUE] +Pay == phase = "cart" /\\ phase' = "paid" /\\ n' = 1 /\\ UNCHANGED mark +Ship == phase = "paid" /\\ phase' = "shipped" /\\ UNCHANGED <> +Spec == Init /\\ [][Pay \\/ Ship]_vars +Ok == phase \\in {"cart", "paid", "shipped"} +NotShipped == phase # "shipped" +==== +`; +const shipAgreed = `---- MODULE ShipAgreed ---- +VARIABLES phase, n, mark +vars == <> +Init == phase = "cart" /\\ n = "0" /\\ mark = [on |-> TRUE] +Pay == phase = "cart" /\\ phase' = "paid" /\\ n' = "1" /\\ UNCHANGED mark +Ship == phase = "paid" /\\ phase' = "shipped" /\\ mark' = "off" /\\ UNCHANGED n +Spec == Init /\\ [][Pay \\/ Ship]_vars +Ok == phase \\in {"cart", "paid", "shipped"} +NotShipped == phase # "shipped" +==== +`; +const shipMap = (state) => `export const readerActions = ["Pay", "Ship"]; +export const state = ${state}; +export const step = (a) => ({ Pay: "you pay", Ship: "we ship" })[a]; +`; +const stopsWith = (r, message) => { + assert.equal(r.status, 2, r.stdout); + const [, kept] = + r.stderr.match(/ The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/) ?? []; + assert.equal(r.stderr, `${message} The files that this run wrote are in ${kept}.\n`); +}; +const codeAgainstAgreed = [ + ...["ShipCode.tla", "--against", "ShipAgreed.tla", "--label", "the agreed chart"], + ...["--against-config", "ShipCode.cfg"], +]; +function shipAgainst(map, models = codeAgainstAgreed, agreed = shipAgreed) { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "ShipCode.tla"), shipCode); + writeFileSync(join(dir, "ShipAgreed.tla"), agreed); + writeFileSync(join(dir, "ShipCode.cfg"), "SPECIFICATION Spec\nINVARIANT Ok\n"); + writeFileSync(join(dir, "ship.map.mjs"), map); + mkdirSync(join(dir, "tmp")); + return node("statechart.mjs", ["from-spec", ...models, "--map", "ship.map.mjs"], dir, { + TMPDIR: join(dir, "tmp"), + }); +} + +test( + "from-spec --against stops when the mapping reads a field that one model has as a number and the other as a string", + needsTlc, + () => { + stopsWith( + shipAgainst(shipMap('(v) => `${v.phase}, ${v.n === 0 ? "free" : "charged"}`')), + "ship.map.mjs: the agreed chart has n as a value of another type, and the mapping reads v.n on its states. Label states and steps only from what both models have.", + ); + stopsWith( + shipAgainst(shipMap('(v) => `${v.phase}, ${v.n === "0" ? "free" : "charged"}`'), [ + "ShipAgreed.tla", + "--config", + "ShipCode.cfg", + "--against", + "ShipCode.tla", + ]), + "ship.map.mjs: the spec has n as a value of another type, and the mapping reads v.n on its states. Label states and steps only from what both models have.", + ); + }, +); + +test( + "from-spec --against stops when the mapping reads a field that is a record in some of the spec's states and not in others", + needsTlc, + () => { + stopsWith( + shipAgainst(shipMap('(v) => `${v.phase}, ${v.mark.on ? "marked" : "plain"}`')), + "ship.map.mjs: the agreed chart has mark as a value of another type, and the mapping reads v.mark on its states. Label states and steps only from what both models have.", + ); + + const where = shipAgainst( + shipMap( + '(v) => (v.phase === "shipped" ? v.phase : `${v.phase}, ${v.mark.on ? "marked" : "plain"}`)', + ), + ); + assert.equal(where.status, 0, where.stderr); + }, +); + +test( + "from-spec --against checks the fields of a record that is a record in only some of the spec's states", + needsTlc, + () => { + const recordStates = shipMap( + '(v) => (v.phase === "shipped" ? v.phase : `${v.phase}, ${v.mark.on === true ? "marked" : "plain"}`)', + ); + stopsWith( + shipAgainst(recordStates, codeAgainstAgreed, shipAgreed.replaceAll("[on |->", "[off |->")), + 'ship.map.mjs: the agreed chart has no mark.on, and the mapping reads v.mark.on on its states. Label states and steps only from what both models have, or test "on" in v.mark first.', + ); + stopsWith( + shipAgainst(recordStates, codeAgainstAgreed, shipAgreed.replaceAll("TRUE", '"yes"')), + "ship.map.mjs: the agreed chart has mark.on as a value of another type, and the mapping reads v.mark.on on its states. Label states and steps only from what both models have.", + ); + }, +); + +test( + "tlc-to-chart stops when the mapping reads a field of another type that only a trace of the second model has", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "ShipCode.tla"), shipCode); + writeFileSync(join(dir, "ShipAgreed.tla"), shipAgreed); + writeFileSync(join(dir, "ShipCode.cfg"), "SPECIFICATION Spec\nINVARIANT NotShipped\n"); + writeFileSync(join(dir, "ShipAgreed.cfg"), "SPECIFICATION Spec\nINVARIANT NotShipped\n"); + tlc(dir, "ShipCode"); + tlc(dir, "ShipAgreed"); + renameSync(join(dir, "ShipAgreed.out"), join(dir, "ShipAgreedTraces.out")); + writeFileSync(join(dir, "ShipAgreed.cfg"), "SPECIFICATION Spec\nCONSTRAINT NotShipped\n"); + tlc(dir, "ShipAgreed"); + assert.doesNotMatch(readFileSync(join(dir, "ShipAgreed.dot"), "utf8"), /shipped/); + writeFileSync( + join(dir, "ship.map.mjs"), + shipMap('(v) => `${v.phase}, ${v.mark.on ? "marked" : "plain"}`'), + ); + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "ShipCode.dot", "--map", "ship.map.mjs", "--traces", "ShipCode.out"], + ...["--against-dot", "ShipAgreed.dot", "--against-traces", "ShipAgreedTraces.out"], + ...["--against-rules", "NotShipped", "--label", "the agreed chart", "--out", "chart.mmd"], + ], + dir, + ); + assert.equal(r.status, 1, r.stdout); + assert.match( + r.stderr, + /^Error: the agreed chart has mark as a value of another type, and the mapping reads v\.mark on its states\./m, + ); + }, +); + +test( + "from-spec --against names the group when the mapping groups a state of the spec by a field that state lacks", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "CartCode.tla"), cartCode); + writeFileSync(join(dir, "CartAgreed.tla"), cartAgreed); + writeFileSync(join(dir, "CartCode.cfg"), "SPECIFICATION Spec\nINVARIANT Ok\n"); + writeFileSync( + join(dir, "group.map.mjs"), + `export const readerActions = ["Pay", "Back"]; +export const state = (v) => v.phase; +export const step = (a) => ({ Pay: "you pay", Back: "you go back" })[a]; +export const group = (v) => \`mark \${v.mark.on}\`; +`, + ); + mkdirSync(join(dir, "tmp")); + const r = node( + "statechart.mjs", + [ + "from-spec", + "CartCode.tla", + "--against", + "CartAgreed.tla", + "--label", + "the agreed chart", + "--against-config", + "CartCode.cfg", + "--map", + "group.map.mjs", + ], + dir, + { TMPDIR: join(dir, "tmp") }, + ); + stopsWith( + r, + 'group.map.mjs: the mapping puts a state of the agreed chart in group "mark undefined", so it reads a field the agreed chart does not have there. Label states and steps only from what both models have.', ); }, ); @@ -2220,6 +5022,44 @@ test( }, ); +test( + "from-spec draws parallel regions whose arrows together make a step back to the same state, which the overview leaves out", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Knobs.tla"), + `---- MODULE Knobs ---- +VARIABLES dial, light +vars == <> +Init == dial = "low" /\\ light = "off" +Turn == dial = "low" /\\ light = "off" /\\ dial' \\in {"low", "mid"} /\\ light' \\in {"off", "on"} +Push == dial = "mid" /\\ dial' = "high" /\\ UNCHANGED light +Next == Turn \\/ Push +Spec == Init /\\ [][Next]_vars +==== +`, + ); + writeFileSync(join(dir, "Knobs.cfg"), "SPECIFICATION Spec\nCHECK_DEADLOCK FALSE\n"); + writeFileSync( + join(dir, "Knobs.map.mjs"), + `export const regionNames = ["Dial", "Light"]; +export const state = (v) => \`\${v.dial}: \${v.light}\`; +export const step = (a) => ({ Turn: "you turn", Push: "you push" })[a]; +`, + ); + const r = node("statechart.mjs", ["from-spec", "Knobs.tla"], dir); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match(r.stdout, /\nIt has 2 parallel regions, of 3 and 2 states\. /); + const again = node( + "compare-charts.mjs", + ["tlc/regions-all.mmd", "tlc/regions-proof.mmd"], + join(dir, "Knobs-statechart"), + ); + assert.equal(again.stdout, "same\n", again.stderr); + }, +); + test( "the mapping's guard words show on an arrow only when every model step behind it has them", needsTlc, @@ -2379,6 +5219,8 @@ test( "probe.map.mjs", "--traces", "Door-NoLostReply.out", + "--broken", + "Door.out", "--highlight", "0", ...extra, @@ -2422,11 +5264,10 @@ export const group = (v) => (v.reply === "shown" ? "Reply shown" : "Waiting"); assert.deepEqual( coarse.stdout.split("\n").filter((l) => /^ {2}(class|classDef) /.test(l)), [ - " classDef plain fill:#f4f6f7,stroke:#566573,color:#1b2631", - " classDef plainWrong fill:#f4f6f7,stroke:#c0392b,stroke-width:4px,color:#7b241c", - " class legendWrong plainWrong", - " class s0 plainWrong", - " class s1 plain", + " classDef plainMixed fill:#f4f6f7,stroke:#d35400,stroke-width:4px,color:#7e5109", + " class legendMixed plainMixed", + " class s0 plainMixed", + " class s1 plainMixed", ], ); const summary = JSON.parse(readFileSync(join(tlcDir, "coarse.json"), "utf8")); @@ -2493,19 +5334,25 @@ test( dir, ); assert.equal(r.status, 1, r.stderr); - const chart = readFileSync(join(dir, "Order-statechart", "chart.mmd"), "utf8"); + const transitions = (f) => + parseMermaid(readFileSync(join(dir, "Order-statechart", f), "utf8")).regions.Main + .transitions; return { story: readFileSync(join(dir, "Order-statechart", "stories.md"), "utf8"), - fixEdges: parseMermaid(chart) - .regions.Main.transitions.filter((t) => t.label.includes("with the fix")) + fixEdges: transitions("chart.mmd") + .filter((t) => t.label.includes("with the fix")) .map((t) => `${t.from} --> ${t.to} : ${t.label}`), + bugLoops: transitions("bug-1.mmd") + .filter((t) => t.from === t.to) + .map((t) => `${t.from} --> ${t.to} : ${t.label}`) + .sort(), }; }; const hidden = withFix(chargeAgain); assert.deepEqual(hidden.fixEdges, []); assert.match( hidden.story, - /^With the fix you see the same screens and steps, so the fix does not change this path\.$/m, + /^With the fix you see the same states and steps, so the fix does not change this path\.$/m, ); const once = withFix(chargeOnce); assert.deepEqual(once.fixEdges, [ @@ -2523,13 +5370,242 @@ test( assert.deepEqual(unseen.fixEdges, ["Paying --> Paid : with the fix, the bank charges you"]); assert.match( unseen.story, - /^With the fix, after step 1, "the bank charges you" happens and takes you to: Paid\.$/m, + /^With the fix, step 3 \(the bank charges you\) takes you to: Paid\.$/m, ); const blocked = withFix(chargeFirstOnly, unlabeledMap); - assert.deepEqual(blocked.fixEdges, []); + assert.deepEqual(blocked.fixEdges, [ + "Paying --> Paying : with the fix, the bank charges you cannot happen at step 3 of bug 1", + ]); + assert.deepEqual(blocked.bugLoops, [ + "Paying --> Paying : 2, 3. the bank charges you", + "Paying --> Paying : with the fix, the bank charges you cannot happen at step 3", + ]); assert.match( blocked.story, - /yet the rule holds, so the state labels do not show what the fix changes/, + /^With the fix, step 3 \(the bank charges you\) cannot happen, so you stay at: Paying\.$/m, + ); + }, +); + +test( + "a step that the fix cannot take keeps its own loop when the fix keeps another bug on the same state", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const shop = (name, steps) => `---- MODULE ${name} ---- +VARIABLES screen, charged, refunded, declined +vars == <> +Init == screen = "Cart" /\\ charged = FALSE /\\ refunded = FALSE /\\ declined = FALSE +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED <> +${steps} +Spec == Init /\\ [][Next]_vars +NoCharge == ~charged +NoRefund == ~refunded +==== +`; + writeFileSync( + join(dir, "Shop.tla"), + shop( + "Shop", + `Charge == screen = "Paying" /\\ ~charged /\\ charged' = TRUE /\\ UNCHANGED <> +Refund == screen = "Paying" /\\ ~refunded /\\ refunded' = TRUE /\\ UNCHANGED <> +Next == Pay \\/ Charge \\/ Refund`, + ), + ); + writeFileSync( + join(dir, "ShopFixed.tla"), + shop( + "ShopFixed", + `Charge == screen = "Paying" /\\ ~declined /\\ declined' = TRUE /\\ UNCHANGED <> +Next == Pay \\/ Charge`, + ), + ); + writeFileSync( + join(dir, "Shop.cfg"), + "SPECIFICATION Spec\nINVARIANT NoCharge\nINVARIANT NoRefund\n", + ); + writeFileSync( + join(dir, "Shop.map.mjs"), + `export const readerActions = ["Pay", "Charge", "Refund"]; +export const state = (v) => v.screen; +export const step = (a, v, w) => ({ Pay: "you pay", Charge: w.charged ? "the bank charges you" : "the bank declines", Refund: "the bank refunds you" })[a]; +export const cause = () => null; +`, + ); + const r = node("statechart.mjs", ["from-spec", "Shop.tla", "--fixed", "ShopFixed.tla"], dir); + assert.equal(r.status, 1, r.stderr); + const loops = parseMermaid(readFileSync(join(dir, "Shop-statechart", "chart.mmd"), "utf8")) + .regions.Main.transitions.filter((t) => t.from === t.to) + .map((t) => t.label) + .sort(); + assert.deepEqual(loops, [ + "the bank charges you or refunds you with the fix, the bank declines", + "with the fix, the bank refunds you cannot happen at step 2 of bug 2", + ]); + }, +); + +test( + "the replay on the fix skips only the steps that the story hides, so a step that the fix stops cannot happen", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Order.tla"), + `---- MODULE Order ---- +EXTENDS Naturals +VARIABLES screen, charges +vars == <> +Init == screen = "Cart" /\\ charges = 0 +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED charges +Charge == screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen +Spec == Init /\\ [][Pay \\/ Charge]_vars +NoDoubleCharge == charges <= 1 +==== +`, + ); + writeFileSync( + join(dir, "OrderFixed.tla"), + `---- MODULE OrderFixed ---- +EXTENDS Naturals +VARIABLES screen, charges +vars == <> +Init == screen = "Cart" /\\ charges = 0 +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED charges +Charge == screen = "Paying" /\\ charges < 1 /\\ charges' = charges + 1 /\\ UNCHANGED screen +Refund == charges = 1 /\\ charges' = 0 /\\ UNCHANGED screen +Spec == Init /\\ [][Pay \\/ Charge \\/ Refund]_vars +NoDoubleCharge == charges <= 1 +==== +`, + ); + writeFileSync(join(dir, "Order.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Order.map.mjs"), "export const state = (v) => v.screen;\n"); + const r = node("statechart.mjs", ["from-spec", "Order.tla", "--fixed", "OrderFixed.tla"], dir); + assert.equal(r.status, 1, r.stderr); + const story = readFileSync(join(dir, "Order-statechart", "stories.md"), "utf8"); + assert.match( + story, + /\n0\. Start: Cart\n1\. Pay\. Now: Paying\n2\. Charge\. Now: Paying\n3\. Charge\. Now: Paying\n\nWith the fix, step 3 \(Charge\) cannot happen, so you stay at: Paying\.\n/, + ); + }, +); + +const retry = (name, act, again) => `---- MODULE ${name} ---- +EXTENDS Naturals +VARIABLES screen, charges, key +vars == <> +Init == screen = "Cart" /\\ charges = 0 /\\ key = FALSE +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ charges' = 1 /\\ UNCHANGED key +FetchKey == screen = "Paying" /\\ ~key /\\ key' = TRUE /\\ UNCHANGED <> +${act} == screen = "Paying" /\\ ${again} +Spec == Init /\\ [][Pay \\/ FetchKey \\/ ${act}]_vars +NoDoubleCharge == charges <= 1 +==== +`; + +test( + "the replay takes only the trace's own steps, matches a step by its words between the same two states, and quotes the second model's words", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Retry.tla"), + retry( + "Retry", + "Retry", + "charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED <>", + ), + ); + writeFileSync(join(dir, "Retry.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync( + join(dir, "Retry.map.mjs"), + `export const state = (v) => v.screen; +export const step = (a, before, after) => + a === "Pay" ? "you pay" + : a === "FetchKey" ? "the key arrives" + : after.charges > before.charges ? "you retry and the bank charges you again" + : "you retry and see the first charge"; +`, + ); + const withFix = (act, again) => { + writeFileSync(join(dir, "RetryFixed.tla"), retry("RetryFixed", act, again)); + const r = node( + "statechart.mjs", + ["from-spec", "Retry.tla", "--fixed", "RetryFixed.tla"], + dir, + ); + assert.equal(r.status, 1, r.stderr); + const story = readFileSync(join(dir, "Retry-statechart", "stories.md"), "utf8"); + assert.match( + story, + /\n0\. Start: Cart\n1\. You pay\. Now: Paying\n2\. You retry and the bank charges you again\. Now: Paying\n\n/, + ); + return story.split("\n").find((l) => l.startsWith("With the fix")); + }; + assert.equal( + withFix("Retry", `key /\\ screen' = "Paid" /\\ UNCHANGED <>`), + "With the fix, step 2 (you retry and the bank charges you again) cannot happen, so you stay at: Paying.", + ); + assert.equal( + withFix("Retry", `screen' = "Paid" /\\ UNCHANGED <>`), + 'With the fix, step 2 (you retry and the bank charges you again) happens as "you retry and see the first charge" and takes you to: Paid.', + ); + assert.equal( + withFix( + "Recharge", + "charges < 2 /\\ charges' = charges + 1 /\\ key' = TRUE /\\ UNCHANGED screen", + ), + "With the fix you see the same states and steps, so the fix does not change this path.", + ); + }, +); + +const upload = (name, cache, copy) => `---- MODULE ${name} ---- +EXTENDS Naturals +VARIABLES screen, cached, copies, merged +vars == <> +Init == screen = "Picking" /\\ cached = FALSE /\\ copies = 0 /\\ merged = FALSE +Pick == screen = "Picking" /\\ screen' = "Ready" /\\ UNCHANGED <> +Cache == screen = "Ready" /\\ ~cached /\\ cached' = TRUE /\\ ${cache} +Upload == screen = "Ready" /\\ cached /\\ copies < 2 /\\ copies' = copies + 1 /\\ ${copy} +Spec == Init /\\ [][Pick \\/ Cache \\/ Upload]_vars +OneCopy == copies <= 1 \\/ merged +==== +`; + +test( + "the replay says which hidden step goes elsewhere in the fix, and when the rule holds on the same states and steps", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const keepCache = "UNCHANGED <>"; + const keepCopy = "UNCHANGED <>"; + writeFileSync(join(dir, "Upload.tla"), upload("Upload", keepCache, keepCopy)); + writeFileSync(join(dir, "Upload.cfg"), "SPECIFICATION Spec\nINVARIANT OneCopy\n"); + writeFileSync(join(dir, "Upload.map.mjs"), "export const state = (v) => v.screen;\n"); + const withFix = (cache, copy) => { + writeFileSync(join(dir, "UploadFixed.tla"), upload("UploadFixed", cache, copy)); + const r = node( + "statechart.mjs", + ["from-spec", "Upload.tla", "--fixed", "UploadFixed.tla"], + dir, + ); + assert.equal(r.status, 1, r.stderr); + const story = readFileSync(join(dir, "Upload-statechart", "stories.md"), "utf8"); + assert.match( + story, + /\n0\. Start: Picking\n1\. Pick\. Now: Ready\n {3}\(1 hidden step: Cache\)\n2\. Upload\. Now: Ready\n3\. Upload\. Now: Ready\n\n/, + ); + return story.split("\n").find((l) => l.startsWith("With the fix")); + }; + assert.equal( + withFix(`screen' = "Cached" /\\ UNCHANGED <>`, keepCopy), + 'With the fix, after step 1, "Cache" happens and takes you to: Cached.', + ); + assert.equal( + withFix(keepCache, "merged' = (copies = 1) /\\ UNCHANGED <>"), + "With the fix you see the same states and steps, yet the rule holds, so the state labels do not show what the fix changes.", ); }, ); @@ -2612,7 +5688,7 @@ export const cause = () => null; assert.doesNotMatch(story, /do not show what/); assert.match( story, - /^With the badge fix, after step 3, "a message arrives" happens and takes you to: Settings, muted, badge\.$/m, + /^With the badge fix, step 4 \(a message arrives\) takes you to: Settings, muted, badge\.$/m, ); const settings = classes.find((c) => c.end === "Settings, muted"); assert.deepEqual(fixEdges(`bug-${settings.n}.mmd`), [ @@ -2739,6 +5815,16 @@ Spec == \\* not [][Inc]_x unnamed.stderr, /^Count\.tla: 1 step has the name Next, the whole next-state action/, ); + writeFileSync( + join(dir, "Count.tla"), + count("Reset == x = 3 /\\ x' = 0\nNext == (Inc \\/ Reset) /\\ x \\in 0..3"), + ); + const conjunct = node("statechart.mjs", ["from-spec", "Count.tla"], dir); + assert.equal(conjunct.status, 0, conjunct.stderr); + assert.equal( + conjunct.stderr, + "Count.tla: 4 steps have the name Next, the whole next-state action. TLC goes down from Next through each \\/, \\E, LET, and definition, and names each step after the last definition on that path. It stops at any other expression, such as /\\, IF, or CASE. If the reader must tell these steps apart, join a definition for each kind of step with \\/ in Next, as in Next == Pay \\/ Charge, and move each conjunct of Next into those definitions.\n", + ); writeFileSync(join(dir, "Count.tla"), count("Next == Inc")); const named = node("statechart.mjs", ["from-spec", "Count.tla"], dir); assert.equal(named.status, 0, named.stderr); @@ -2821,6 +5907,19 @@ const ownTmp = (dir, name) => { return tmp; }; const tlcStarted = (tmp) => readdirSync(tmp).some((f) => f.startsWith("tlc-")); +const tlcChecking = (tmp) => { + try { + return readdirSync(tmp, { recursive: true }).some((f) => + /^tlc-[^/]+\/[^/]+\/[^/]+\.fp$/.test(f), + ); + } catch { + return false; + } +}; +const dumpChecking = (tmp) => + readdirSync(tmp).some( + (f) => f.startsWith("statechart-") && existsSync(join(tmp, f, "tlc", "Pay.cfg")), + ) && tlcChecking(tmp); const staged = (dir) => readdirSync(dir).filter((f) => f.startsWith(".")); test( @@ -2860,8 +5959,7 @@ const changedMidRun = async ({ wrap = [], env = {} }) => { const out = writeSlowPay(dir, 20000); const tmp = ownTmp(dir, "tmp-a"); const first = startRun(dir, { wrap, env: { ...env, TMPDIR: tmp } }); - await until(() => tlcStarted(tmp)); - await sleep(500); + await until(() => dumpChecking(tmp)); writeSlowPay(dir, 20000, "Receipt"); const second = startRun(dir, { wrap, env: { ...env, TMPDIR: ownTmp(dir, "tmp-b") } }); const [paid, receipt] = await Promise.all([first.done, second.done]); @@ -2883,6 +5981,35 @@ test( () => changedMidRun({}), ); +test( + "a run whose inputs get new times and a new inode but the same content still charts them", + needsTlc, + async () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const out = writeSlowPay(dir, 20000); + const tmp = ownTmp(dir, "tmp"); + const run = startRun(dir, { env: { TMPDIR: tmp } }); + await until(() => tlcStarted(tmp)); + await sleep(500); + const later = new Date(Date.now() + 60_000); + for (const name of ["Pay.tla", "Pay.cfg", "Pay.map.mjs"]) { + const file = join(dir, name); + const before = statSync(file); + writeFileSync(`${file}.new`, readFileSync(file)); + renameSync(`${file}.new`, file); + utimesSync(file, later, later); + const after = statSync(file); + assert.notEqual(after.ino, before.ino); + assert.notEqual(after.ctimeMs, before.ctimeMs); + } + const r = await run.done; + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, ""); + assert.match(readFileSync(join(out, "chart.mmd"), "utf8"), /Paid/); + assert.deepEqual(staged(dir), []); + }, +); + const unshare = ["unshare", "--user", "--map-root-user", "--pid", "--fork", "--mount-proc"]; const hidePidNamespace = `#define _GNU_SOURCE #include @@ -2937,19 +6064,89 @@ test( ); const failRenames = `#define _GNU_SOURCE +#include #include #include +#include +#include #include #include +#include #include static size_t hits; +static int raced; +static int removeOne(const char *path, const struct stat *s, int type, struct FTW *at) { + return remove(path); +} +static int hasFiles(const char *dir) { + DIR *d = opendir(dir); + if (!d) return 0; + struct dirent *e; + int found = 0; + while (!found && (e = readdir(d))) found = strcmp(e->d_name, ".") && strcmp(e->d_name, ".."); + closedir(d); + return found; +} +static int named(const char *path, const char *tail) { + size_t n = strlen(path), t = strlen(tail); + return n >= t && strcmp(path + n - t, tail) == 0 && (n == t || path[n - t - 1] == '/'); +} +static void makeAnotherRunsFolder(const char *shared) { + static int (*make)(const char *, mode_t); + if (!make) make = dlsym(RTLD_NEXT, "mkdir"); + char chart[4096]; + make(shared, 0777); + snprintf(chart, sizeof chart, "%s/chart.mmd", shared); + FILE *f = fopen(chart, "w"); + fputs("another run\\n", f); + fclose(f); +} +static int chosen(const char *race) { + const char *run = getenv("ANOTHER_RUN"); + return run && !raced && strcmp(run, race) == 0; +} +static int racing(const char *race) { + return chosen(race) && (raced = 1); +} +static void takeOver(const char *shared) { + rmdir(shared); + makeAnotherRunsFolder(shared); +} +int mkdir(const char *path, mode_t mode) { + static int (*real)(const char *, mode_t); + static int (*move)(const char *, const char *); + if (!real) real = dlsym(RTLD_NEXT, "mkdir"); + if (!move) move = dlsym(RTLD_NEXT, "rename"); + if (named(path, "shared") && racing("before-copy")) makeAnotherRunsFolder(path); + if (named(path, "shared") && chosen("when-made")) { + int made = real(path, mode); + if (made == 0 && racing("when-made")) takeOver(path); + return made; + } + if (named(path, "shared/tlc") && racing("during-copy")) { + char shared[4096], aside[4200]; + snprintf(shared, sizeof shared, "%.*s", (int)(strlen(path) - strlen("/tlc")), path); + snprintf(aside, sizeof aside, "%s-aside", shared); + move(shared, aside); + makeAnotherRunsFolder(shared); + real(path, mode); + } + if (named(path, "shared/tlc") && racing("removed-during-copy")) { + char shared[4096]; + snprintf(shared, sizeof shared, "%.*s", (int)(strlen(path) - strlen("/tlc")), path); + nftw(shared, removeOne, 16, FTW_DEPTH | FTW_PHYS); + real(shared, 0777); + } + return real(path, mode); +} int rename(const char *from, const char *to) { static int (*real)(const char *, const char *); if (!real) real = dlsym(RTLD_NEXT, "rename"); const char *base = strrchr(to, '/'), *fails = getenv("RENAME_FAILS"); const char *cannot = getenv("RENAME_CANNOT"); base = base ? base + 1 : to; - if (cannot && strstr(base, "shared")) { + if (named(to, "shared") && !hasFiles(from) && racing("before-copy")) makeAnotherRunsFolder(to); + if (cannot && strstr(base, "shared") && hasFiles(from)) { errno = strcmp(cannot, "EXDEV") == 0 ? EXDEV : access(to, F_OK) == 0 ? EISDIR : ENOSYS; return -1; } @@ -2957,7 +6154,9 @@ int rename(const char *from, const char *to) { errno = fails[hits++] == 'N' ? ENOTEMPTY : EIO; return -1; } - return real(from, to); + int renamed = real(from, to); + if (renamed == 0 && named(to, "shared") && racing("when-made")) takeOver(to); + return renamed; } `; const failingRenames = (() => { @@ -2986,63 +6185,141 @@ test( const [, kept] = back.stderr.match( /^cannot replace shared: EIO: .*\. The run left shared as it was\. The files that this run wrote are in (\/\S+)\.\n$/, ); - assert.equal(readFileSync(join(kept, "chart.mmd"), "utf8"), chart); - assert.equal(readFileSync(join(shared, "chart.mmd"), "utf8"), chart); + assert.equal(readFileSync(join(kept, "chart.mmd"), "utf8"), chart); + assert.equal(readFileSync(join(shared, "chart.mmd"), "utf8"), chart); + assert.deepEqual(staged(dir), []); + + const stuck = run("NII"); + assert.equal(stuck.status, 2, stuck.stderr); + const named = stuck.stderr.match( + /^cannot replace shared: EIO: .*\. This run moved the previous shared to (\.shared-\w+-old) and could not put it back\. The files that this run wrote are in \/\S+\.\n$/, + ); + assert.ok(named, stuck.stderr); + const aside = named[1]; + assert.equal(existsSync(shared), false); + assert.equal(readFileSync(join(dir, aside, "chart.mmd"), "utf8"), chart); + assert.deepEqual(staged(dir), [aside]); + }, +); + +test( + "a run on a filesystem that cannot rename copies its files into place", + { skip: needsTlc.skip || (!failingRenames && "cannot compile a rename preload") }, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeLamp(dir); + const args = ["from-spec", "Lamp.tla", "--out", "shared"]; + for (const [cannot, label] of [ + ["ENOSYS", "you flip the switch"], + ["EXDEV", "you press the switch"], + ["ENOSYS", "you tap the switch"], + ]) { + writeFileSync( + join(dir, "Lamp.map.mjs"), + `export const state = (v) => (v.on ? "On" : "Off");\nexport const step = () => "${label}";\n`, + ); + const r = node("statechart.mjs", args, dir, { + LD_PRELOAD: failingRenames, + RENAME_CANNOT: cannot, + }); + assert.equal(r.status, 0, r.stderr); + assert.match(readFileSync(join(dir, "shared", "chart.mmd"), "utf8"), new RegExp(label)); + assert.deepEqual(staged(dir), []); + } + }, +); + +test( + "a run that cannot rename replaces a folder that another run makes before its copy, and keeps one that another run makes during its copy", + { skip: needsTlc.skip || (!failingRenames && "cannot compile a rename preload") }, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeLamp(dir); + const shared = join(dir, "shared", "chart.mmd"); + const run = (anotherRun) => + node("statechart.mjs", ["from-spec", "Lamp.tla", "--out", "shared"], dir, { + LD_PRELOAD: failingRenames, + RENAME_CANNOT: "ENOSYS", + ANOTHER_RUN: anotherRun, + }); + + const before = run("before-copy"); + assert.equal(before.status, 0, before.stderr); + assert.match(readFileSync(shared, "utf8"), /you flip the switch/); + assert.deepEqual(staged(dir), []); + + const during = run("during-copy"); + assert.equal(during.status, 2, during.stderr); + const named = during.stderr.match( + /^cannot replace shared: \S*EEXIST\S*: .*\. This run moved the previous shared to (\.shared-\w+-old) and could not put it back\. The files that this run wrote are in \/\S+\.\n$/, + ); + assert.ok(named, during.stderr); + assert.equal(readFileSync(shared, "utf8"), "another run\n"); + assert.deepEqual(readdirSync(join(dir, "shared", "tlc")), []); + assert.match(readFileSync(join(dir, named[1], "chart.mmd"), "utf8"), /you flip the switch/); + assert.deepEqual(staged(dir), [named[1]]); + }, +); + +test( + "a run that cannot rename stops when another run removes its folder during its copy and makes an empty one in its place, and keeps that folder", + { skip: needsTlc.skip || (!failingRenames && "cannot compile a rename preload") }, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeLamp(dir); + const r = node("statechart.mjs", ["from-spec", "Lamp.tla", "--out", "shared"], dir, { + LD_PRELOAD: failingRenames, + RENAME_CANNOT: "ENOSYS", + ANOTHER_RUN: "removed-during-copy", + }); + assert.equal(r.status, 2, r.stderr); + assert.match( + r.stderr, + /^cannot replace shared: another run replaced the folder during the copy\. The run left shared as it was\. The files that this run wrote are in \/\S+\.\n$/, + ); + assert.ok(existsSync(join(dir, "shared"))); assert.deepEqual(staged(dir), []); - - const stuck = run("NII"); - assert.equal(stuck.status, 2, stuck.stderr); - const named = stuck.stderr.match( - /^cannot replace shared: EIO: .*\. This run moved the previous shared to (\.shared-\w+-old) and could not put it back\. The files that this run wrote are in \/\S+\.\n$/, - ); - assert.ok(named, stuck.stderr); - const aside = named[1]; - assert.equal(existsSync(shared), false); - assert.equal(readFileSync(join(dir, aside, "chart.mmd"), "utf8"), chart); - assert.deepEqual(staged(dir), [aside]); }, ); test( - "a run on a filesystem that cannot rename copies its files into place", + "a run that cannot rename keeps the folder that another run puts in place of its folder as soon as the run makes it", { skip: needsTlc.skip || (!failingRenames && "cannot compile a rename preload") }, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeLamp(dir); - const args = ["from-spec", "Lamp.tla", "--out", "shared"]; - for (const [cannot, label] of [ - ["ENOSYS", "you flip the switch"], - ["EXDEV", "you press the switch"], - ["ENOSYS", "you tap the switch"], - ]) { - writeFileSync( - join(dir, "Lamp.map.mjs"), - `export const state = (v) => (v.on ? "On" : "Off");\nexport const step = () => "${label}";\n`, - ); - const r = node("statechart.mjs", args, dir, { - LD_PRELOAD: failingRenames, - RENAME_CANNOT: cannot, - }); - assert.equal(r.status, 0, r.stderr); - assert.match(readFileSync(join(dir, "shared", "chart.mmd"), "utf8"), new RegExp(label)); - assert.deepEqual(staged(dir), []); - } + const r = node("statechart.mjs", ["from-spec", "Lamp.tla", "--out", "shared"], dir, { + LD_PRELOAD: failingRenames, + RENAME_CANNOT: "ENOSYS", + ANOTHER_RUN: "when-made", + }); + assert.equal(r.status, 2, r.stderr); + assert.match( + r.stderr, + /^cannot replace shared: \S*EEXIST\S*: .*\. The run left shared as it was\. The files that this run wrote are in \/\S+\.\n$/, + ); + assert.equal(readFileSync(join(dir, "shared", "chart.mmd"), "utf8"), "another run\n"); + assert.deepEqual(staged(dir), []); }, ); test( "a run whose input changes before --png renders stops before it renders, and keeps its files", - needsTlc, + { skip: needsTlc.skip || needsPng.skip }, async () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeSlowPay(dir, 20000); - const bin = ownTmp(dir, "bin"); const rendered = join(dir, "rendered"); - writeFileSync(join(bin, "npx"), `#!/bin/sh\ntouch "${rendered}"\nexit 1\n`, { mode: 0o755 }); const tmp = ownTmp(dir, "tmp"); const run = startRun(dir, { args: ["from-spec", "Pay.tla", "--png"], - env: { TMPDIR: tmp, PATH: `${bin}${delimiter}${process.env.PATH}` }, + env: { + TMPDIR: tmp, + XDG_CACHE_HOME: fakeRenderer(), + RENDERED: rendered, + TLA2TOOLS: tools.jar, + JAVA: tools.java, + }, }); await until(() => tlcStarted(tmp)); const map = join(dir, "Pay.map.mjs"); @@ -3057,8 +6334,98 @@ test( }, ); +const writeCount = (dir) => { + writeFileSync( + join(dir, "Count.tla"), + "---- MODULE Count ----\nEXTENDS Naturals, \\* the bound\n Limits\nVARIABLE n\nInit == n = 0\nUp == n < Max /\\ n' = n + 1\nNext == Up\nSpec == Init /\\ [][Next]_n\nSmall == n <= 2\nLast == n <= 2\n====\n", + ); + writeFileSync(join(dir, "Limits.tla"), "---- MODULE Limits ----\nMax == 2\n====\n"); + writeFileSync(join(dir, "Count.cfg"), "SPECIFICATION Spec\nINVARIANT Small\nINVARIANT Last\n"); + writeFileSync(join(dir, "labels.mjs"), 'export const up = "you add one";\n'); + writeFileSync( + join(dir, "Count.map.mjs"), + 'import { up } from "./labels.mjs";\nexport const state = (v) => `count ${v.n}`;\nexport const step = () => up;\n', + ); + return join(dir, "Count-statechart"); +}; +const EDIT_DURING_TLC = `#!/bin/sh +case "$*" in +*Count-Small.cfg*) cp "$EDITED" "$TARGET";; +*Count-Last.cfg*) [ -z "$RESTORE" ] || cp "$RESTORE" "$TARGET";; +esac +exec "$REAL_JAVA" "$@" +`; +const editDuringTlc = (file, edit, putBack) => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const out = writeCount(dir); + const side = ownTmp(dir, "side"); + const target = join(dir, file); + writeFileSync(join(side, "edited"), edit(readFileSync(target, "utf8"))); + writeFileSync(join(side, "original"), readFileSync(target)); + writeFileSync(join(side, "java"), EDIT_DURING_TLC, { mode: 0o755 }); + const r = node("statechart.mjs", ["from-spec", "Count.tla"], dir, { + JAVA: join(side, "java"), + TLA2TOOLS: tools.jar, + REAL_JAVA: tools.java, + TARGET: target, + EDITED: join(side, "edited"), + RESTORE: putBack ? join(side, "original") : "", + }); + return { r, out }; +}; + +test( + "a spec or a module it EXTENDS that changes during a run and changes back gets the results of the files that the run started with", + needsTlc, + () => { + for (const [file, edit] of [ + ["Count.tla", (s) => s.replace("n < Max", "n < 5")], + ["Limits.tla", (s) => s.replace("Max == 2", "Max == 5")], + ]) { + const { r, out } = editDuringTlc(file, edit, true); + assert.equal(r.status, 0, `${file}: ${r.stdout}${r.stderr}`); + assert.match(r.stdout, /^ {2}Small holds\.\n {2}Last holds\.$/m, file); + assert.match(r.stdout, /^No state breaks an INVARIANT rule\.$/m, file); + assert.ok(existsSync(join(out, "chart.mmd")), file); + } + }, +); + +test( + "a module that the spec EXTENDS or a file that the map imports, changed during a run, stops the run with exit 2", + needsTlc, + () => { + for (const [file, edit] of [ + ["Limits.tla", (s) => s.replace("Max == 2", "Max == 5")], + ["labels.mjs", (s) => s.replace("one", "two")], + ]) { + const { r, out } = editDuringTlc(file, edit, false); + assert.equal(r.status, 2, `${file}: ${r.stdout}${r.stderr}`); + assert.match( + r.stderr, + new RegExp( + `^${file.replace(".", "\\.")} changed while this run read it, so the run left Count-statechart as it was\\. Run the command again\\. The files that this run wrote are in \\S+\\.\\n$`, + ), + ); + assert.equal(existsSync(out), false, file); + } + }, +); + +test("a copy keeps the folders of its input under the snapshot folder, on a Windows drive or share too", () => { + assert.equal(copyPath("/tmp/r", "/home/u/Count.tla", posix), "/tmp/r/home/u/Count.tla"); + assert.equal( + copyPath("C:\\Temp\\r", "C:\\Users\\u\\Count.tla", win32), + "C:\\Temp\\r\\C\\Users\\u\\Count.tla", + ); + assert.equal( + copyPath("C:\\Temp\\r", "\\\\host\\share\\Count.tla", win32), + "C:\\Temp\\r\\hostshare\\Count.tla", + ); +}); + test( - "a rule that TLC finds broken before TLC throws a Java exception is broken, and the run draws the paths TLC printed", + "a rule that TLC finds broken before TLC stops on an error is broken, and the run draws the paths TLC printed", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); @@ -3103,7 +6470,7 @@ exit $s assert.match(r.stdout, /^ {2}NoDoubleCharge is broken \(expected\)\.$/m); assert.match( r.stderr, - /^Pay-statechart\/tlc\/Pay-NoDoubleCharge\.out: TLC found that NoDoubleCharge is broken, and then failed with java\.lang\.ArrayIndexOutOfBoundsException: Index 123 out of bounds for length 4\. .* Do not weaken NoDoubleCharge because of it\./m, + /^Pay-statechart\/tlc\/Pay-NoDoubleCharge\.out: TLC found that NoDoubleCharge is broken, and then stopped on an error: java\.lang\.ArrayIndexOutOfBoundsException: Index 123 out of bounds for length 4\. The paths that TLC printed before it stopped are complete, but TLC did not check every state, so the charts can miss states that break NoDoubleCharge\. A smaller constant can let TLC finish\.$/m, ); assert.match(r.stdout, /^## 1\. Paying, charged twice$/m); }, @@ -3174,6 +6541,253 @@ test( }, ); +test( + "a state graph too large for one string prints the rule results, keeps TLC's output, and draws no chart", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const java = join(dir, "java"); + writeFileSync( + java, + `#!/bin/sh +"$REAL_JAVA" "$@"; s=$? +while [ $# -gt 0 ]; do + [ "$1" = -dump ] && "$NODE" -e 'require("fs").truncateSync(process.argv[1], +process.argv[2])' "$3" "$SIZE" + shift +done +exit $s +`, + { mode: 0o755 }, + ); + const tmp = ownTmp(dir, "tmp"); + const r = node( + "statechart.mjs", + ["from-spec", "Pay.tla", "--expect-broken", "NoDoubleCharge"], + dir, + { + JAVA: java, + TLA2TOOLS: tools.jar, + REAL_JAVA: tools.java, + NODE: process.execPath, + SIZE: String(constants.MAX_STRING_LENGTH + 1), + TMPDIR: tmp, + }, + ); + assert.equal(r.status, 2, r.stderr); + assert.match(r.stdout, /^ {2}NoDoubleCharge is broken \(expected\)\.$/m); + const [, kept] = r.stderr.match( + /^Pay\.tla: TLC's state graph has [\d,]+ bytes, more than the [\d,]+ that the command can read, so it draws no chart\. The rule results above stand\. Smaller constants make a smaller graph\. The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/, + ); + assert.match( + readFileSync(join(dir, kept, "tlc", "Pay-NoDoubleCharge.out"), "utf8"), + /^Error: Invariant NoDoubleCharge is violated/m, + ); + assert.equal(existsSync(join(dir, "Pay-statechart")), false); + rmSync(dir, { recursive: true, force: true }); + }, +); + +test( + "a graph dump that grows past one string is stopped, and the rule results stay", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const pid = join(dir, "dump.pid"); + const java = join(dir, "java"); + writeFileSync( + java, + `#!/bin/sh +for a; do [ "$a" = -dump ] && dumping=1; done +[ -z "$dumping" ] && exec "$REAL_JAVA" "$@" +while [ "$1" != -dump ]; do shift; done +printf 'TLC2 Version 2.19\\n' +"$NODE" -e 'require("fs").writeFileSync(process.argv[1], ""); require("fs").truncateSync(process.argv[1], +process.argv[2])' "$3" "$SIZE" +echo $$ > "$PID" +exec sleep 20 +`, + { mode: 0o755 }, + ); + const tmp = ownTmp(dir, "tmp"); + const r = node( + "statechart.mjs", + ["from-spec", "Pay.tla", "--expect-broken", "NoDoubleCharge"], + dir, + { + JAVA: java, + TLA2TOOLS: tools.jar, + REAL_JAVA: tools.java, + NODE: process.execPath, + SIZE: String(constants.MAX_STRING_LENGTH + 1), + PID: pid, + TMPDIR: tmp, + }, + ); + assert.equal(r.status, 2, r.stderr); + assert.match(r.stdout, /^ {2}NoDoubleCharge is broken \(expected\)\.$/m); + const [, kept] = r.stderr.match( + /^Pay\.tla: TLC's state graph passed [\d,]+ bytes, the most that the command can read, so the command stopped TLC and draws no chart\. The rule results above stand\. Smaller constants make a smaller graph\. The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/, + ); + assert.throws(() => process.kill(Number(readFileSync(pid, "utf8")), 0), { code: "ESRCH" }); + assert.match( + readFileSync(join(dir, kept, "tlc", "Pay-NoDoubleCharge.out"), "utf8"), + /^Error: Invariant NoDoubleCharge is violated/m, + ); + assert.equal(existsSync(join(dir, "Pay-statechart")), false); + rmSync(dir, { recursive: true, force: true }); + }, +); + +test("TLC output that grows past one string stops TLC, and the command exits 2", needsTlc, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay("Pay", `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const pid = join(dir, "tlc.pid"); + const java = join(dir, "java"); + writeFileSync( + java, + `#!/bin/sh +echo $$ > "$PID" +exec "$NODE" -e ' +const size = +process.argv[1]; +const chunk = "x".repeat(1 << 16); +let sent = 0; +process.stdout.write("TLC2 Version 2.19\\n"); +(function more() { + while (sent <= size) { + sent += chunk.length; + if (!process.stdout.write(chunk)) return process.stdout.once("drain", more); + } + setTimeout(() => {}, 20000); +})(); +' "$SIZE" +`, + { mode: 0o755 }, + ); + const tmp = ownTmp(dir, "tmp"); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, { + JAVA: java, + TLA2TOOLS: tools.jar, + NODE: process.execPath, + SIZE: String(constants.MAX_STRING_LENGTH), + PID: pid, + TMPDIR: tmp, + }); + assert.equal(r.status, 2, r.stderr); + const [, out] = r.stderr.match( + /^(\/\S+\/tlc\/Pay-NoDoubleCharge\.out): TLC printed more than [\d,]+ characters, the most that the command can read, so the command stopped TLC\. Smaller constants make TLC print less\.\n$/, + ); + assert.throws(() => process.kill(Number(readFileSync(pid, "utf8")), 0), { code: "ESRCH" }); + assert.equal(existsSync(out), true); + assert.equal(existsSync(join(dir, "Pay-statechart")), false); + rmSync(dir, { recursive: true, force: true }); +}); + +test( + "the rule results print before the graph dump, and stay when TLC stops on an error in the dump", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const java = join(dir, "java"); + writeFileSync( + java, + `#!/bin/sh +for a; do + if [ "$a" = -dump ]; then + printf 'TLC2 Version 2.19\\nError: TLC threw an unexpected exception.\\n' + exit 1 + fi +done +exec "$REAL_JAVA" "$@" +`, + { mode: 0o755 }, + ); + const tmp = ownTmp(dir, "tmp"); + const r = node( + "statechart.mjs", + ["from-spec", "Pay.tla", "--expect-broken", "NoDoubleCharge"], + dir, + { JAVA: java, TLA2TOOLS: tools.jar, REAL_JAVA: tools.java, TMPDIR: tmp }, + ); + assert.equal(r.status, 2, r.stderr); + assert.match(r.stdout, /^ {2}NoDoubleCharge is broken \(expected\)\.$/m); + const [, kept] = r.stderr.match( + /^(\/\S+)\/tlc\/Pay\.out: TLC stopped on an error that is not a broken rule: TLC threw an unexpected exception\.$/m, + ); + assert.match( + readFileSync(join(kept, "tlc", "Pay-NoDoubleCharge.out"), "utf8"), + /^Error: Invariant NoDoubleCharge is violated/m, + ); + rmSync(dir, { recursive: true, force: true }); + }, +); + +test( + "an error after TLC runs keeps TLC's output, and the message names its folder", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync( + join(dir, "Pay.map.mjs"), + payMap.replace( + /^export const state = .*$/m, + `export const state = () => { throw new Error("the map cannot read this state"); };`, + ), + ); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, { + TMPDIR: ownTmp(dir, "tmp"), + }); + assert.equal(r.status, 2, r.stderr); + const [, kept] = r.stderr.match( + / The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/, + ); + assert.match( + readFileSync(join(dir, kept, "tlc", "Pay-NoDoubleCharge.out"), "utf8"), + /^Error: Invariant NoDoubleCharge is violated/m, + ); + assert.equal(existsSync(join(dir, "Pay-statechart")), false); + rmSync(dir, { recursive: true, force: true }); + }, +); + test( "runs of two specs that overlap on one output folder all finish, and the folder always holds one run's files", needsTlc, @@ -3228,64 +6842,13 @@ test( }, ); -test( - "the output folder model keeps every rule, and breaks one with a shared TLC folder, without each guard, or when a failed rename deletes the folder moved aside", - needsTlc, - () => { - const dir = mkdtempSync(join(tmpdir(), "statechart-")); - const model = (ext) => readFileSync(join(scripts, "model", `OutputFolder${ext}`), "utf8"); - const r = node( - "statechart.mjs", - ["from-spec", join(scripts, "model", "OutputFolder.tla"), "--out", join(dir, "out")], - dir, - ); - assert.equal(r.status, 0, r.stdout + r.stderr); - assert.match(r.stdout, /\nNo state breaks a rule\.\n/); - for (const [i, [rule, guard, without, story]] of [ - [ - "ChartsOneSpec", - "/\\ read' = Set(read, r, read[r] \\cup {spec})", - "/\\ read' = [q \\in Runs |-> read[q] \\cup {spec}]", - /You start run 2 in a second shell\..*\n.*Run 2 moves its folder into place\. Now: Two runs going: chart of both specs/, - ], - [ - "ChartsOneSpec", - '/\\ pc[r] = "publish" /\\ spec = snap[r] /\\ ~out.notes', - '/\\ pc[r] = "publish" /\\ ~out.notes', - /You edit the spec\..*\n.*Run 1 moves its folder into place\. Now: One run going: chart of both specs/, - ], - [ - "KeepsYourFiles", - '/\\ pc[r] = "aside" /\\ ~aside[r].notes\n', - '/\\ pc[r] = "aside"\n', - /Run 1 deletes the old folder and finishes\. Now: Done: chart of the spec\n/, - ], - [ - "NeverLosesChart", - " /\\ \\/ /\\ aside[r].exists /\\ ~out.exists\n /\\ out' = aside[r]\n /\\ aside' = Set(aside, r, NoFolder)\n \\/ UNCHANGED <>\n", - " /\\ out' = out\n /\\ aside' = Set(aside, r, IF aside[r].notes THEN aside[r] ELSE NoFolder)\n", - /Run 2 moves the old folder aside\. Now: .*, the old chart moved aside\n.*Run 2 cannot rename a folder and stops\. Now: One run going: no folder\n/, - ], - [ - "NeverLosesChart", - '/\\ pc[r] = "swap" /\\ out.exists /\\ aside[r].exists /\\ ~out.notes', - '/\\ pc[r] = "swap" /\\ out.exists /\\ aside[r].exists', - /You save notes in the folder\..*\n.*Run 2 finds another run's folder in place and deletes the old folder\. Now: Two runs going: your notes\n/, - ], - ].entries()) { - const mutant = join(dir, `${i}-${rule}`); - mkdirSync(mutant); - assert.ok(model(".tla").includes(guard), guard); - writeFileSync(join(mutant, "OutputFolder.tla"), model(".tla").replace(guard, without)); - for (const ext of [".cfg", ".map.mjs"]) - writeFileSync(join(mutant, `OutputFolder${ext}`), model(ext)); - const m = node( - "statechart.mjs", - ["from-spec", "OutputFolder.tla", "--expect-broken", rule], - mutant, - ); - assert.equal(m.status, 0, m.stdout + m.stderr); - assert.match(m.stdout, story); - } - }, -); +test("no file in scripts/ passes 1000 lines", () => { + const long = readdirSync(scripts, { withFileTypes: true }) + .filter((e) => e.isFile()) + .map((e) => [ + e.name, + readFileSync(join(scripts, e.name), "utf8").replace(/\n$/, "").split("\n").length, + ]) + .filter(([, lines]) => lines > 1000); + assert.deepEqual(long, [], "move cohesive parts of each listed file into their own modules"); +}); diff --git a/harness/pstack/claude-code/skills/statechart/scripts/tlc-to-chart-args.mjs b/harness/pstack/claude-code/skills/statechart/scripts/tlc-to-chart-args.mjs new file mode 100644 index 000000000..a2e0a3124 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/tlc-to-chart-args.mjs @@ -0,0 +1,52 @@ +import process from "node:process"; + +export const args = { + traces: [], + fixedTraces: [], + againstTraces: [], + fixedRules: [], + againstRules: [], + specs: [], + next: [], + view: "customer", + highlight: 1, +}; +for (let i = 2; i < process.argv.length; i++) { + const a = process.argv[i]; + const next = () => process.argv[++i]; + if (a === "--dot") args.dot = next(); + else if (a === "--map") args.map = next(); + else if (a === "--traces") args.traces.push(next()); + else if (a === "--broken") args.broken = next(); + else if (a === "--fixed-dot") args.fixedDot = next(); + else if (a === "--fixed-traces") args.fixedTraces.push(next()); + else if (a === "--fixed-rules") args.fixedRules.push(...next().split(",")); + else if (a === "--fixed-broken") args.fixedBroken = next(); + else if (a === "--against-dot") args.againstDot = next(); + else if (a === "--against-traces") args.againstTraces.push(next()); + else if (a === "--against-broken") args.againstBroken = next(); + else if (a === "--against-rules") args.againstRules.push(...next().split(",")); + else if (a === "--listed") args.listed = next(); + else if (a === "--label") args.label = next(); + else if (a === "--spec") args.specs.push(next()); + else if (a === "--next") args.next.push(next()); + else if (a === "--view") args.view = next(); + else if (a === "--highlight") args.highlight = Number(next()); + else if (a === "--path-only") args.pathOnly = true; + else if (a === "--full-labels") args.fullLabels = true; + else if (a === "--flat") args.flat = true; + else if (a === "--loops") args.loops = true; + else if (a === "--regions") args.regions = true; + else if (a === "--proof") args.proof = next(); + else if (a === "--regions-all") args.regionsAll = next(); + else if (a === "--cover") args.cover = true; + else if (a === "--title") args.title = next(); + else if (a === "--out") args.out = next(); + else if (a === "--story") args.story = next(); + else if (a === "--summary") args.summary = next(); + else { + console.error(`unknown argument ${a}.`); + process.exit(2); + } +} +if (args.listed !== undefined) args.listed = JSON.parse(args.listed); diff --git a/harness/pstack/claude-code/skills/statechart/scripts/tlc-to-chart.mjs b/harness/pstack/claude-code/skills/statechart/scripts/tlc-to-chart.mjs index 37ae46e33..7aefe44be 100644 --- a/harness/pstack/claude-code/skills/statechart/scripts/tlc-to-chart.mjs +++ b/harness/pstack/claude-code/skills/statechart/scripts/tlc-to-chart.mjs @@ -8,7 +8,6 @@ import { GROUP_PALETTE, LEGEND_ID, PLAIN_FILL, - compareCharts, escapeMermaidQuotes, groupClassDef, groupColorProblem, @@ -16,251 +15,30 @@ import { joinAnd, joinLabels, parseDot, - parseMermaid, parseTraces, - productOf, - ruleSentences, stateKey, tlcOutputWarning, wrap, STATUS_CLASSES, } from "./lib.mjs"; - -const USAGE = "Usage: node tlc-to-chart.mjs --dot FILE --map FILE [options]"; -const HELP = `${USAGE} - -Collapses a TLC state graph onto the reader's states and writes a Mermaid stateDiagram-v2. -statechart.mjs runs it once for the overview and once for each bug chart. - - --dot FILE The state graph from TLC -dump dot,actionlabels FILE, for the current code. - Required. - --map FILE The mapping module. Required. It exports state(vars, facts), and - optionally detail(vars, facts), step(action, before, after), - guard(action, before, after), timer(action, before, after), - cause(vars, facts, invariant), group(vars, facts), groupColors, - regionNames, readerActions, and legend. guard is when the step - can happen, in plain words. An edge reads "step [words]" when every - model step behind it has words. timer is how many seconds the code waits - before a timer step, and the step reads "after N s, step". - cause is the hidden reason a bad state is bad: counterexamples with the - same invariant and cause are one bug. When cause returns nothing, the - end state stands in for it. readerActions are the actions the - reader takes. group is the group of a state, below. legend, such as - { works: "Works on main" }, gives the legend's words for the works, - wrong, and fixed statuses. - facts.enabled is the set of actions the state enables. facts.settled is - true when only reader actions are enabled, so nothing else happens until - the reader acts. - --traces FILE TLC output that holds counterexamples. Repeat it. TLC -continue gives one - per bad state. It refuses a file from a TLC run that stopped early or hit - an error other than a broken rule. When TLC found a rule broken and then - failed with a Java exception, it keeps the paths that TLC printed and - warns that the chart can miss states that break the rule. - --fixed-dot FILE The state graph of the model with the open fix. Each bug's steps replay - on it. - --fixed-traces FILE TLC output for the same invariants on the fixed model. Repeat it. It marks - states fixed or still wrong, and adds the bugs that only the fixed model - has after the current model's bugs. - --fixed-rules LIST The invariants that the fixed model's TLC runs check, as in A,B. It goes - with --fixed-traces. A rule that the fixed model does not check stays - as wrong as it is today, and its stories say "not checked". - --against-dot FILE The state graph of the spec that the current code must follow. - Each bug's steps replay on it. Do not use it with --fixed-dot. - --against-traces FILE TLC output for the spec. Repeat it. Its bugs are drawn wrong, and the - bugs that only the spec has number after the code's bugs. - --against-rules LIST The invariants that the spec's TLC runs check. It goes with - --against-traces. Stories of a rule that the spec does not check say - "not checked". - --listed JSON What the spec's configs expect, as in - {"spec":"Spec.tla","breaks":{"A.cfg":"Rule"}}: each config whose - "\\* expect:" line names a rule, and that rule. It goes with - --against-dot. Each story then says whether a config of the spec already - expects TLC to break its rule, and summary.json has second.listed and - each bug's listed configs. - --label TEXT The name of the second model in the chart and the stories. Default: - "the fix" with --fixed-dot, "the spec" with --against-dot. - --spec FILE A TLA+ file whose \\* comment lines above a rule are the rule's sentence in - the stories. Repeat it. The first file that has a sentence wins. - --view NAME customer (state labels, the default), developer (detail labels), or - layered (detail inside state). - --cover Keep only the shortest path to each settled reader state, plus the - counterexample paths. - --highlight N Draw bug N as a numbered path. Default 1. 0 draws none. - --path-only Draw only the highlighted path. - --full-labels Write each label on an edge in full, joined by " or ", and mark the chart - "${FULL_LABELS}". Default: write the leading words that the labels - share once, as in "you tap Pay or Back". - --flat Draw every state on its own, with no boxes. - --regions When every state reads "context: progress", draw two parallel regions, - context and progress, when they are smaller and proved. They draw the - steps of the shortest way to each region state, of each bug path, and - the nearest step with each label. - Otherwise draw the chart the other flags ask for, and print why. - Customer view, --highlight 0, no --flat or --path-only. - --proof FILE With --regions, write the flat chart that the proof compares with. - --regions-all FILE With --regions, also write the regions with every step. - --title TEXT The chart title. - --out FILE The Mermaid chart. Default: stdout. - --story FILE The reader stories, one per bug. - --summary FILE The counts for the chart and the bugs, as JSON. - -h, --help Print this help, and exit 0. - -Each bug replays on the second model by its screens and step words, which are what the reader -sees. The replay takes the second model's steps that keep the screen and that the reader does not -take. A bug step that the reader does not see must keep the screen on the second model too, unless -the second model cannot take it. The replay stops at the first step that the second model shows -the reader differently. There the chart draws "with LABEL, STEP" (or "in LABEL, STEP" for ---against-dot) to where the second model goes, or a loop "... STEP cannot happen" when it cannot -take a step that the reader sees. - -The second model may have other variables and actions than the current model. Variables and -record fields that one start state has and the other does not are left out when the two models -are compared. One mapping draws both models, so it may read only what both have. The command -stops when the mapping reads a variable or field of the current model on a state of the second -model that lacks it. Test "name" in v first to label such a state another way. A step of an -action that the second model does not have is skipped when it changes no shared variable, and -matches any step to the same shared state when it does. - -In the customer view, states whose labels read "context: progress" with the same context sit in -one box named context, and each shows only its progress. A context with one state stays a plain -state. An arrow from a box's border stands for the same step from every state in the box: it is -drawn only when every state in the box takes that step, with the same labels, to the same state. -The layered view puts each detail state in a box named by its state label. - -With --regions, each "context: progress" label is one state in the context region and one in the -progress region. A step that happens beside only some states of the other region gets a guard: -"[when A or B]" or "[unless A or B]", whichever is shorter, and a "%% when [not] ids" line under it. -The mapping's guard words, when the step has them, replace "when A or B" in the label. -The proof combines the regions with every step as compare-charts.mjs does, and needs exactly the -flat chart's states, start states, and labeled steps. Each state's color must be the worse color -of its two region states. Then the drawn arrows, combined, must make only steps of the flat chart: -a step that they make wrongly gets back every arrow that the full regions take there. -compare-charts.mjs on the --regions-all and --proof files repeats the first proof. -mapping.regionNames = [context, progress] names the two regions. - -Outside regions, two states have at most one arrow. Its lines are the numbered steps, then the -current model's steps, then the second model's steps, with a blank line between each group. - -Without group(), the legend shows only the statuses the chart uses. A chart with a second model -has the line "%% second model (fix|against): LABEL" right after stateDiagram-v2. Each chart -starts with front matter that keeps Mermaid from shrinking it to the page or rewrapping its labels. - -Groups: group(vars, facts) returns the name of the group that a model state is in, such as -"Toast shown", or nothing. A chart state is in a group when all its model states are in it. A -region state is in a group when every state that it is part of is in it. Each group fills its -states with one color, and the legend lists only the groups that the chart draws: first the -groups in groupColors, in that order, then the others from A to Z. groupColors, such as -{ "Toast shown": "#d6eaf8" }, gives a group a light "#rgb" or "#rrggbb" color. A group with no -color, or null, takes the next free palette color: light blue, orange, purple, yellow, teal, tan, -then pink. A state in no group, or in more than one, is gray. summary.json counts the drawn -states of each group in chartGroups, and lists the chart states with more than one in mixedGroups. -TLC decides the status, and a group cannot change it. On top of the group color, a state that -breaks a rule has a thick red border, and a fixed state or a state only in the second model a -thick green border. The legend then shows that border. A thick blue border marks the bug path on -states that work. On other states of the path, the numbered arrows show it. A group cannot have -the name of a status, such as "Breaks a rule". - -Writes only the files that --out, --story, and --summary name. - -Exit codes - 0 It wrote the chart. - 1 It cannot use an input: a mapping without state(), a mapping that reads what the second - model lacks, TLC output that it refuses, or a --highlight with no such bug. The error says - why. - 2 A bad argument.`; -const more = "Run node tlc-to-chart.mjs --help for the flags."; - -const args = { - traces: [], - fixedTraces: [], - againstTraces: [], - fixedRules: [], - againstRules: [], - specs: [], - view: "customer", - highlight: 1, -}; -for (let i = 2; i < process.argv.length; i++) { - const a = process.argv[i]; - const next = () => process.argv[++i]; - if (a === "-h" || a === "--help") { - console.log(HELP); - process.exit(0); - } else if (a === "--dot") args.dot = next(); - else if (a === "--map") args.map = next(); - else if (a === "--traces") args.traces.push(next()); - else if (a === "--fixed-dot") args.fixedDot = next(); - else if (a === "--fixed-traces") args.fixedTraces.push(next()); - else if (a === "--fixed-rules") args.fixedRules.push(...next().split(",")); - else if (a === "--against-dot") args.againstDot = next(); - else if (a === "--against-traces") args.againstTraces.push(next()); - else if (a === "--against-rules") args.againstRules.push(...next().split(",")); - else if (a === "--listed") args.listed = next(); - else if (a === "--label") args.label = next(); - else if (a === "--spec") args.specs.push(next()); - else if (a === "--view") args.view = next(); - else if (a === "--highlight") args.highlight = Number(next()); - else if (a === "--path-only") args.pathOnly = true; - else if (a === "--full-labels") args.fullLabels = true; - else if (a === "--flat") args.flat = true; - else if (a === "--regions") args.regions = true; - else if (a === "--proof") args.proof = next(); - else if (a === "--regions-all") args.regionsAll = next(); - else if (a === "--cover") args.cover = true; - else if (a === "--title") args.title = next(); - else if (a === "--out") args.out = next(); - else if (a === "--story") args.story = next(); - else if (a === "--summary") args.summary = next(); - else { - console.error(`unknown argument ${a}. ${more}`); - process.exit(2); - } -} -const fail = (message) => { - console.error(`${message} ${more}`); - process.exit(2); -}; -if (!args.dot || !args.map) fail(USAGE); -if (!["customer", "developer", "layered"].includes(args.view)) - fail(`--view must be customer, developer, or layered, not ${args.view}.`); -if (args.fixedDot && args.againstDot) fail("use --fixed-dot or --against-dot, not both."); -if (args.fixedTraces.length && !args.fixedDot) fail("--fixed-traces needs --fixed-dot."); -if (args.againstTraces.length && !args.againstDot) fail("--against-traces needs --against-dot."); -if (!args.fixedTraces.length !== !args.fixedRules.length) - fail("--fixed-traces and --fixed-rules go together. The rules are what those TLC runs check."); -if (!args.againstTraces.length !== !args.againstRules.length) - fail( - "--against-traces and --against-rules go together. The rules are what those TLC runs check.", - ); -if (args.listed !== undefined) { - if (!args.againstDot) - fail("--listed says what the spec's configs expect; it needs --against-dot."); - const shape = `--listed must be JSON such as {"spec":"Spec.tla","breaks":{"A.cfg":"Rule"}}`; - try { - args.listed = JSON.parse(args.listed); - } catch (e) { - fail(`${shape}: ${e.message}.`); - } - const { spec, breaks } = isRecord(args.listed) ? args.listed : {}; - if ( - typeof spec !== "string" || - !isRecord(breaks) || - Object.values(breaks).some((r) => typeof r !== "string") - ) - fail(`${shape}.`); -} -if (args.label !== undefined && !args.fixedDot && !args.againstDot) - fail("--label names the second model; it needs --fixed-dot or --against-dot."); -if (args.proof && !args.regions) fail("--proof writes the regions proof; it needs --regions."); -if (args.regionsAll && !args.regions) - fail("--regions-all writes every regions arrow; it needs --regions."); -if ( - args.regions && - (args.view !== "customer" || args.flat || args.pathOnly || args.highlight !== 0) -) - fail( - "--regions draws the whole customer chart: use --highlight 0, without --flat or --path-only.", - ); +import { args } from "./tlc-to-chart-args.mjs"; +import { ruleBook } from "./rule-book.mjs"; +import { SEP, addGuard, edgeId, frontMatter, guarded } from "./chart-parts.mjs"; +import { + badStates, + byOrder, + diverges, + firstBad, + labelStatus, + readBroken, + worst, +} from "./bug-states.mjs"; +import { chartEdges } from "./chart-edges.mjs"; +import { regionsChart } from "./regions-chart.mjs"; +import { mapView } from "./map-view.mjs"; +import { stepsIntoBugs, waysLine } from "./ways-in.mjs"; +import { divergence, replaySecond } from "./second-replay.mjs"; +import { readCheck } from "./read-check.mjs"; const role = args.fixedDot ? "fix" : args.againstDot ? "against" : null; const L = args.label ?? (role === "fix" ? "the fix" : "the spec"); @@ -272,13 +50,14 @@ if (typeof mapping.state !== "function") throw new Error("the mapping module must export state(vars)"); if (args.view !== "customer" && typeof mapping.detail !== "function") throw new Error(`--view ${args.view} needs detail(vars) in the mapping module`); -const SEP = "\u0000"; -const edgeId = (from, to) => `${from}${SEP}${SEP}${to}`; const rules = new Map(); -for (const f of args.specs) - for (const [name, sentence] of ruleSentences(fs.readFileSync(f, "utf8"))) - if (!rules.has(name)) rules.set(name, sentence); +const ruleReads = new Map(); +for (const spec of args.specs) + for (const [name, { sentence, reads }] of ruleBook(spec)) { + if (sentence && !rules.has(name)) rules.set(name, sentence); + ruleReads.set(name, new Set([...(ruleReads.get(name) ?? []), ...reads])); + } const readerActions = new Set(mapping.readerActions ?? []); const factsFrom = (enabled) => ({ @@ -290,11 +69,16 @@ function keyOf(vars, facts) { const outer = mapping.state(vars, facts); if (typeof outer !== "string") throw new Error(`state() must return a string; got ${JSON.stringify(outer)}`); - return args.view === "layered" ? `${outer}${SEP}${mapping.detail(vars, facts)}` : outer; + if (args.view !== "layered") return outer; + return [outer, String(mapping.detail(vars, facts))] + .map((part) => part.trim()) + .filter(Boolean) + .join(SEP); } function stepLabel(action, before, after, view = args.view) { const said = typeof mapping.step === "function" ? mapping.step(action, before, after) : action; - const named = view === "customer" || said === action ? said : `${said} [${action}]`; + const named = + view === "customer" || said === action ? said : said ? `${said} [${action}]` : action; const seconds = typeof mapping.timer === "function" ? mapping.timer(action, before, after) : null; if (seconds === undefined || seconds === null || seconds === "" || !named) return named; if (typeof seconds !== "number" || !Number.isFinite(seconds) || seconds <= 0) @@ -309,64 +93,40 @@ function stepGuard(action, before, after) { .replace(/[[\]]/g, "") .trim(); } -function guarded(guards, label) { - const words = guards?.get(label); - return words && !words.has("") ? `${label} [${[...words].sort().join(" or ")}]` : label; -} -const addGuard = (guards, label, words) => { - if (!guards.has(label)) guards.set(label, new Set()); - guards.get(label).add(words); -}; -function lacking(a, b, prefix = []) { - return Object.keys(a).flatMap((k) => - !(k in b) - ? [[...prefix, k]] - : isRecord(a[k]) && isRecord(b[k]) - ? lacking(a[k], b[k], [...prefix, k]) - : [], +const definedLabel = (label, what) => { + if (typeof label !== "string" || !/\bundefined\b/.test(label)) return label; + const does = what === "group" ? `puts a state of ${L} in group` : `labels a ${what} of ${L}`; + throw new Error( + `the mapping ${does} "${label.replaceAll(SEP, " ")}", so it reads a field ${L} does not have there. Label states and steps only from what both models have.`, ); -} -const holder = (o, p) => p.slice(0, -1).reduce((x, k) => (isRecord(x?.[k]) ? x[k] : null), o); -function omit(vars, paths) { - if (!paths.length) return vars; - const out = structuredClone(vars); - for (const p of paths) { - const o = holder(out, p); - if (o) delete o[p.at(-1)]; - } - return out; -} -function guard(vars, paths, at = "v") { - if (!paths.length || !isRecord(vars)) return vars; - const missing = new Set(); - const deeper = new Map(); - for (const [k, ...rest] of paths) - if (!rest.length) missing.add(k); - else deeper.set(k, [...(deeper.get(k) ?? []), rest]); - return new Proxy(vars, { - get(target, k) { - if (missing.has(k) && !(k in target)) { - const name = `${at}.${k}`.slice(2); - throw new Error( - `${L} has no ${name}, and the mapping reads ${at}.${k} on its states. Label states and steps only from what both models have, or test "${k}" in ${at} first.`, - ); - } - return deeper.has(k) ? guard(target[k], deeper.get(k), `${at}.${k}`) : target[k]; - }, - }); -} +}; -function modelOf(graph) { +function modelOf(graph, guarded = (vars) => vars, checked = (label) => label) { const byId = new Map([...graph.nodes.keys()].map((id) => [id, new Set()])); for (const e of [...graph.edges, ...graph.loops]) byId.get(e.from).add(e.action); - const index = new Map([...graph.nodes.values()].map((n) => [stateKey(n.vars), byId.get(n.id)])); + const stateKeys = new Map([...graph.nodes.values()].map((n) => [n.id, stateKey(n.vars)])); + const index = new Map([...stateKeys].map(([id, k]) => [k, byId.get(id)])); const facts = (vars) => factsFrom(index.get(stateKey(vars)) ?? new Set()); + const forMap = (vars) => guarded(mapView(vars)); return { graph, + stateKeys, facts, - key: (vars) => keyOf(vars, facts(vars)), - customer: (vars) => String(mapping.state(vars, facts(vars))), + key: (vars) => checked(keyOf(forMap(vars), facts(vars)), "state"), + customer: (vars) => checked(String(mapping.state(forMap(vars), facts(vars))), "state"), + step: (action, before, after, view) => + checked(stepLabel(action, forMap(before), forMap(after), view), "step"), + guard: (action, before, after) => + checked(stepGuard(action, forMap(before), forMap(after)), "step"), + cause: (vars, invariant) => { + const why = + typeof mapping.cause === "function" + ? mapping.cause(forMap(vars), facts(vars), invariant) + : null; + return why === undefined || why === null ? null : checked(String(why), "bug"); + }, + group: (vars) => checked(mapping.group(forMap(vars), facts(vars)), "group"), }; } @@ -379,17 +139,17 @@ function quotient(m) { [...graph.nodes.values()].filter((n) => n.init).map((n) => keys.get(n.id)), ); const edges = new Map(); - for (const e of graph.edges) { + for (const e of [...graph.edges, ...graph.loops]) { const from = keys.get(e.from); const to = keys.get(e.to); - if (from === to) continue; + if (from === to && !args.loops) continue; const id = edgeId(from, to); if (!edges.has(id)) edges.set(id, { from, to, counts: new Map(), guards: new Map() }); const [before, after] = [graph.nodes.get(e.from).vars, graph.nodes.get(e.to).vars]; - const label = stepLabel(e.action, before, after); + const label = m.step(e.action, before, after); if (!label) continue; edges.get(id).counts.set(label, (edges.get(id).counts.get(label) ?? 0) + 1); - addGuard(edges.get(id).guards, label, stepGuard(e.action, before, after)); + addGuard(edges.get(id).guards, label, m.guard(e.action, before, after)); } for (const e of edges.values()) e.labels = new Set([...e.counts].sort((a, b) => b[1] - a[1]).map(([l]) => l)); @@ -402,192 +162,73 @@ function quotient(m) { const graph = parseDot(fs.readFileSync(args.dot, "utf8")); const main = modelOf(graph); const now = quotient(main); -const mainInit = [...graph.nodes.values()].find((n) => n.init).vars; const secondDot = args.fixedDot ?? args.againstDot; const secondGraph = secondDot ? parseDot(fs.readFileSync(secondDot, "utf8")) : null; -const secondInit = secondGraph && [...secondGraph.nodes.values()].find((n) => n.init).vars; -const lacks = secondGraph ? lacking(mainInit, secondInit) : []; -const extra = secondGraph ? lacking(secondInit, mainInit) : []; -const second = - secondGraph && - modelOf({ - ...secondGraph, - nodes: new Map( - [...secondGraph.nodes].map(([id, n]) => [ - id, - { ...n, raw: n.vars, vars: guard(n.vars, lacks) }, - ]), - ), - }); +const rawTraces = readTraces(args.traces); +const secondTraceFiles = role === "fix" ? args.fixedTraces : args.againstTraces; +const secondRules = new Set(role === "fix" ? args.fixedRules : args.againstRules); +const rawSecondTraces = readTraces(secondTraceFiles); +const traces = firstBad(rawTraces); +const secondTraces = firstBad(rawSecondTraces); +const { lacks, extra, guard } = secondGraph + ? readCheck({ graph, rawTraces, secondGraph, rawSecondTraces, L }) + : {}; +const second = secondGraph && modelOf(secondGraph, guard, definedLabel); const secondQ = second && quotient(second); -function readerPath(trace, key, labelOf = stepLabel) { +const hiddenStep = (rule, sameLabel, action, before, after) => + sameLabel && + !readerActions.has(action) && + ![...(ruleReads.get(rule) ?? [])].some( + (v) => v in before && stateKey(before[v]) !== stateKey(after[v]), + ); + +function readerPath(trace, m, key, view) { + const last = trace.steps.length - 1; const out = [{ key: key(trace.steps[0].vars), label: null, modelStep: 0 }]; for (let i = 1; i < trace.steps.length; i++) { const { action } = trace.steps[i]; const k = key(trace.steps[i].vars); - if (k === out[out.length - 1].key && !readerActions.has(action)) continue; const [before, after] = [trace.steps[i - 1].vars, trace.steps[i].vars]; - const label = labelOf(action, before, after) || action; - const guard = stepGuard(action, before, after); + const sameLabel = k === out[out.length - 1].key; + if (i < last && hiddenStep(trace.invariant, sameLabel, action, before, after)) continue; + const label = m.step(action, before, after, view) || action; + const guard = m.guard(action, before, after); out.push({ key: k, label, guard, action, modelStep: i }); } return out; } -const readerLabel = (action, before, after) => stepLabel(action, before, after, "customer"); -function replaySecond(trace) { - if (!second) return null; - const shared = (vars) => stateKey(omit(vars, lacks)); - const sharedOf = (n) => stateKey(omit(n.raw, extra)); - const out = new Map(); - for (const e of [...second.graph.edges, ...second.graph.loops]) { - if (!out.has(e.from)) out.set(e.from, []); - out.get(e.from).push(e); - } - const actions = new Set([...second.graph.edges, ...second.graph.loops].map((e) => e.action)); - const nodeOf = (id) => second.graph.nodes.get(id); - const screen = new Map( - [...second.graph.nodes.values()].map((n) => [n.id, second.customer(n.vars)]), - ); - const words = (e) => readerLabel(e.action, nodeOf(e.from).vars, nodeOf(e.to).vars) || e.action; - const settle = (ids) => { - const all = new Set(ids); - for (const id of all) - for (const e of out.get(id) ?? []) - if (!readerActions.has(e.action) && screen.get(e.to) === screen.get(id)) all.add(e.to); - return all; - }; - const seen = readerPath(trace, main.customer, readerLabel); - const shownAt = new Map(seen.map((p, j) => [p.modelStep, j])); - let here = settle( - [...second.graph.nodes.values()] - .filter((n) => n.init && screen.get(n.id) === seen[0].key) - .map((n) => n.id), - ); - if (here.size === 0) return { reason: "starts on another screen" }; - const passed = new Set(here); - let j = 0; - for (let i = 1; i < trace.steps.length; i++) { - const { action } = trace.steps[i]; - const hidden = !shownAt.has(i); - if (!hidden) j = shownAt.get(i); - const want = seen[j]; - const [before, after] = [trace.steps[i - 1].vars, trace.steps[i].vars]; - const known = actions.has(action); - if (!known && (hidden || shared(before) === shared(after))) continue; - const tries = [...here].flatMap((id) => - (out.get(id) ?? []).filter((e) => - known ? e.action === action : sharedOf(nodeOf(e.to)) === shared(after), - ), - ); - if (hidden && tries.length === 0) continue; - const same = known - ? tries.filter((e) => screen.get(e.to) === want.key && (hidden || words(e) === want.label)) - : tries; - if (same.length) { - here = settle(same.map((e) => e.to)); - for (const id of here) passed.add(id); - continue; - } - const outcomes = new Map(); - for (const e of tries) { - const id = `${screen.get(e.to)}${SEP}${words(e)}`; - outcomes.set(id, [...(outcomes.get(id) ?? []), e]); - } - const taken = [...outcomes.values()].sort((a, b) => b.length - a.length)[0]?.[0]; - const from = main.key(before); - const said = hidden ? readerLabel(action, before, after) || action : want.label; - const at = { step: j, said, from, hidden }; - if (!taken) - return { - ...at, - key: from, - label: stepLabel(action, before, after) || action, - disabled: true, - }; - const [b, a] = [nodeOf(taken.from).vars, nodeOf(taken.to).vars]; - return { ...at, key: second.key(a), label: stepLabel(taken.action, b, a) || taken.action }; - } - const bad = new Set( - rawSecondTraces - .filter((t) => t.invariant === trace.invariant) - .map((t) => stateKey(t.steps[t.steps.length - 1].vars)), - ); - const checked = secondRules.has(trace.invariant); - const ruleHolds = checked && ![...passed].some((id) => bad.has(stateKey(nodeOf(id).vars))); - return { sameEnd: true, checked, ruleHolds }; -} +const mainActions = new Set([...graph.edges, ...graph.loops].map((e) => e.action)); -function readTraces(files, paths = []) { +function readTraces(files) { return files.flatMap((f) => { const text = fs.readFileSync(f, "utf8"); const warning = tlcOutputWarning(text, f); if (warning) console.error(`${f}: ${warning}`); - return parseTraces(text).map((t) => ({ - ...t, - steps: t.steps.map((s) => ({ ...s, vars: guard(s.vars, paths) })), - })); + return parseTraces(text); }); } -function firstBad(raw) { - const badStates = new Map(); - for (const t of raw) { - if (!badStates.has(t.invariant)) badStates.set(t.invariant, new Set()); - badStates.get(t.invariant).add(stateKey(t.steps[t.steps.length - 1].vars)); - } - const cuts = []; - const seen = new Set(); - for (const t of raw) { - const first = t.steps.findIndex((s) => badStates.get(t.invariant).has(stateKey(s.vars))); - const cut = { invariant: t.invariant, steps: t.steps.slice(0, first + 1) }; - const id = `${t.invariant}${SEP}${stateKey(cut.steps[cut.steps.length - 1].vars)}`; - if (seen.has(id)) continue; - seen.add(id); - cuts.push(cut); - } - return cuts; -} -const rawTraces = readTraces(args.traces); -const secondTraceFiles = role === "fix" ? args.fixedTraces : args.againstTraces; -const secondRules = new Set(role === "fix" ? args.fixedRules : args.againstRules); -const rawSecondTraces = readTraces(secondTraceFiles, lacks); -const traces = firstBad(rawTraces); -const secondTraces = firstBad(rawSecondTraces); function group(cuts, m) { const byId = new Map(); for (const trace of cuts) { - const rp = readerPath(trace, m.customer); + const rp = readerPath(trace, m, m.customer); const last = trace.steps[trace.steps.length - 1].vars; - const said = - typeof mapping.cause === "function" - ? mapping.cause(last, m.facts(last), trace.invariant) - : null; - const cause = said === undefined || said === null ? null : String(said); + const cause = m.cause(last, trace.invariant); const id = `${trace.invariant}${SEP}${cause ?? rp[rp.length - 1].key}`; const prev = byId.get(id); + const ends = new Set([...(prev?.ends ?? []), stateKey(last)]); if ( !prev || rp.length < prev.steps || (rp.length === prev.steps && trace.steps.length < prev.trace.steps.length) ) { - byId.set(id, { - id, - cause, - steps: rp.length, - trace, - invariant: trace.invariant, - count: (prev?.count ?? 0) + 1, - }); - } else prev.count++; + byId.set(id, { id, cause, steps: rp.length, trace, invariant: trace.invariant, ends }); + } else prev.ends = ends; } return byId; } -const byOrder = (a, b) => - a.steps - b.steps || - a.trace.steps.length - b.trace.steps.length || - (a.id < b.id ? -1 : a.id > b.id ? 1 : 0); const inSecond = (l) => `${within} ${L}, ${l}`; const modelEdges = new Map(now.edges); if (secondQ) @@ -599,70 +240,74 @@ if (secondQ) guards: new Map([...e.guards].map(([l, words]) => [inSecond(l), words])), secondOnly: true, }); -function divergence(s) { - if (s?.key === undefined) return s; - const plain = edgeId(s.from, s.key); - const text = inSecond(`${s.label}${s.disabled ? " cannot happen" : ""}`); - const drawn = !s.disabled && [s.label, text].some((l) => modelEdges.get(plain)?.labels.has(l)); - return { ...s, text, edge: drawn ? plain : `${plain}${SEP}second` }; -} const withPath = (c, m) => { - const path = readerPath(c.trace, m.key); + const path = readerPath(c.trace, m, m.key); return { ...c, path, end: path[path.length - 1].key }; }; const classesById = group(traces, main); -const mainClasses = [...classesById.values()] - .sort(byOrder) - .map((c) => ({ ...withPath(c, main), second: divergence(replaySecond(c.trace)) })); +const mainClasses = [...classesById.values()].sort(byOrder).map((c, i) => { + const bug = withPath(c, main); + return { + ...bug, + second: divergence( + replaySecond(c.trace, { + main, + lacks, + extra, + second, + hiddenStep, + readerPath, + mainActions, + secondRules, + rawSecondTraces, + }), + bug.path, + i + 1, + { inSecond, modelEdges }, + ), + }; +}); const introduced = second ? [...group(secondTraces, second).values()] .filter((c) => !classesById.has(c.id)) .sort(byOrder) .map((c) => ({ ...withPath(c, second), second: null, introduced: true })) : []; -const classes = [...mainClasses, ...introduced]; +const classes = [ + ...stepsIntoBugs(mainClasses, main, rawTraces, args), + ...stepsIntoBugs(introduced, second, rawSecondTraces, args), +]; -const diverges = (c) => c.second?.key !== undefined; -function badKeys(m, keys, raw) { - const g = m.graph; - const bad = new Set(raw.map((t) => stateKey(t.steps[t.steps.length - 1].vars))); - const out = new Set(raw.map((t) => m.key(t.steps[t.steps.length - 1].vars))); - const ok = (id) => !bad.has(stateKey(g.nodes.get(id).vars)); - const next = new Map(); - for (const e of g.edges) { - if (!next.has(e.from)) next.set(e.from, []); - next.get(e.from).push(e.to); - } - const clean = new Set([...g.nodes.values()].filter((n) => n.init && ok(n.id)).map((n) => n.id)); - const queue = [...clean]; - while (queue.length) { - for (const to of next.get(queue.shift()) ?? []) { - if (clean.has(to) || !ok(to)) continue; - clean.add(to); - queue.push(to); - } - } - const reached = new Set([...clean].map((id) => keys.get(id))); - for (const k of keys.values()) if (!reached.has(k)) out.add(k); - return out; -} -const badNow = badKeys(main, now.keys, rawTraces); -const uncheckedTraces = rawTraces.filter((t) => !secondRules.has(t.invariant)); -const badSecond = second - ? new Set([ - ...badKeys(second, secondQ.keys, rawSecondTraces), - ...(role === "fix" ? badKeys(main, now.keys, uncheckedTraces) : []), - ]) - : new Set(); +const brokenNow = readBroken(args.broken); +const statusNow = labelStatus(main, now.keys, badStates(rawTraces, brokenNow)); +const statusSecond = second + ? labelStatus( + second, + secondQ.keys, + badStates(rawSecondTraces, readBroken(args.fixedBroken ?? args.againstBroken)), + ) + : () => undefined; +const statusUnchecked = + role === "fix" + ? labelStatus( + main, + now.keys, + badStates(rawTraces, brokenNow, (r) => !secondRules.has(r)), + ) + : () => undefined; const allNodes = new Set([...now.nodes, ...(secondQ?.nodes ?? [])]); const initial = new Set([...now.initial, ...(secondQ?.initial ?? [])]); -const onlySecond = (k) => second && !now.nodes.has(k); const status = (k) => { - if (role === "against") - return badNow.has(k) || badSecond.has(k) ? "wrong" : onlySecond(k) ? "fixed" : "works"; - if (role === "fix") - return badSecond.has(k) ? "wrong" : badNow.has(k) || onlySecond(k) ? "fixed" : "works"; - return badNow.has(k) ? "wrong" : "works"; + if (role === "against") { + const s = worst(statusNow(k), statusSecond(k)); + return s === "works" && !now.nodes.has(k) ? "fixed" : s; + } + if (role === "fix") { + const s = worst(statusSecond(k), statusUnchecked(k)); + if (s === "wrong" && !["wrong", "mixed"].includes(statusNow(k))) return "worse"; + return s === "works" && statusNow(k) !== "works" ? "fixed" : s; + } + return statusNow(k); }; const chosen = args.highlight > 0 ? classes[args.highlight - 1] : null; @@ -671,75 +316,23 @@ if (args.highlight > 0 && classes.length > 0 && !chosen) const onPath = new Set(chosen ? chosen.path.map((p) => p.key) : []); if (chosen && diverges(chosen)) onPath.add(chosen.second.key); -const edgesAlong = (c) => - c.path.slice(1).map((p, i) => ({ - id: edgeId(c.path[i].key, p.key), - from: c.path[i].key, - to: p.key, - label: p.label, - guard: p.guard, - })); -function chartEdges() { - const edges = new Map(modelEdges); - for (const { second: s } of classes.filter(diverges)) { - if (modelEdges.has(s.edge)) continue; - if (!edges.has(s.edge)) - edges.set(s.edge, { from: s.from, to: s.key, labels: new Set(), secondOnly: true }); - edges.get(s.edge).labels.add(s.text); - } - const pathNumbers = new Map(); - (chosen ? edgesAlong(chosen) : []).forEach((step, i) => { - if (!pathNumbers.has(step.id)) pathNumbers.set(step.id, []); - const shown = step.guard ? `${step.label} [${step.guard}]` : step.label; - pathNumbers.get(step.id).push({ n: i + 1, label: step.label, shown }); - if (!edges.has(step.id)) - edges.set(step.id, { from: step.from, to: step.to, labels: new Set([step.label]) }); - }); - if (!args.cover) return { edges, pathNumbers, coverEdges: null }; - - const coverEdges = new Set(); - const bfs = (q, targets) => { - const parent = new Map([...q.initial].map((k) => [k, null])); - const queue = [...q.initial]; - const outEdges = new Map(); - for (const [id, e] of q.edges) { - if (!outEdges.has(e.from)) outEdges.set(e.from, []); - outEdges.get(e.from).push([id, e]); - } - while (queue.length) { - const k = queue.shift(); - for (const [id, e] of outEdges.get(k) ?? []) { - if (parent.has(e.to)) continue; - parent.set(e.to, id); - queue.push(e.to); - } - } - for (const t of targets) { - for (let id = parent.get(t); id; id = parent.get(q.edges.get(id).from)) coverEdges.add(id); - } - }; - bfs(now, now.settled); - if (secondQ) - bfs( - secondQ, - [...secondQ.settled].filter((k) => !now.nodes.has(k)), - ); - const pathLabels = new Map(); - for (const c of classes) { - for (const step of edgesAlong(c)) { - coverEdges.add(step.id); - pathLabels.set(step.id, (pathLabels.get(step.id) ?? new Set()).add(step.label)); - } - if (diverges(c)) coverEdges.add(c.second.edge); - } - for (const id of coverEdges) { - if (edges.has(id)) continue; - const [from, to] = id.split(`${SEP}${SEP}`); - edges.set(id, { from, to, labels: pathLabels.get(id) }); - } - return { edges, pathNumbers, coverEdges }; -} -const { edges, pathNumbers, coverEdges } = chartEdges(); +const coverOff = + args.cover && + ![graph, secondGraph] + .filter(Boolean) + .some((g) => [...g.edges, ...g.loops].some((e) => readerActions.has(e.action))) + ? "no step in TLC's graph is a reader action" + : null; +if (coverOff && !args.pathOnly) console.error(`--cover: drew every state, because ${coverOff}`); +const { edges, pathNumbers, coverEdges } = chartEdges({ + args, + now, + secondQ, + modelEdges, + classes, + chosen, + coverOff, +}); const LEGEND = { works: ["legendWorks", mapping.legend?.works ?? (role === "fix" ? "Works today" : "Works")], @@ -751,6 +344,17 @@ const LEGEND = { "legendFixed", mapping.legend?.fixed ?? (role === "fix" ? `Fixed or new with ${L}` : `Only in ${L}`), ], + mixed: [ + "legendMixed", + mapping.legend?.mixed ?? `Breaks a rule in some cases${role === "fix" ? ` with ${L}` : ""}`, + ], + after: [ + "legendAfter", + mapping.legend?.after ?? `Reached only after a bug${role === "fix" ? ` with ${L}` : ""}`, + ], + ...(role === "fix" + ? { worse: ["legendWorse", mapping.legend?.worse ?? `Breaks a rule only with ${L}`] } + : {}), }; const grouping = typeof mapping.group === "function"; @@ -760,7 +364,7 @@ const groupNames = []; const fills = new Map(); if (grouping) { const add = (k, vars, m) => { - const g = mapping.group(vars, m.facts(vars)); + const g = m.group(vars); if (g !== undefined && g !== null && typeof g !== "string") throw new Error(`group() must return a group name or nothing; got ${JSON.stringify(g)}`); if (!groupsOf.has(k)) groupsOf.set(k, new Set()); @@ -811,7 +415,14 @@ if (grouping) { } const mixedGroups = [...groupsOf].filter(([, gs]) => gs.size > 1).map(([k]) => k); -const BORDER = { works: "", wrong: "Wrong", fixed: "Fixed" }; +const BORDER = { + works: "", + wrong: "Wrong", + fixed: "Fixed", + mixed: "Mixed", + after: "After", + worse: "Worse", +}; const classFor = (g, s, path) => grouping ? `${g ? `group${groupNames.indexOf(g)}` : "plain"}${BORDER[s] || (path ? "Path" : "")}` @@ -826,15 +437,6 @@ const groupCounts = (drawn) => })) .filter((g) => g.states > 0); -const frontMatter = (title) => [ - "---", - ...(title ? [`title: ${JSON.stringify(title)}`] : []), - "config:", - " markdownAutoWrap: false", - " state:", - " useMaxWidth: false", - "---", -]; function head(drawn) { const out = [...frontMatter(args.title), "stateDiagram-v2"]; if (role) out.push(` %% second model (${role}): ${L}`); @@ -853,7 +455,7 @@ function head(drawn) { const styleClasses = new Set([...drawn.map((d) => d.cls), ...legend.map(([, , c]) => c)]); const bases = [["plain", PLAIN_FILL], ...groupNames.map((g, i) => [`group${i}`, fills.get(g)])]; for (const [base, fill] of bases) - for (const border of ["", "Wrong", "Fixed", "Path"]) + for (const border of [...Object.values(BORDER), "Path"]) if (styleClasses.has(base + border)) out.push(` ${groupClassDef(base, fill, border)}`); } else for (const c of STATUS_CLASSES) out.push(` ${c}`); if (legend.length) { @@ -865,293 +467,22 @@ function head(drawn) { return out; } -function regionsChart({ states, starts, edges, classes, proofPath, allPath }) { - const norm = (s) => s.replaceAll(";", ",").replace(/\s+/g, " ").trim(); - const split = (k) => { - const at = k.indexOf(": "); - return at > 0 && at + 2 < k.length ? [norm(k.slice(0, at)), norm(k.slice(at + 2))] : null; - }; - const nodes = [...states].sort(); - if (!nodes.every(split)) return null; - const steps = []; - for (const e of edges.values()) { - if (e.labels.size === 0) return { why: "a step has no label" }; - for (const l of e.labels) - steps.push({ - from: split(e.from), - to: split(e.to), - label: norm(l), - words: e.guards?.get(l) ?? new Set([""]), - }); - } - const bar = (p) => p.join(" | "); - const flat = { - Main: { - states: new Set(nodes.map((k) => bar(split(k)))), - initial: new Set(starts.map((k) => bar(split(k)))), - transitions: steps.map((s) => ({ from: bar(s.from), to: bar(s.to), label: s.label })), - }, - }; - const ids = new Map([...flat.Main.states].map((s, i) => [s, `p${i}`])); - const proof = [ - ...frontMatter(), - "stateDiagram-v2", - ...[...ids].map(([s, id]) => ` state "${escapeMermaidQuotes(s)}" as ${id}`), - ...[...flat.Main.initial].map((s) => ` [*] --> ${ids.get(s)}`), - ...steps.map((s) => ` ${ids.get(bar(s.from))} --> ${ids.get(bar(s.to))} : ${s.label}`), - ]; - if (proofPath) fs.writeFileSync(proofPath, proof.join("\n") + "\n"); - - const beside = [new Map(), new Map()]; - for (const k of nodes) { - const p = split(k); - for (const i of [0, 1]) { - if (!beside[i].has(p[i])) beside[i].set(p[i], new Set()); - beside[i].get(p[i]).add(p[1 - i]); - } - } - const size = beside[0].size + beside[1].size; - if (size >= nodes.length) - return { why: `${size} region states are not fewer than the ${nodes.length} flat states` }; - - const rank = ["works", "fixed", "wrong"]; - const regionStatus = [new Map(), new Map()]; - for (const k of nodes) { - const p = split(k); - for (const i of [0, 1]) { - const prev = regionStatus[i].get(p[i]); - if (!prev || rank.indexOf(status(k)) < rank.indexOf(prev)) - regionStatus[i].set(p[i], status(k)); - } - } - for (const k of nodes) { - const p = split(k); - const shown = rank[Math.max(...[0, 1].map((i) => rank.indexOf(regionStatus[i].get(p[i]))))]; - if (shown !== status(k)) - return { - why: `"${k}" is "${LEGEND[status(k)][1]}", and its two region states would show "${LEGEND[shown][1]}"`, - }; - } - - const moves = [new Map(), new Map()]; - for (const s of steps) - for (const i of [0, 1]) { - const id = [s.from[i], s.label, s.to[i]].join(SEP); - if (!moves[i].has(id)) - moves[i].set(id, { - from: s.from[i], - label: s.label, - to: s.to[i], - beside: new Set(), - guards: new Map(), - }); - moves[i].get(id).beside.add(s.from[1 - i]); - for (const w of s.words) addGuard(moves[i].get(id).guards, s.label, norm(w)); - } - for (const m of moves) - for (const [id, x] of m) - if ( - x.from === x.to && - ![...m.values()].some((y) => y.from === x.from && y.label === x.label && y.to !== x.from) - ) - m.delete(id); - const rid = beside.map((m, i) => new Map([...m.keys()].sort().map((s, j) => [s, `r${i}s${j}`]))); - const regionGroup = [new Map(), new Map()]; - for (const k of nodes) { - const p = split(k); - for (const i of [0, 1]) { - const prev = regionGroup[i].get(p[i]); - regionGroup[i].set(p[i], prev === undefined || prev === groupOf(k) ? groupOf(k) : ""); - } - } - const drawn = rid.map((m, i) => - [...m.keys()].map((s) => drawnState(regionGroup[i].get(s), regionStatus[i].get(s))), - ); - const draw = (only) => { - const out = head(drawn.flat()); - out.push(` state "At the same time" as regions {`); - let count = 0; - for (const i of [0, 1]) { - if (i) out.push(" --"); - const name = mapping.regionNames?.[i]; - const pad = name ? " " : " "; - if (name) out.push(` state "${escapeMermaidQuotes(wrap(name))}" as region${i} {`); - for (const s of new Set(starts.map((k) => split(k)[i]))) - out.push(`${pad}[*] --> ${rid[i].get(s)}`); - for (const [s, id] of rid[i]) - out.push(`${pad}state "${escapeMermaidQuotes(wrap(s))}" as ${id}`); - const shownMoves = [...moves[i]] - .filter(([id]) => !only || only[i].has(id)) - .map(([, m]) => m) - .sort((a, b) => a.from.localeCompare(b.from) || a.label.localeCompare(b.label)); - for (const m of shownMoves) { - const all = beside[i].get(m.from); - const not = [...all].filter((s) => !m.beside.has(s)).sort(); - const yes = [...m.beside].sort(); - const unless = not.length > 0 && not.length < yes.length; - const guard = not.length === 0 ? null : unless ? not : yes; - const worded = guarded(m.guards, m.label); - const text = - worded !== m.label || !guard - ? worded - : `${m.label} [${unless ? "unless" : "when"} ${guard.join(" or ")}]`; - out.push(`${pad}${rid[i].get(m.from)} --> ${rid[i].get(m.to)} : ${wrap(text, 30)}`); - if (guard) - out.push( - `${pad}%% when ${unless ? "not " : ""}${guard.map((s) => rid[1 - i].get(s)).join(", ")}`, - ); - count++; - } - if (name) out.push(" }"); - } - out.push(" }"); - for (const i of [0, 1]) - [...rid[i].values()].forEach((id, j) => out.push(` class ${id} ${drawn[i][j].cls}`)); - return { mermaid: out.join("\n") + "\n", count }; - }; - - const all = draw(null); - const differences = compareCharts(parseMermaid(all.mermaid).regions, flat); - if (differences.length) { - const n = differences.reduce((sum, [, xs]) => sum + xs.length, 0); - return { - why: `the regions do not combine into the flat chart, with ${n} difference(s), such as ${differences[0][0]}: ${differences[0][1][0]}`, - }; - } - if (allPath) fs.writeFileSync(allPath, all.mermaid); - - const parts = new Map(nodes.map((k) => [bar(split(k)), split(k)])); - const key = (from, label, to) => `${from} --> ${to} : ${label}`; - const real = new Set(steps.map((s) => key(bar(s.from), s.label, bar(s.to)))); - const next = new Map(); - for (const s of steps) { - if (!next.has(bar(s.from))) next.set(bar(s.from), []); - next.get(bar(s.from)).push(s); - } - const keep = [new Set(), new Set()]; - const movesOf = (s) => - [0, 1] - .filter((i) => s.from[i] !== s.to[i]) - .map((i) => [i, [s.from[i], s.label, s.to[i]].join(SEP)]); - const price = (s) => movesOf(s).filter(([i, id]) => !keep[i].has(id)).length * nodes.length + 1; - const wanted = []; - const want = (s) => { - wanted.push(s); - for (const [i, id] of movesOf(s)) keep[i].add(id); - }; - const stepsOf = (e, said) => { - const labels = [...e.labels].map(norm); - return (labels.includes(said) ? [said] : labels).map((label) => ({ - from: split(e.from), - to: split(e.to), - label, - })); - }; - for (const c of classes) { - for (const step of edgesAlong(c)) { - const e = edges.get(step.id); - if (e) stepsOf(e, norm(step.label)).forEach(want); - } - if (diverges(c)) stepsOf(edges.get(c.second.edge), "").forEach(want); - } - const cheapest = () => { - const cost = new Map(starts.map((k) => [bar(split(k)), 0])); - const parent = new Map(); - const open = new Set(cost.keys()); - while (open.size) { - const k = [...open].reduce((a, b) => (cost.get(b) < cost.get(a) ? b : a)); - open.delete(k); - for (const s of next.get(k) ?? []) { - const c = cost.get(k) + price(s); - if (c < (cost.get(bar(s.to)) ?? Infinity)) { - cost.set(bar(s.to), c); - parent.set(bar(s.to), s); - open.add(bar(s.to)); - } - } - } - return { cost, parent }; - }; - const targets = [ - ...[0, 1].flatMap((i) => - [...beside[i].keys()].sort().map((r) => ({ at: (k) => parts.get(k)[i] === r })), - ), - ...[...new Set(steps.map((s) => s.label))] - .filter((l) => !wanted.some((s) => s.label === l)) - .sort() - .map((l) => ({ ends: (s) => s.label === l })), - ]; - while (targets.length) { - const { cost, parent } = cheapest(); - let best = null; - targets.forEach((t, n) => { - const ways = t.at - ? [...cost.keys()].filter(t.at).map((k) => ({ k, c: cost.get(k), last: null })) - : steps - .filter(t.ends) - .filter((s) => cost.has(bar(s.from))) - .map((s) => ({ k: bar(s.from), c: cost.get(bar(s.from)) + price(s), last: s })); - for (const w of ways) if (!best || w.c < best.c) best = { ...w, n }; - }); - if (!best) break; - targets.splice(best.n, 1); - const path = best.last ? [best.last] : []; - for (let s = parent.get(best.k); s; s = parent.get(bar(s.from))) path.push(s); - path.forEach(want); - } - const takes = (x, label) => - [0, 1].flatMap((i) => - [...moves[i]] - .filter(([, m]) => m.from === x[i] && m.label === label && m.beside.has(x[1 - i])) - .map(([id]) => [i, id]), - ); - let scoped; - let made; - for (;;) { - scoped = draw(keep); - made = productOf(parseMermaid(scoped.mermaid).regions); - const madeKeys = new Set(made.transitions.map((t) => key(t.from, t.label, t.to))); - const redo = [ - ...made.transitions - .filter((t) => parts.has(t.from) && !real.has(key(t.from, t.label, t.to))) - .map((t) => [parts.get(t.from), t.label]), - ...wanted - .filter((s) => made.states.has(bar(s.from))) - .filter((s) => !madeKeys.has(key(bar(s.from), s.label, bar(s.to)))) - .map((s) => [s.from, s.label]), - ]; - const before = keep[0].size + keep[1].size; - for (const [x, label] of redo) for (const [i, id] of takes(x, label)) keep[i].add(id); - if (keep[0].size + keep[1].size === before) break; - } - const sound = - [...made.states].every((s) => parts.has(s)) && - made.transitions.every((t) => real.has(key(t.from, t.label, t.to))); - if (!sound) scoped = all; - - return { - mermaid: scoped.mermaid, - states: [beside[0].size, beside[1].size], - transitions: scoped.count, - all: all.count, - flat: { states: nodes.length, transitions: steps.length }, - status: Object.fromEntries( - rank.map((s) => [ - s, - regionStatus.reduce((n, m) => n + [...m.values()].filter((x) => x === s).length, 0), - ]), - ), - groups: groupCounts(drawn.flat()), - }; -} const regions = args.regions ? regionsChart({ states: allNodes, starts: [...initial], edges, + loops: Boolean(args.loops), classes, proofPath: args.proof, allPath: args.regionsAll, + mapping, + status, + LEGEND, + groupOf, + drawnState, + groupCounts, + head, }) : null; if (regions?.why) console.error(`regions: not drawn, because ${regions.why}`); @@ -1174,8 +505,9 @@ const inBox = (k, box) => { if (!boxes.has(box)) boxes.set(box, []); boxes.get(box).push(k); }; -if (args.view === "layered") for (const k of sorted) inBox(k, k.split(SEP)[0]); -else if (args.view === "customer" && !args.flat) { +if (args.view === "layered" && !args.flat) { + for (const k of sorted) if (k.includes(SEP)) inBox(k, k.slice(0, k.indexOf(SEP))); +} else if (args.view === "customer" && !args.flat) { for (const k of sorted) { const at = k.indexOf(": "); if (at > 0 && at + 2 < k.length) inBox(k, k.slice(0, at)); @@ -1187,8 +519,8 @@ for (const [box, members] of boxes) for (const k of members) boxOf.set(k, box); const declared = [...[...boxes.values()].flat(), ...sorted.filter((k) => !boxOf.has(k))]; const stateIds = new Map(declared.map((k, i) => [k, `s${i}`])); const idOf = (k) => stateIds.get(k); -const caption = (k) => - args.view === "layered" ? k.split(SEP)[1] : boxOf.has(k) ? k.slice(boxOf.get(k).length + 2) : k; +const show = (k) => (args.view === "layered" ? k.replace(SEP, ": ") : k); +const caption = (k) => (boxOf.has(k) ? show(k).slice(boxOf.get(k).length + 2) : show(k)); function numbered(steps) { const runs = []; @@ -1205,7 +537,7 @@ for (const [id, e] of edges) { if (!flatStates.has(e.from) || !flatStates.has(e.to)) continue; if (args.pathOnly && chosen && !pathNumbers.has(id) && id !== chosen.second?.edge) continue; if (coverEdges && !coverEdges.has(id)) continue; - const pair = edgeId(e.from, e.to); + const pair = e.apart ? id : edgeId(e.from, e.to); if (!arrows.has(pair)) arrows.set(pair, { from: e.from, to: e.to, groups: [[], [], []], plain: true, labels: [] }); const a = arrows.get(pair); @@ -1241,7 +573,9 @@ const transitions = [...arrows.values()].map(({ from, to, groups, plain, labels })); const fromBox = []; const folded = new Set(); +const plainNames = new Set(sorted.filter((k) => !boxOf.has(k)).map(caption)); for (const [box, members] of boxes) { + if (plainNames.has(box)) continue; const mine = new Set(members); const byStep = new Map(); for (const l of transitions) { @@ -1277,7 +611,6 @@ const mermaid = regions?.mermaid ?? out.join("\n") + "\n"; if (args.out) fs.writeFileSync(args.out, mermaid); else process.stdout.write(mermaid); -const show = (k) => (args.view === "layered" ? k.replace(SEP, ": ") : k); const ruleOf = (c) => rules.get(c.invariant) ?? `\`${c.invariant}\``; function secondLine(c) { const W = cap(within); @@ -1286,21 +619,23 @@ function secondLine(c) { if (!s) return null; if (s.sameEnd) return !s.checked - ? `${W} ${L} you see the same screens and steps, but ${L} does not check this rule.` + ? `${W} ${L} you see the same states and steps, but ${L} does not check this rule.` : s.ruleHolds - ? `${W} ${L} you see the same screens and steps, yet the rule holds, so the state labels do not show what ${L} changes.` - : `${W} ${L} you see the same screens and steps, so ${L} does not change this path.`; + ? `${W} ${L} you see the same states and steps, yet the rule holds, so the state labels do not show what ${L} changes.` + : `${W} ${L} you see the same states and steps, so ${L} does not change this path.`; if (s.reason) return `${cap(L)} cannot take these steps: it ${s.reason}.`; const outcome = s.disabled ? "cannot happen, so you stay at" : s.key === s.from ? "keeps you at" : "takes you to"; + const as = s.own && s.own !== s.said ? ` as "${s.own}"` : ""; if (s.hidden) { const when = s.step ? `after step ${s.step}` : "before your first step"; - return `${W} ${L}, ${when}, "${s.said}" happens and ${outcome}: ${show(s.key)}.`; + return `${W} ${L}, ${when}, "${s.said}" happens${as} and ${outcome}: ${show(s.key)}.`; } - return `${W} ${L}, step ${s.step} (${s.said}) ${outcome}: ${show(s.key)}.`; + const happens = as ? `happens${as} and ` : ""; + return `${W} ${L}, step ${s.step} (${s.said}) ${happens}${outcome}: ${show(s.key)}.`; } const listedIn = (c) => Object.keys(args.listed.breaks).filter((cfg) => args.listed.breaks[cfg] === c.invariant); @@ -1320,17 +655,24 @@ if (args.story) { lines.push(`Rule broken: ${ruleOf(c)}`); if (c.cause) lines.push(`Why: ${c.cause.replace(/\.$/, "")}.`); if (args.listed) lines.push(listedLine(c)); - lines.push( - c.count === 1 - ? "This is the only way to get here in the model." - : `There are ${c.count} ways to get here in the model. This is the shortest.`, - "", - ); - c.path.forEach((p, j) => - lines.push( - j === 0 ? `0. Start: ${show(p.key)}` : `${j}. ${cap(p.label)}. Now: ${show(p.key)}`, - ), - ); + const ways = waysLine(c); + lines.push(...(ways ? [ways] : []), ""); + lines.push(`0. Start: ${show(c.path[0].key)}`); + const m = c.introduced ? second : main; + c.path.slice(1).forEach((p, j) => { + const from = c.path[j].modelStep + 1; + const hidden = c.trace.steps + .slice(from, p.modelStep) + .map( + (s, h) => + m.step(s.action, c.trace.steps[from + h - 1].vars, s.vars, "customer") || s.action, + ); + if (hidden.length) + lines.push( + ` (${hidden.length} hidden step${hidden.length === 1 ? "" : "s"}: ${[...new Set(hidden)].join(", ")})`, + ); + lines.push(`${j + 1}. ${cap(p.label)}. Now: ${show(p.key)}`); + }); lines.push(""); const said = secondLine(c); if (said) lines.push(said, ""); @@ -1352,12 +694,37 @@ const secondOf = (c) => : c.second ? c.second.sameEnd ? !c.second.checked - ? "same screens, rule not checked" + ? "same states, rule not checked" : c.second.ruleHolds - ? "same screens, but the rule holds" + ? "same states, but the rule holds" : "unchanged" : (c.second.reason ?? show(c.second.key)) : null; +function coverSummary() { + if (coverOff || regions?.mermaid) + return { drawn: false, why: coverOff ?? "the overview has parallel regions" }; + const pairs = new Set(); + const missed = new Map(); + for (const [id, e] of edges) { + const pair = e.apart ? id : edgeId(e.from, e.to); + pairs.add(pair); + if (coverEdges.has(id)) continue; + if (!missed.has(pair)) missed.set(pair, { from: e.from, to: e.to, labels: [] }); + missed.get(pair).labels.push(...[...e.labels].map((l) => guarded(e.guards, l))); + } + return { + drawn: true, + states: allNodes.size, + transitions: pairs.size, + leftOutStates: [...allNodes] + .filter((k) => !flatStates.has(k)) + .map(show) + .sort(), + leftOutTransitions: [...missed.values()] + .map((m) => `${show(m.from)} --> ${show(m.to)}: ${joinLabels(m.labels)}`) + .sort(), + }; +} const summary = { view: args.view, ...(role @@ -1377,11 +744,16 @@ const summary = { chartStatus: regions?.mermaid ? regions.status : Object.fromEntries( - ["works", "wrong", "fixed"].map((s) => [ + Object.keys(LEGEND).map((s) => [ s, - [...flatStates].filter((k) => status(k) === s).length, + [...(args.pathOnly && chosen ? onPath : allNodes)].filter((k) => status(k) === s).length, ]), ), + mixedStatus: [...allNodes] + .filter((k) => status(k) === "mixed") + .map(show) + .sort(), + ...(args.cover ? { cover: coverSummary() } : {}), ...(grouping ? { chartGroups: regions?.mermaid ? regions.groups : groupCounts(drawn), @@ -1409,7 +781,9 @@ const summary = { cause: c.cause, readerSteps: c.path.length - 1, modelSteps: c.trace.steps.length, - counterexamples: c.count, + counterexamples: c.ends.size, + lastSteps: c.lastSteps, + startsHere: c.startsHere, [role ?? "fix"]: secondOf(c), ...(args.listed ? { listed: listedIn(c) } : {}), })), diff --git a/harness/pstack/claude-code/skills/statechart/scripts/tlc-tools.mjs b/harness/pstack/claude-code/skills/statechart/scripts/tlc-tools.mjs new file mode 100644 index 000000000..09d46de4a --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/tlc-tools.mjs @@ -0,0 +1,69 @@ +import { spawnSync } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import process from "node:process"; + +// TLC's v1.8.0 is a rolling pre-release whose jar is re-uploaded, so pin the newest immutable +// release. The scripts read 1.7.4's output, which has no trace spec. +export const TLC_TOOL = + "github:tlaplus/tlaplus[asset_pattern=tla2tools.jar,checksum=sha256:936a262061c914694dfd669a543be24573c45d5aa0ff20a8b96b23d01e050e88]@1.7.4"; +export const JAVA_TOOL = "java@temurin-25"; + +const SET_TLC_VARS = + "set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later)."; +const indent = (text) => text.replace(/^/gm, " "); + +export function findTlc() { + const { TLA2TOOLS, JAVA } = process.env; + for (const [name, file] of Object.entries({ JAVA, TLA2TOOLS })) + if (file && !fs.existsSync(file)) throw new Error(`${name} is ${file}, which does not exist.`); + if (TLA2TOOLS) return { jar: TLA2TOOLS, java: JAVA || "java" }; + const command = (tools) => + `cd "$(mktemp -d)" && MISE_CEILING_PATHS="$PWD" mise install ${tools.map((t) => `'${t}'`).join(" ")}`; + const empty = fs.mkdtempSync(path.join(os.tmpdir(), "statechart-mise-")); + const env = { ...process.env, MISE_CEILING_PATHS: empty }; + const run = (...args) => spawnSync("mise", args, { cwd: empty, env, encoding: "utf8" }); + const mise = (...args) => { + const r = run(...args); + if (r.error) + throw new Error( + `Cannot find TLC. It installs with mise, which cannot start: ${r.error.message}. Install mise (https://mise.jdx.dev), or ${SET_TLC_VARS}`, + ); + if (r.status === 0) return r.stdout.trim(); + const said = (r.stderr ?? "").trim(); + if (!said || /not installed|mise bin/.test(said)) return ""; + throw new Error( + `Cannot find TLC, because mise failed:\n${indent(said)}\nThe install command: ${command(JAVA ? [TLC_TOOL] : [JAVA_TOOL, TLC_TOOL])}\nFix that, or ${SET_TLC_VARS}`, + ); + }; + const resolve = () => { + const home = mise("where", TLC_TOOL); + const jar = home && path.join(home, "tla2tools.jar"); + const java = JAVA || mise("which", "java", "--tool", JAVA_TOOL); + return { jar: jar && fs.existsSync(jar) ? jar : "", java }; + }; + try { + const found = resolve(); + if (found.jar && found.java) return found; + const tools = [!found.java && JAVA_TOOL, !found.jar && TLC_TOOL].filter(Boolean); + const names = [!found.jar && "TLC", !found.java && "Java"].filter(Boolean); + console.error(`Installing ${names.join(" and ")} with mise. This happens once.`); + const r = run("install", "--quiet", ...tools); + const installed = resolve(); + if (installed.jar && installed.java) return installed; + const said = + `${r.stderr ?? ""}${r.stdout ?? ""}`.trim() || + `mise install exited ${r.status}, with no output`; + throw new Error( + `Cannot find TLC, and mise could not install it:\n${indent(said)}\nThe install command: ${command(tools)}\nOr ${SET_TLC_VARS}`, + ); + } finally { + fs.rmSync(empty, { recursive: true, force: true }); + } +} + +export function cannotRunTlc({ java, jar }, r) { + const said = `java exited ${r.status ?? r.signal}:\n${indent(`${r.stderr}${r.stdout}`.trim())}`; + return `cannot run TLC with ${java} -cp ${jar}: ${r.error?.message ?? said}\nFix that, or ${SET_TLC_VARS}`; +} diff --git a/harness/pstack/claude-code/skills/statechart/scripts/ways-in.mjs b/harness/pstack/claude-code/skills/statechart/scripts/ways-in.mjs new file mode 100644 index 000000000..e94658c80 --- /dev/null +++ b/harness/pstack/claude-code/skills/statechart/scripts/ways-in.mjs @@ -0,0 +1,69 @@ +import { joinAnd } from "./lib.mjs"; +import { badOf, reachableAvoiding } from "./bug-states.mjs"; + +export function stepsIntoBugs(cs, m, raw, { story, summary, next }) { + if (!cs.length || (!story && !summary)) return cs; + const { graph: g, stateKeys } = m; + const unnamed = new Set(next); + const rules = new Set(cs.map((c) => c.invariant)); + const clean = new Map([...rules].map((r) => [r, reachableAvoiding(m, badOf(raw, r))])); + const byEnd = new Map(); + for (const c of cs) for (const k of c.ends) byEnd.set(k, [...(byEnd.get(k) ?? []), c]); + const ways = new Map(cs.map((c) => [c, { steps: new Map(), n: 0, nameless: 0, action: null }])); + for (const e of g.edges) { + const into = byEnd.get(stateKeys.get(e.to))?.filter((c) => clean.get(c.invariant).has(e.from)); + if (!into?.length) continue; + const [before, after] = [g.nodes.get(e.from).vars, g.nodes.get(e.to).vars]; + const label = m.step(e.action, before, after) || e.action; + const moves = m.key(before) !== m.key(after); + for (const c of into) { + const w = ways.get(c); + const s = w.steps.get(label) ?? { n: 0, moves: false }; + w.steps.set(label, { n: s.n + 1, moves: s.moves || moves }); + w.n++; + if (!unnamed.has(e.action)) continue; + w.nameless++; + w.action = e.action; + } + } + const starts = [...g.nodes.values()].filter((n) => n.init).map((n) => stateKeys.get(n.id)); + return cs.map((c) => { + const { steps, n, nameless, action } = ways.get(c); + const lastSteps = [...steps] + .sort((a, b) => b[1].n - a[1].n || (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0)) + .map(([l]) => l); + const onPath = new Set(c.path.slice(1).map((p) => p.label)); + const startsIn = starts.filter((k) => c.ends.has(k)).length; + return { + ...c, + lastSteps, + keptSteps: new Set(lastSteps.filter((l) => !steps.get(l).moves && !onPath.has(l))), + nameless: nameless ? { action, all: nameless === n } : null, + waysIn: n + startsIn, + startsHere: startsIn > 0, + }; + }); +} + +export function waysLine(c) { + const quoted = c.lastSteps.map( + (l) => `"${l}"${c.keptSteps.has(l) ? " (the state stays the same)" : ""}`, + ); + const named = quoted.slice(0, 5); + const more = quoted.length - named.length; + const list = joinAnd(more ? [...named, `${more} more`] : named); + const steps = c.nameless + ? `In the model, steps labeled ${list} lead into this bug. TLC names ${c.nameless.all ? "them" : "some of them"} ${c.nameless.action}, the whole next-state action, so the story cannot count them.` + : c.lastSteps.length === 1 + ? `In the model, one step leads into this bug: ${list}.` + : c.lastSteps.length + ? `In the model, ${c.lastSteps.length} different steps lead into this bug: ${list}.` + : ""; + const start = !c.startsHere + ? "" + : steps + ? "The model can also start in this bug." + : "The model starts in this bug."; + const shortest = c.waysIn > 1 ? "The steps below are the shortest path that TLC found." : ""; + return [steps, start, shortest].filter(Boolean).join(" "); +} diff --git a/pstack/skills/statechart/SKILL.md b/pstack/skills/statechart/SKILL.md index 1818f2a73..66670ebd1 100644 --- a/pstack/skills/statechart/SKILL.md +++ b/pstack/skills/statechart/SKILL.md @@ -1,36 +1,36 @@ --- name: statechart -description: "Draw what an app's user sees as a Mermaid statechart from a TLA+ model. States are gray if they work, red if still wrong, and green if fixed. Each bug TLC finds is a numbered path. Use for /statechart, \"show the PM the bug\", or to turn an agreed chart into a TLA+ skeleton." +description: "Draw how any system moves between states, such as a product flow, a protocol, or a service, as a Mermaid statechart from a TLA+ model. Each bug is a numbered path with blue borders to a state that breaks a rule. Use for /statechart, to show a bug as a path through states, or to turn an agreed chart into a TLA+ skeleton." disable-model-invocation: true --- # Statechart -The chart shows what an app's user sees. A PM reads it to decide whether that behavior is right. Label each state with what is on the screen, such as "Paying". Label each step with what the user does or sees happen, such as "tap Pay" or "the charge goes through". Keep variable and action names out of labels. The scripts call this user the reader, as in `readerActions` and reader steps. +The chart shows how a system moves between states, in the words of the person who reads it. The scripts call that person the reader, as in `readerActions` and reader steps. The reader decides whether the behavior is right. The reader can be a PM who reads a product flow, an engineer who reads a protocol or a service, or a reviewer who reads a design. Label each state with what the reader can observe or cares about, such as "Paying" in an app or "Lease held, write pending" in a lock service. Label each step with the event that moves the system, such as "tap Pay" in an app, "the request reads the count" in a service, or "the lease expires" in a lock service. Keep variable and action names out of labels. -Run `node /scripts/statechart.mjs --help` for the commands and for how to read a chart. Work in a scratch folder outside the repo. The command writes its files next to its input. +Run `node /scripts/statechart.mjs --help` for the commands and for how to read a chart. Work in a scratch folder outside the repo. The command writes its files next to its input. The first run installs TLC and Java with mise, which downloads about 305 MB and needs the network. The help's TLC section tells how to use a TLC and Java that you have. ## Pick the direction -Use `from-spec` when a TLA+ model exists or the code is concrete enough to model. Use `from-chart` when there is no model and the PM should agree on the behavior before anyone writes or changes the code. +Use `from-spec` when a TLA+ model exists, or when the code or design is concrete enough to model. Use `from-chart` when there is no model, and the people who own the behavior should agree on it before anyone builds or changes it. -If the code already has a TLA+ spec, start from a copy of that spec. Add the variables the user sees, and keep the spec's existing invariants. To audit the code against that spec, make the copy model what the code does, and pass the spec unchanged as `--against`. Check it with the config that its authors say it passes, and pass that config as `--against-config` unless it has the spec's file name. One mapping labels both models, so label states and steps only from variables that both have. In a code model or a new spec, tie every variable and action to code that a reviewer can open, and take its name from that code. Make each action one indivisible step in the code. Put the file and line of the code that checks each condition of an action in a comment on that condition, as in `\* pay.ts:120`. +If the code already has a TLA+ spec, start from a copy of that spec. Add the variables that the labels and rules need, and keep the spec's existing invariants. To audit the code against that spec, make the copy model what the code does, and pass the spec unchanged as `--against`. Check it with the config that its authors say it passes, and pass that config as `--against-config` unless it has the spec's file name. One mapping labels both models, so label states and steps from variables that both have. A label that reads a variable that only the copy has must test `"name" in v` first, and label the spec's states another way. When a model describes code, tie every variable and action to code that a reviewer can open, and take its name from that code. Make each action one indivisible step in the code. Put the file and line of the code that checks each condition of an action in a comment on that condition, as in `\* pay.ts:120`. ## Spec to chart -1. Choose the chart's states. When the PM cares about something the screen does not show, such as a second charge while the screen still says "Paying", name the state "Paying, charged twice". Start with few states, and split a state only when the split separates two bugs. -2. Write each rule the PM agrees to as one invariant, with the PM's sentence as its comment. Each rule names what the user sees on a screen or gets from a request, and the code that can change it. Refuse a rule that lacks one of them. For a rule that applies once everything settles, define `Settled == ~ENABLED System`, where `System` is the disjunction of the actions the user does not take. Report each broken rule as a bug. Never weaken a rule to make the chart pass. -3. Run `from-spec`, then fill in the mapping file it writes. Put the labels in `state` and `step`, and list the user's actions in `readerActions` by action name. When two bugs end on the same screen through different code paths, add `cause` and make it name the code path. To color the states by what the PM cares about, such as "Toast shown", return that group from `group`. The legend names the groups, and states that break a rule still get a red border. -4. If there is an open fix, model it from the code and pass that model as `--fixed`. +1. Choose the chart's states. When the reader cares about something that a label would hide, put it in the label, such as "Paying, charged twice" when the screen still says "Paying" after a second charge. Start with few states. Split a state when the split separates two bugs, or when the reader must see a step that leaves the label the same, such as a request that reads a count, because the chart does not draw such a step. +2. Write each rule that the reader agrees to as one invariant, with the reader's sentence as its comment. A rule can be about any state in the model: what a screen shows, what a request returns, a stored value, or a message between parts. When the model describes code, the rule also names the code that can change that state. If the model has no variable for what a rule is about, add the variable. Do not drop the rule. For a rule that applies once everything settles, define `Settled == ~ENABLED System`, where `System` is the disjunction of the actions that happen without the reader. For a rule that applies at a narrower point, such as after each request finishes, write a predicate for that point and make the rule hold where it is true. For a goal that the system must be able to reach, write a rule that says the goal never happens, and pass that rule with `--expect-broken`. The path that breaks it is then a way to the goal. The chart still draws the goal as a state that breaks a rule, and `stories.md` tells it as a bug, so name each goal rule in the hand-off. Report each other broken rule as a bug. Never weaken a rule to make the chart pass. +3. Run `from-spec`, then fill in the mapping file it writes. Put the labels in `state` and `step`. When the chart must separate what the reader does from what happens around the reader, list the reader's actions in `readerActions` by action name. Leave the list empty when every action counts the same. When the overview is too large to read, add `--cover`. The overview then keeps only the shortest path to each settled state, plus every bug path, and the command lists what it leaves out. When two bugs end in the same state for different reasons, add `cause` and make it name each reason. To color the states by what the reader cares about, such as "Order shipped", return that group from `group`. The legend names the groups, and in the PNG a state that breaks a rule is red in place of its group color. +4. If there is a proposed fix, model it and pass that model as `--fixed`. 5. Render each chart with `--png` and look at every PNG. Check that each numbered path reads in order. If a label reads like code, fix `state` or `step` and run the command again. -6. Replay each bug's story against the real code, as a test or a live repro, one step per model action. Drive only entry points that a user or client reaches, and assert only what they see. A test that asserts a flag that it sets, or that searches the source text, proves nothing. When a run proves part of the story and reading the code proves the rest, say which part each proves. When the test copies code that it cannot import, give the commit and line range of the copy, and list what the copy leaves out. If a bug does not reproduce, assume the model is wrong until you show otherwise. Fix the model, not the rule. +6. When the model describes code, replay each bug's story against the real code, as a test or a live repro, one step per model action. Drive only entry points that a user, a client, or another part of the system reaches, and assert only what they can observe. A test that asserts a flag that it sets, or that searches the source text, proves nothing. When a run proves part of the story and reading the code proves the rest, say which part each proves. When the test copies code that it cannot import, give the commit and line range of the copy, and list what the copy leaves out. If a bug does not reproduce, assume the model is wrong until you show otherwise. Fix the model, not the rule. For a design with no code yet, the TLC trace is the evidence, and the hand-off must say so. ## Chart to spec -1. Draw the chart the PM agrees to. Use one region for each thing that changes independently, such as the connection and the sync badge. Add one `%% never` line per rule, with the PM's sentence. Draw "the app closes" as an edge to the state the user sees next. -2. Run `from-chart`. If a rule breaks, the agreed chart has a design bug. Take the bug to the PM. -3. Add the real code's guards and variables to the spec that `from-chart` wrote, and run `from-chart` again. For each difference it prints, ask the PM whether the code is wrong or the chart needs a correction. +1. Draw the chart that the reader agrees to. Use one region for each part that changes independently, such as a device's connection and the job that the device runs. Add one `%% never` line per rule, with the reader's sentence. Draw an event that stops or restarts a part, such as "the app closes" or "the process restarts", as an edge to the state that comes next. Run `render CHART.mmd` to get a PNG of the chart for the reader. +2. Run `from-chart`. If a rule breaks, the agreed chart has a design bug. Take the bug to the people who agreed to the chart. +3. Add the real code's guards and variables to the spec that `from-chart` wrote, and run `from-chart` again. For each difference it prints, ask the reader whether the code is wrong or the chart needs a correction. ## Hand off -Give the PM the PNG and the Mermaid source of each chart. Add one line per bug with the rule it breaks, its steps as the chart numbers them, whether it reproduced, and whether the open fix fixes it. With `--against`, also say whether the spec already lists it: in the configs that `stories.md` names, in the spec's notes on what the model leaves out, or not at all. Do not report that a rule holds while the command lists conditions with no file and line. Take every count you quote from `summary.json`. Attach `stories.md` and the command that regenerates everything. Say what the model leaves out. +Give the PNG and the Mermaid source of each chart to the person who must decide. Add one line per bug with the rule it breaks, its steps as the chart numbers them, whether it reproduced, and whether the proposed fix fixes it. A broken PROPERTY has no path on the chart, so give the TLC counterexample that `stories.md` names for it. With `--against`, also say whether the spec already lists it: in the configs that `stories.md` names, in the spec's notes on what the model leaves out, or not at all. When the model describes code, do not report that a rule holds while the command lists conditions with no file and line. Take every count you quote from `summary.json`. Attach `stories.md` and the command that regenerates everything. Say what the model leaves out. diff --git a/pstack/skills/statechart/scripts/bug-states.mjs b/pstack/skills/statechart/scripts/bug-states.mjs new file mode 100644 index 000000000..3c8e9f21f --- /dev/null +++ b/pstack/skills/statechart/scripts/bug-states.mjs @@ -0,0 +1,81 @@ +import fs from "node:fs"; +import { BROKEN_MARK, parseValue, stateKey } from "./lib.mjs"; +import { SEP } from "./chart-parts.mjs"; + +export function firstBad(raw) { + const badStates = new Map(); + for (const t of raw) { + if (!badStates.has(t.invariant)) badStates.set(t.invariant, new Set()); + badStates.get(t.invariant).add(stateKey(t.steps[t.steps.length - 1].vars)); + } + const cuts = []; + const seen = new Set(); + for (const t of raw) { + const first = t.steps.findIndex((s) => badStates.get(t.invariant).has(stateKey(s.vars))); + const cut = { invariant: t.invariant, steps: t.steps.slice(0, first + 1) }; + const id = `${t.invariant}${SEP}${stateKey(cut.steps[cut.steps.length - 1].vars)}`; + if (seen.has(id)) continue; + seen.add(id); + cuts.push(cut); + } + return cuts; +} + +export const byOrder = (a, b) => + a.steps - b.steps || + a.trace.steps.length - b.trace.steps.length || + (a.id < b.id ? -1 : a.id > b.id ? 1 : 0); + +const lastKey = (t) => stateKey(t.steps[t.steps.length - 1].vars); +export const badOf = (raw, rule) => new Set(raw.filter((t) => t.invariant === rule).map(lastKey)); +export function reachableAvoiding({ graph: g, stateKeys }, bad) { + const ok = (id) => !bad.has(stateKeys.get(id)); + const next = new Map(); + for (const e of g.edges) { + if (!next.has(e.from)) next.set(e.from, []); + next.get(e.from).push(e.to); + } + const clean = new Set([...g.nodes.values()].filter((n) => n.init && ok(n.id)).map((n) => n.id)); + for (const id of clean) for (const to of next.get(id) ?? []) if (ok(to)) clean.add(to); + return clean; +} + +export const diverges = (c) => c.second?.key !== undefined; +export function readBroken(file) { + if (!file) return []; + const lines = fs.readFileSync(file, "utf8").split("\n"); + const marked = new RegExp(`^<<\\s*"${BROKEN_MARK}",`); + const out = []; + for (let i = 0; i < lines.length; i++) { + if (!marked.test(lines[i])) continue; + let text = lines[i]; + while (typeof parseValue(text) === "string" && i + 1 < lines.length) text += `\n${lines[++i]}`; + const [, vars, broken] = parseValue(text); + out.push({ key: stateKey(vars), rules: broken }); + } + return out; +} +export const badStates = (raw, broken, counts = () => true) => + new Set([ + ...raw.filter((t) => counts(t.invariant)).map(lastKey), + ...broken.filter((b) => b.rules.some(counts)).map((b) => b.key), + ]); +export function labelStatus(m, keys, bad) { + const good = reachableAvoiding(m, bad); + const kinds = new Map(); + for (const n of m.graph.nodes.values()) { + const k = keys.get(n.id); + if (!kinds.has(k)) kinds.set(k, new Set()); + kinds.get(k).add(bad.has(m.stateKeys.get(n.id)) ? "bad" : good.has(n.id) ? "good" : "after"); + } + return (k) => { + const has = kinds.get(k); + if (!has) return undefined; + if (has.has("bad")) return has.has("good") ? "mixed" : "wrong"; + return has.has("good") ? "works" : "after"; + }; +} + +export const RANK = ["works", "fixed", "after", "mixed", "wrong", "worse"]; +export const worst = (...all) => + all.filter(Boolean).reduce((a, b) => (RANK.indexOf(b) > RANK.indexOf(a) ? b : a), "works"); diff --git a/pstack/skills/statechart/scripts/chart-edges.mjs b/pstack/skills/statechart/scripts/chart-edges.mjs new file mode 100644 index 000000000..0f0db11dc --- /dev/null +++ b/pstack/skills/statechart/scripts/chart-edges.mjs @@ -0,0 +1,68 @@ +import { edgeId } from "./chart-parts.mjs"; +import { diverges } from "./bug-states.mjs"; + +export const edgesAlong = (c) => + c.path.slice(1).map((p, i) => ({ + id: edgeId(c.path[i].key, p.key), + from: c.path[i].key, + to: p.key, + label: p.label, + guard: p.guard, + })); + +export function chartEdges({ args, now, secondQ, modelEdges, classes, chosen, coverOff }) { + const edges = new Map(modelEdges); + for (const { second: s } of classes.filter(diverges)) { + if (modelEdges.has(s.edge)) continue; + if (!edges.has(s.edge)) + edges.set(s.edge, { + from: s.from, + to: s.key, + labels: new Set(), + secondOnly: true, + apart: s.apart, + }); + edges.get(s.edge).labels.add(s.overview); + } + const pathNumbers = new Map(); + (chosen ? edgesAlong(chosen) : []).forEach((step, i) => { + if (!pathNumbers.has(step.id)) pathNumbers.set(step.id, []); + const shown = step.guard ? `${step.label} [${step.guard}]` : step.label; + pathNumbers.get(step.id).push({ n: i + 1, label: step.label, shown }); + if (!edges.has(step.id)) + edges.set(step.id, { from: step.from, to: step.to, labels: new Set([step.label]) }); + }); + const bugSteps = classes.flatMap(edgesAlong); + const pathEdges = new Map(); + for (const step of bugSteps) { + if (edges.has(step.id)) continue; + if (!pathEdges.has(step.id)) + pathEdges.set(step.id, { from: step.from, to: step.to, labels: new Set() }); + pathEdges.get(step.id).labels.add(step.label); + } + for (const [id, e] of pathEdges) edges.set(id, e); + if (!args.cover || coverOff) return { edges, pathNumbers, coverEdges: null }; + + const coverEdges = new Set(bugSteps.map((step) => step.id)); + const bfs = (q, targets) => { + const parent = new Map([...q.initial].map((k) => [k, null])); + const outEdges = new Map(); + for (const [id, e] of q.edges) { + if (!outEdges.has(e.from)) outEdges.set(e.from, []); + outEdges.get(e.from).push([id, e]); + } + for (const k of parent.keys()) + for (const [id, e] of outEdges.get(k) ?? []) if (!parent.has(e.to)) parent.set(e.to, id); + for (const t of targets) { + for (let id = parent.get(t); id; id = parent.get(q.edges.get(id).from)) coverEdges.add(id); + } + }; + bfs(now, now.settled); + if (secondQ) + bfs( + secondQ, + [...secondQ.settled].filter((k) => !now.nodes.has(k)), + ); + for (const c of classes.filter(diverges)) coverEdges.add(c.second.edge); + return { edges, pathNumbers, coverEdges }; +} diff --git a/pstack/skills/statechart/scripts/chart-parts.mjs b/pstack/skills/statechart/scripts/chart-parts.mjs new file mode 100644 index 000000000..eaa391d4e --- /dev/null +++ b/pstack/skills/statechart/scripts/chart-parts.mjs @@ -0,0 +1,21 @@ +export const SEP = "\u0000"; +export const edgeId = (from, to) => `${from}${SEP}${SEP}${to}`; + +export function guarded(guards, label) { + const words = guards?.get(label); + return words && !words.has("") ? `${label} [${[...words].sort().join(" or ")}]` : label; +} +export const addGuard = (guards, label, words) => { + if (!guards.has(label)) guards.set(label, new Set()); + guards.get(label).add(words); +}; + +export const frontMatter = (title) => [ + "---", + ...(title ? [`title: ${JSON.stringify(title)}`] : []), + "config:", + " markdownAutoWrap: false", + " state:", + " useMaxWidth: false", + "---", +]; diff --git a/pstack/skills/statechart/scripts/chart-to-tla.mjs b/pstack/skills/statechart/scripts/chart-to-tla.mjs index 7dfa337d2..911c58698 100644 --- a/pstack/skills/statechart/scripts/chart-to-tla.mjs +++ b/pstack/skills/statechart/scripts/chart-to-tla.mjs @@ -5,49 +5,20 @@ import process from "node:process"; import { parseMermaid } from "./lib.mjs"; const USAGE = "Usage: node chart-to-tla.mjs CHART.mmd --module NAME [--dir OUT]"; -const HELP = `${USAGE} - -Writes a TLA+ spec, a TLC config, and a reader mapping from a Mermaid stateDiagram-v2. -statechart.mjs from-chart runs it, then checks each INVARIANT in NAME.cfg alone. - -Each region becomes one variable whose values are the region's states. Each transition label -becomes one action. Regions that share a label move together: every region that can take it does, -and the others stay. A "%% when" line after a transition is its guard: the region can take that -transition only while the other regions are in, or with "not" are not in, the states it names. An -unlabeled transition is its own action. Each -"%% never NAME: A & B -- sentence" line becomes the invariant NAME == ~(A /\\ B). A name in a -never line is a state, or a composite that stands for every state inside it. - - --module NAME The TLA+ module name. Required. - --dir OUT The folder for the files. Default: the chart's folder. - -h, --help Print this help, and exit 0. - -Writes NAME.tla, NAME.cfg, and NAME.map.mjs into OUT, and replaces files with those names. -NAME.cfg has SPECIFICATION Spec, INVARIANT TypeOK, and one INVARIANT per never line. -NAME.map.mjs gives tlc-to-chart.mjs the reader's words: the region states joined by " | ". - -Exit codes - 0 It wrote the files. - 1 It cannot read the chart. The error says why. - 2 A bad argument.`; -const more = "Run node chart-to-tla.mjs --help for the flags."; const args = {}; for (let i = 2; i < process.argv.length; i++) { const a = process.argv[i]; - if (a === "-h" || a === "--help") { - console.log(HELP); - process.exit(0); - } else if (a === "--module") args.module = process.argv[++i]; + if (a === "--module") args.module = process.argv[++i]; else if (a === "--dir") args.dir = process.argv[++i]; else if (!a.startsWith("-") && !args.chart) args.chart = a; else { - console.error(`unknown argument ${a}. ${more}`); + console.error(`unknown argument ${a}.`); process.exit(2); } } if (!args.chart || !/^[A-Za-z][A-Za-z0-9_]*$/.test(args.module ?? "")) { - console.error(`${USAGE}\n${more}`); + console.error(USAGE); process.exit(2); } const dir = args.dir ?? path.dirname(args.chart); @@ -167,9 +138,7 @@ for (const a of actions) { if (still.length) out.push(` /\\ UNCHANGED <<${still.join(", ")}>>`); } out.push(""); -out.push( - actions.length ? `Next == ${actions.map((a) => a.name).join(" \\/ ")}` : "Next == UNCHANGED vars", -); +out.push(actions.length ? `Next == ${actions.map((a) => a.name).join(" \\/ ")}` : "Next == FALSE"); out.push(""); out.push("Spec == Init /\\ [][Next]_vars"); diff --git a/pstack/skills/statechart/scripts/compare-charts.mjs b/pstack/skills/statechart/scripts/compare-charts.mjs index 587125a9f..730698be4 100644 --- a/pstack/skills/statechart/scripts/compare-charts.mjs +++ b/pstack/skills/statechart/scripts/compare-charts.mjs @@ -1,51 +1,19 @@ #!/usr/bin/env node import fs from "node:fs"; import process from "node:process"; -import { FULL_LABELS, LEGEND_ID, compareCharts, parseMermaid } from "./lib.mjs"; +import { FULL_LABELS, LEGEND_ID, NO_LOOPS, compareCharts, parseMermaid } from "./lib.mjs"; const USAGE = "Usage: node compare-charts.mjs EXPECTED.mmd ACTUAL.mmd"; -const HELP = `${USAGE} - -Compares two Mermaid stateDiagram-v2 charts: reachable states, start states, and labeled -transitions. statechart.mjs from-chart runs it to check that the spec still draws the chart that -the reader agreed to. - -EXPECTED is the agreed chart. ACTUAL is what tlc-to-chart.mjs drew from the spec in the customer -view with --highlight 0 --full-labels --flat, without --cover, --fixed-dot, or --against-dot. Each -chart compares as the -machine that its steps reach from its start states. A chart with parallel regions compares as its -combined machine, so a guard that blocks a step in only some combinations of regions shows as a -difference. A "%% when ID, ID" or "%% when not ID, ID" line right after a transition is its guard: -the step happens only while another region is in, or not in, one of those states. The label ends -with the guard's words in [ ], which are not part of the label. A combined state is the region -states joined by " | " in region order, which is what chart-to-tla.mjs's mapping writes. A step that does not change the state is not compared, because -tlc-to-chart.mjs does not draw it. An edge label "a or b" counts as two labels. An unlabeled edge -counts only when no labeled edge joins the same two states, because tlc-to-chart.mjs draws only -the labels. - - -h, --help Print this help, and exit 0. - -Prints each difference and then "N difference(s)", or prints "same". Writes no file. - -Exit codes - 0 The charts are the same. - 1 The charts differ. - 2 A bad argument, a chart that it cannot read, or an ACTUAL chart that is not drawn for - comparison.`; -const more = "Run node compare-charts.mjs --help for the flags."; const files = []; for (const a of process.argv.slice(2)) { - if (a === "-h" || a === "--help") { - console.log(HELP); - process.exit(0); - } else if (a.startsWith("-") || files.length === 2) { - console.error(`unknown argument ${a}. ${more}`); + if (a.startsWith("-") || files.length === 2) { + console.error(`unknown argument ${a}.`); process.exit(2); } else files.push(a); } if (files.length < 2) { - console.error(`${USAGE}\n${more}`); + console.error(USAGE); process.exit(2); } const [expectedFile, actualFile] = files; @@ -93,7 +61,9 @@ if (numbered) { process.exit(2); } -const rows = compareCharts(expectedRegions, actualRegions); +const rows = compareCharts(expectedRegions, actualRegions, { + loops: !actualText.includes(NO_LOOPS), +}); let differences = 0; for (const [title, xs] of rows) { console.log(`${title}:`); diff --git a/pstack/skills/statechart/scripts/lib.mjs b/pstack/skills/statechart/scripts/lib.mjs index 49f1675bc..238f1745e 100644 --- a/pstack/skills/statechart/scripts/lib.mjs +++ b/pstack/skills/statechart/scripts/lib.mjs @@ -1,15 +1,8 @@ import fs from "node:fs"; -import process from "node:process"; +import path from "node:path"; -export function findTlc() { - const { TLA2TOOLS, JAVA } = process.env; - if (!TLA2TOOLS) - throw new Error( - "Cannot find TLC. Download tla2tools.jar from a TLA+ release:\n https://github.com/tlaplus/tlaplus/releases\nThen set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later).", - ); - if (!fs.existsSync(TLA2TOOLS)) throw new Error(`TLA2TOOLS is ${TLA2TOOLS}, which does not exist.`); - return { jar: TLA2TOOLS, java: JAVA || "java" }; -} +const tuples = new WeakSet(); +export const isTuple = (v) => tuples.has(v); export function parseValue(text) { const src = text.trim(); @@ -45,7 +38,9 @@ export function parseValue(text) { ws(); if (peek("<<")) { i += 2; - return list(">>", value); + const items = list(">>", value); + tuples.add(items); + return items; } if (peek("{")) { i++; @@ -106,7 +101,7 @@ export function parseValue(text) { } // One state as TLC prints it: "/\ x = 1" lines, or "x = 1" for one variable. -function parseState(text) { +export function parseState(text) { const lines = text.split("\n").filter((l) => l.trim() !== ""); const conjuncts = []; for (const line of lines) { @@ -154,7 +149,7 @@ export function ruleSentences(tla) { export const tlaVariables = (code) => [...code.matchAll(/\bVARIABLES?\s+(\w+(?:\s*,\s*\w+)*)/g)].flatMap((m) => m[1].split(/\s*,\s*/)); -function splitCommentsKeepingOffsets(text) { +export function splitCommentsKeepingOffsets(text) { let code = ""; let notes = ""; let block = 0; @@ -199,6 +194,9 @@ function splitCommentsKeepingOffsets(text) { } const IDENTIFIER = /(?)/g; +const QUALIFIED = /(?>|[A-Za-z_]\w*)/g; const CITATION = /[\w./-]+\.[A-Za-z]\w*:\d+|#L\d+/; const DEPTH = { "(": 1, "[": 1, "{": 1, ")": -1, "]": -1, "}": -1 }; @@ -246,24 +244,13 @@ function conjuncts(bare, from, to, lineStart) { return [first, ...cuts.map((c) => c + 2)].map((a, i) => [a, cuts[i] ?? to]); } -export function uncitedConditions(text) { - const { code, notes } = splitCommentsKeepingOffsets(text); - const bare = code.replace(/"(?:[^"\\\n]|\\.)*"/g, (s) => " ".repeat(s.length)); - const variables = new Set(tlaVariables(bare)); - const starts = [0, ...[...bare.matchAll(/\n/g)].map((m) => m.index + 1)]; - const lineOf = (at) => { - let lo = 0; - let hi = starts.length - 1; - while (lo < hi) { - const mid = (lo + hi + 1) >> 1; - if (starts[mid] <= at) lo = mid; - else hi = mid - 1; - } - return lo; - }; - const lineStart = (at) => starts[lineOf(at)]; - const lines = { bare: bare.split("\n"), notes: notes.split("\n") }; +export const blankStrings = (code) => + code.replace(/"(?:[^"\\\n]|\\.)*"/g, (s) => " ".repeat(s.length)); +export function definitions(text, instancedReads = new Map(), parameters = []) { + const { code, notes } = splitCommentsKeepingOffsets(text); + const bare = blankStrings(code); + const variables = new Set([...parameters, ...tlaVariables(bare)]); const defs = new Map(); const nextItem = /^(?![ \t\n]|\/\\|\\\/)/gm; for (const m of bare.matchAll( @@ -271,20 +258,24 @@ export function uncitedConditions(text) { )) { const from = m.index + m[0].length; nextItem.lastIndex = from; - defs.set(m[1], { from, to: nextItem.exec(bare)?.index ?? bare.length, line: lineOf(m.index) }); + defs.set(m[1], { at: m.index, from, to: nextItem.exec(bare)?.index ?? bare.length }); } + const known = (id) => defs.has(id) || instancedReads.has(id); const direct = (body) => { - const ids = body.match(IDENTIFIER) ?? []; + const plain = body.replace(UNCHANGED_CLAUSE, " ").replace(PRIMED, " "); + const qualified = (plain.match(QUALIFIED) ?? []).filter(known); + const ids = plain.replace(QUALIFIED, (q) => (known(q) ? " " : q)).match(IDENTIFIER) ?? []; return { reads: ids.filter((id) => variables.has(id)), - writes: [...body.matchAll(/(? m[1]) - .filter((id) => variables.has(id)), + writes: [...body.matchAll(PRIMED)].map((m) => m[1]).filter((id) => variables.has(id)), acts: PRIMES_OR_UNCHANGED.test(body), - calls: [...new Set(ids.filter((id) => defs.has(id)))], + calls: [...new Set([...ids.filter(known), ...qualified])], }; }; - const own = new Map([...defs].map(([name, d]) => [name, direct(bare.slice(d.from, d.to))])); + const own = new Map([ + ...[...instancedReads].map(([name, reads]) => [name, direct(reads.join(" "))]), + ...[...defs].map(([name, d]) => [name, direct(bare.slice(d.from, d.to))]), + ]); const through = (start) => { const out = { reads: new Set(start.reads), writes: new Set(start.writes), acts: start.acts }; const seen = new Set(start.calls); @@ -301,6 +292,24 @@ export function uncitedConditions(text) { } return out; }; + return { code, notes, bare, defs, direct, own, through }; +} + +export function uncitedConditions(text) { + const { code, notes, bare, defs, direct, own, through } = definitions(text); + const starts = [0, ...[...bare.matchAll(/\n/g)].map((m) => m.index + 1)]; + const lineOf = (at) => { + let lo = 0; + let hi = starts.length - 1; + while (lo < hi) { + const mid = (lo + hi + 1) >> 1; + if (starts[mid] <= at) lo = mid; + else hi = mid - 1; + } + return lo; + }; + const lineStart = (at) => starts[lineOf(at)]; + const lines = { bare: bare.split("\n"), notes: notes.split("\n") }; const cited = (from, to, { above }) => { for (let l = from; l <= to; l++) if (CITATION.test(lines.notes[l])) return true; for (let l = from - 1; above && !lines.bare[l].trim() && lines.notes[l].trim(); l--) @@ -317,7 +326,7 @@ export function uncitedConditions(text) { if (part.acts || [...part.reads].every((v) => changes.has(v))) continue; const first = lineOf(a + bare.slice(a, b).search(/\S/)); const last = lineOf(a + bare.slice(a, b).trimEnd().length - 1); - if (cited(first, last, { above: first > d.line })) continue; + if (cited(first, last, { above: first > lineOf(d.at) })) continue; found.push({ line: first + 1, action, @@ -328,6 +337,33 @@ export function uncitedConditions(text) { return found; } +const TEMPORAL = /\[\]|(?(?!>)|~>|-\+->|\\(?:AA|EE)\b|\b[WS]F_/; + +export function checkedOnlyInInitialStates(spec, property) { + const bodies = new Map(); + const modules = new Set(); + const read = (module) => { + const file = path.join(path.dirname(spec), `${module}.tla`); + if (modules.has(module) || !fs.existsSync(file)) return; + modules.add(module); + const { bare, defs } = definitions(fs.readFileSync(file, "utf8")); + for (const [name, d] of defs) + bodies.set(name, [...(bodies.get(name) ?? []), bare.slice(d.from, d.to)]); + for (const [, names] of bare.matchAll(/\b(?:EXTENDS|INSTANCE)\s+(\w+(?:\s*,\s*\w+)*)/g)) + names.split(/\s*,\s*/).forEach(read); + }; + read(path.basename(spec, ".tla")); + const seen = new Set(); + const temporal = (name) => { + if (seen.has(name)) return false; + seen.add(name); + return (bodies.get(name) ?? []).some( + (body) => TEMPORAL.test(body) || (body.match(IDENTIFIER) ?? []).some(temporal), + ); + }; + return bodies.has(property) && !temporal(property); +} + function unescapeDot(s) { let out = ""; for (let i = 0; i < s.length; i++) { @@ -350,7 +386,7 @@ export function parseDot(text) { for (const line of text.split("\n")) { let m = edgeRe.exec(line); if (m) { - const e = { from: m[1], to: m[2], action: m[3] ? unescapeDot(m[3]) : "" }; + const e = { from: m[1], to: m[2], action: m[3] ? unescapeDot(m[3]).split("(")[0] : "" }; const id = `${e.from} ${e.to} ${e.action}`; if (seen.has(id)) continue; seen.add(id); @@ -408,8 +444,8 @@ export function parseTraces(text) { const [vars, end] = stateAt(i); i = end; if (head[1] === "1") { - current = { invariant, steps: [] }; - traces.push(current); + current = invariant && { invariant, steps: [] }; + if (current) traces.push(current); } current?.steps.push({ action: head[2] === "Initial" ? null : head[2], vars }); } @@ -420,41 +456,36 @@ export function parseTraces(text) { // action property, a bare state predicate, which TLC checks only in the initial states, as a // property of the initial state, and any other formula as a temporal property. export const BROKEN_PROPERTY = - /^Error: (Invariant \S+ is violated|Action property \S+ is violated\.|Property \S+ is violated by the initial state:|Temporal properties were violated\.)/m; + /^Error: (Invariant .+? is violated|Action property .+? is violated\.|Property .+? is violated by the initial state:|Temporal properties were violated\.)/m; // With -continue, TLC prints "Model checking completed" even after broken invariants, and not -// when it stopped early. TLC can also throw a Java exception of its own while it reports a state -// that breaks a rule, and stop early. An error in the rule's formula throws one of TLC's own -// exceptions in place of java.*. +// when it stopped early. export function tlcOutputWarning(text, file) { const lines = text.split("\n"); - const named = (re) => new Set(lines.map((l) => re.exec(l)?.[1]).filter(Boolean)); - const broken = named(/^Error: Invariant (\S+) is violated/); - const failed = [...named(/^Error: Evaluating invariant (\S+) failed\./)]; - const thrown = [...text.matchAll(/^The exception was a (\S+)\n: (?:([\w.$]+):)?(.*)$/gm)].map( - ([, cls, cause, message]) => `${cause ?? cls}: ${message.trim()}`, - ); - const crashed = - failed.length > 0 && - failed.every((r) => broken.has(r)) && - thrown.length > 0 && - thrown.every((t) => t.startsWith("java.")); - const reported = crashed - ? /^Error: (Invariant \S+ is violated|The behavior up to this point is|Evaluating invariant \S+ failed\.|TLC threw an unexpected exception\.|The error occurred when TLC was evaluating the nested)/ - : /^Error: (Invariant \S+ is violated|Action property \S+ is violated\.|Property \S+ is violated by the initial state:|The behavior up to this point is|Temporal properties were violated\.|The following behavior constitutes a counter-example:)/; + const broken = [ + ...new Set( + lines.map((l) => /^Error: Invariant (\S+) is violated/.exec(l)?.[1]).filter(Boolean), + ), + ]; + const reported = + /^Error: (Invariant .+? is violated|Action property .+? is violated\.|Property .+? is violated by the initial state:|The behavior up to this point is|Temporal properties were violated\.|The following behavior constitutes a counter-example:)/; const other = lines.find((l) => l.startsWith("Error: ") && !reported.test(l)); + if (other && broken.length) { + const thrown = /^The exception was a (\S+)\n: (?:([\w$]+(?:\.[\w$]+)+):)?(.*)$/m.exec(text); + const error = thrown + ? `${thrown[2] ?? thrown[1]}: ${thrown[3].trim()}` + : other.slice(7).replace(/\.$/, ""); + const rules = joinAnd(broken); + return `TLC found that ${rules} ${broken.length === 1 ? "is" : "are"} broken, and then stopped on an error: ${error}. The paths that TLC printed before it stopped are complete, but TLC did not check every state, so the charts can miss states that break ${rules}. A smaller constant can let TLC finish.`; + } if (other) throw new Error( `${file}: TLC stopped on an error that is not a broken rule: ${other.slice(7)}`, ); - if (crashed) { - const rules = failed.join(", "); - return `TLC found that ${rules} ${failed.length === 1 ? "is" : "are"} broken, and then failed with ${thrown[0]}. TLC stopped before it checked every state, so the charts can miss states that break ${rules}. The failure is in TLC, not in the rule. Do not weaken ${rules} because of it. A smaller constant can let TLC finish.`; - } // TLC stops at the first broken temporal property, and at an initial state that breaks a // PROPERTY, and then does not print that it completed. if ( - !/^(Model checking completed\.|Error: (Temporal properties were violated\.|Property \S+ is violated by the initial state:))/m.test( + !/^(Model checking completed\.|Error: (Temporal properties were violated\.|Property .+? is violated by the initial state:))/m.test( text, ) ) @@ -508,6 +539,8 @@ export const escapeMermaidQuotes = (s) => String(s).replaceAll('"', "#quot;"); export const LEGEND_ID = "legend"; export const FULL_LABELS = "%% labels in full"; +export const NO_LOOPS = "%% self-loops left out"; +export const BROKEN_MARK = "statechart broken"; export function parseMermaid(text) { const names = new Map(); @@ -764,7 +797,7 @@ function guardHolds(guard, at) { return guard.not ? !hits.some(Boolean) : hits.every(Boolean); } -export function productOf(regions) { +export function productOf(regions, { loops = true } = {}) { const names = Object.keys(regions); const rs = Object.values(regions); const join = (parts) => parts.join(" | "); @@ -793,7 +826,7 @@ export function productOf(regions) { const queue = [...starts]; const step = (from, to, label) => { const [a, b] = [join(from), join(to)]; - if (a === b) return; + if (a === b && !loops) return; out.transitions.push({ from: a, to: b, label }); if (!seen.has(b)) { seen.add(b); @@ -826,9 +859,9 @@ export function productOf(regions) { return out; } -export function compareCharts(expectedRegions, actualRegions) { - const expected = productOf(expectedRegions); - const actual = productOf(actualRegions); +export function compareCharts(expectedRegions, actualRegions, { loops = true } = {}) { + const expected = productOf(expectedRegions, { loops }); + const actual = productOf(actualRegions, { loops }); const edgeSet = (m) => { const labeled = new Set(); const out = new Set(); @@ -873,9 +906,15 @@ export const STATUS_CLASSES = [ "classDef works fill:#f4f6f7,stroke:#566573,color:#1b2631", "classDef wrong fill:#fdedec,stroke:#c0392b,color:#7b241c", "classDef fixed fill:#e9f7ef,stroke:#1e8449,color:#145a32", + "classDef mixed fill:#fdebd0,stroke:#d35400,color:#7e5109", + "classDef after fill:#ffffff,stroke:#808b96,color:#566573", + "classDef worse fill:#f4ecf7,stroke:#7d3c98,color:#4a235a", "classDef worksPath fill:#f4f6f7,stroke:#1f3a93,stroke-width:4px,color:#1b2631", "classDef wrongPath fill:#fdedec,stroke:#1f3a93,stroke-width:4px,color:#7b241c", "classDef fixedPath fill:#e9f7ef,stroke:#1f3a93,stroke-width:4px,color:#145a32", + "classDef mixedPath fill:#fdebd0,stroke:#1f3a93,stroke-width:4px,color:#7e5109", + "classDef afterPath fill:#ffffff,stroke:#1f3a93,stroke-width:4px,color:#566573", + "classDef worsePath fill:#f4ecf7,stroke:#1f3a93,stroke-width:4px,color:#4a235a", ]; export const GROUP_PALETTE = [ @@ -910,6 +949,9 @@ export function groupColorProblem(color) { const BORDERS = { Wrong: ["stroke:#c0392b,stroke-width:4px", "#7b241c"], Fixed: ["stroke:#1e8449,stroke-width:4px", "#145a32"], + Mixed: ["stroke:#d35400,stroke-width:4px", "#7e5109"], + After: ["stroke:#808b96,stroke-width:4px", "#566573"], + Worse: ["stroke:#7d3c98,stroke-width:4px", "#4a235a"], Path: ["stroke:#1f3a93,stroke-width:4px", "#1b2631"], }; export function groupClassDef(base, fill, border) { diff --git a/pstack/skills/statechart/scripts/map-view.mjs b/pstack/skills/statechart/scripts/map-view.mjs new file mode 100644 index 000000000..e7e844b6d --- /dev/null +++ b/pstack/skills/statechart/scripts/map-view.mjs @@ -0,0 +1,92 @@ +import { isRecord, isTuple } from "./lib.mjs"; + +export function mapView(value, at = "v") { + if (isRecord(value)) + return Object.fromEntries(Object.entries(value).map(([k, x]) => [k, mapView(x, `${at}.${k}`)])); + if (!Array.isArray(value)) return value; + if (isTuple(value)) + return tupleView( + value.map((x, i) => mapView(x, `${at}[${i + 1}]`)), + at, + ); + return arrayView(value.map((x, i) => mapView(x, `${at}[${i}]`))); +} +const arrayView = (items) => + Object.defineProperties(items, { + flat: { value: (depth = 1) => flatten(items, depth) }, + flatMap: { value: (f, self) => flatten(items.map(f, self), 1) }, + }); +const views = new WeakMap(); +const itemAt = (k) => (typeof k === "string" && /^[1-9]\d*$/.test(k) ? k - 1 : -1); +const flatten = (xs, depth) => + Array.prototype.flatMap.call(xs, (x) => + depth >= 1 && Array.isArray(x) ? flatten(views.get(x)?.items ?? x, depth - 1) : [x], + ); +function tupleView(items, at) { + const view = new Proxy(items, { + get(_, k) { + if (k === "0") + throw new Error( + `the mapping reads ${at}[0], but a tuple reads from 1, as in TLA+. Read ${at}[1] for its first item.`, + ); + const i = itemAt(k); + if (i >= 0) return items[i]; + if (k === "toJSON") return () => items; + if (k === "flat") return (depth = 1) => flatten(items, depth); + if (k === "flatMap") return (f, self) => flatten(items.map(f, self), 1); + const x = Reflect.get(items, k); + if (typeof x !== "function") return x; + return x.bind(x === Object.prototype[k] ? view : items); + }, + set(_, k, x) { + if (k === "0") + throw new Error( + `the mapping writes ${at}[0], but a tuple reads from 1, as in TLA+. Write ${at}[1] for its first item.`, + ); + const i = itemAt(k); + if (i < 0) return Reflect.set(items, k, x); + items[i] = x; + return true; + }, + has(_, k) { + if (k === "0") + throw new Error( + `the mapping looks for index 0 in ${at}, with "in" or with an array method such as concat or flat, but a tuple reads from 1, as in TLA+. Use [...${at}] for an array that reads from 0.`, + ); + const i = itemAt(k); + return i < 0 ? Reflect.has(items, k) : i in items; + }, + ownKeys: () => + Reflect.ownKeys(items).map((k) => + typeof k === "string" && /^(0|[1-9]\d*)$/.test(k) ? String(+k + 1) : k, + ), + getOwnPropertyDescriptor(_, k) { + if (k === "0") return undefined; + const i = itemAt(k); + return Reflect.getOwnPropertyDescriptor(items, i < 0 ? k : i); + }, + }); + views.set(view, { items, at }); + return view; +} +const nativeClone = globalThis.structuredClone; +function cloneView(x, seen) { + if (x === null || typeof x !== "object") return nativeClone(x); + if (seen.has(x)) return seen.get(x); + const view = views.get(x); + if (!view && !Array.isArray(x) && Object.getPrototypeOf(x) !== Object.prototype) + return nativeClone(x); + const out = view || Array.isArray(x) ? [] : {}; + const copy = view ? tupleView(out, view.at) : Array.isArray(x) ? arrayView(out) : out; + seen.set(x, copy); + for (const [k, item] of Object.entries(view?.items ?? x)) out[k] = cloneView(item, seen); + return copy; +} +globalThis.structuredClone = (value, options) => { + try { + return nativeClone(value, options); + } catch (e) { + if (e?.name !== "DataCloneError") throw e; + return cloneView(value, new Map()); + } +}; diff --git a/pstack/skills/statechart/scripts/model/OutputFolder.cfg b/pstack/skills/statechart/scripts/model/OutputFolder.cfg deleted file mode 100644 index 68fd715a8..000000000 --- a/pstack/skills/statechart/scripts/model/OutputFolder.cfg +++ /dev/null @@ -1,6 +0,0 @@ -SPECIFICATION Spec -INVARIANT ChartsOneSpec -INVARIANT KeepsYourFiles -INVARIANT NeverLosesChart -INVARIANT ExitTwoLeavesFolder -INVARIANT OverlapNeverRefuses diff --git a/pstack/skills/statechart/scripts/model/OutputFolder.map.mjs b/pstack/skills/statechart/scripts/model/OutputFolder.map.mjs deleted file mode 100644 index 9618338f6..000000000 --- a/pstack/skills/statechart/scripts/model/OutputFolder.map.mjs +++ /dev/null @@ -1,106 +0,0 @@ -const QUIET = ["idle", "exited", "stopped", "orphanTlc"]; -const going = (v) => v.pc.filter((p) => !QUIET.includes(p)).length; -const changed = (before, after) => - after.pc.findIndex( - (p, i) => p !== before.pc[i] || after.aside[i].exists !== before.aside[i].exists, - ) + 1; - -export const readerActions = ["Start", "Edit", "Note", "Stop", "KillNode"]; - -function context(v) { - if (v.pc.every((p) => p === "idle")) return "Before any run"; - if (going(v) === 2) return "Two runs going"; - return going(v) === 1 ? "One run going" : "Done"; -} - -const SPECS = { 1: "the spec", 2: "the edited spec", "1,2": "both specs" }; - -function contents(v) { - const kept = v.aside.filter((a) => a.exists); - const aside = kept.some((a) => a.notes) - ? ", your notes moved aside" - : kept.some((a) => a.by) - ? ", the old chart moved aside" - : ""; - const parts = [ - v.out.by && `chart of ${SPECS[v.out.read.join()]}`, - v.out.notes && "your notes", - ].filter(Boolean); - if (!v.out.exists) return `no folder${aside}`; - return `${parts.join(" and ") || "empty folder"}${aside}`; -} - -export function state(v) { - return `${context(v)}: ${contents(v)}`; -} - -export function group(v) { - if (v.pc.every((p) => p === "idle")) return "Not started"; - return going(v) ? "Running" : "Finished"; -} - -export function step(action, before, after) { - const r = changed(before, after); - switch (action) { - case "Start": { - const where = going(before) ? " in a second shell" : ""; - const how = after.inputOk[r - 1] ? "" : " with a typo"; - return `you start run ${r}${where}${how}`; - } - case "Refuse": - return before.inputOk[r - 1] - ? `run ${r} says the folder has a file that it did not write` - : `run ${r} says the input is wrong`; - case "Tlc": - return `run ${r}'s TLC reads the spec`; - case "Draw": - return `run ${r} draws the chart in its own folder`; - case "Stale": - return `run ${r} says the spec changed while it read it`; - case "Foreign": - return `run ${r} says the folder has a file that it did not write`; - case "Swap": - case "Checked": - return ""; - case "MoveIn": - return `run ${r} moves its folder into place`; - case "MoveAside": - return `run ${r} moves the old folder aside`; - case "Restore": - return `run ${r} puts the folder with your notes back and stops`; - case "Supersede": - return `run ${r} finds another run's folder in place and deletes the old folder`; - case "KeepAside": - return `run ${r} keeps the folder with your notes aside and says where`; - case "RenameFails": - if (after.aside[r - 1].exists) - return `run ${r} cannot rename a folder and says where the old folder is`; - return after.out.exists && !before.out.exists - ? `run ${r} cannot rename a folder, puts the old folder back, and stops` - : `run ${r} cannot rename a folder and stops`; - case "Clean": - return `run ${r} deletes the old folder and finishes`; - case "Edit": - return "you edit the spec"; - case "Note": - return "you save notes in the folder"; - case "Stop": - return `you stop run ${r} with Ctrl-C`; - case "KillNode": - return `you kill run ${r}'s Node, and its TLC keeps running`; - case "OrphanEnd": - return `run ${r}'s orphaned TLC finishes`; - default: - return action; - } -} - -export function cause(v, facts, invariant) { - return { - ChartsOneSpec: "the run read the spec before and after an edit, and still wrote its chart", - KeepsYourFiles: "a run deleted a folder that had your notes in it", - NeverLosesChart: "a run deleted the last chart before another chart was in place", - ExitTwoLeavesFolder: "a run wrote the folder and then said it could not finish", - OverlapNeverRefuses: "a run stopped only because another run overlapped it", - }[invariant]; -} diff --git a/pstack/skills/statechart/scripts/model/OutputFolder.tla b/pstack/skills/statechart/scripts/model/OutputFolder.tla deleted file mode 100644 index 7765babcb..000000000 --- a/pstack/skills/statechart/scripts/model/OutputFolder.tla +++ /dev/null @@ -1,176 +0,0 @@ ----- MODULE OutputFolder ---- -EXTENDS Naturals - -Runs == {1, 2} -NoRun == 0 -NoFolder == [exists |-> FALSE, by |-> NoRun, read |-> {}, notes |-> FALSE] - -VARIABLES pc, inputOk, snap, read, out, aside, spec, noted, wrote, broke, exitCode - -vars == <> - -Init == - /\ pc = [r \in Runs |-> "idle"] - /\ inputOk = [r \in Runs |-> TRUE] - /\ snap = [r \in Runs |-> 0] - /\ read = [r \in Runs |-> {}] - /\ out = NoFolder - /\ aside = [r \in Runs |-> NoFolder] - /\ spec = 1 - /\ noted = FALSE - /\ wrote = [r \in Runs |-> FALSE] - /\ broke = [r \in Runs |-> FALSE] - /\ exitCode = [r \in Runs |-> "none"] - -Set(f, r, v) == [f EXCEPT ![r] = v] -Going(r) == pc[r] \notin {"idle", "exited", "stopped", "orphanTlc"} -Go(r, to) == pc' = Set(pc, r, to) -Exit(r, code) == pc' = Set(pc, r, "exited") /\ exitCode' = Set(exitCode, r, code) - -Start(r) == - /\ pc[r] = "idle" - /\ r = 2 => pc[1] # "idle" - /\ \E ok \in BOOLEAN : inputOk' = Set(inputOk, r, ok) - /\ snap' = Set(snap, r, spec) - /\ Go(r, "check") - /\ UNCHANGED <> - -Refuse(r) == - /\ pc[r] = "check" - /\ ~inputOk[r] \/ out.notes - /\ Exit(r, "2") - /\ UNCHANGED <> - -Tlc(r) == - /\ \/ pc[r] = "check" /\ inputOk[r] /\ ~out.notes - \/ pc[r] = "tlc" - /\ read' = Set(read, r, read[r] \cup {spec}) - /\ Go(r, IF pc[r] = "check" THEN "tlc" ELSE "draw") - /\ UNCHANGED <> - -Draw(r) == - /\ pc[r] = "draw" - /\ Go(r, "publish") - /\ UNCHANGED <> - -Stale(r) == - /\ pc[r] = "publish" /\ spec # snap[r] - /\ Exit(r, "2") - /\ UNCHANGED <> - -Foreign(r) == - /\ pc[r] = "publish" /\ spec = snap[r] /\ out.notes - /\ Exit(r, "2") - /\ UNCHANGED <> - -Swap(r) == - /\ pc[r] = "publish" /\ spec = snap[r] /\ ~out.notes - /\ Go(r, "swap") - /\ UNCHANGED <> - -MoveIn(r) == - /\ pc[r] = "swap" /\ ~out.exists - /\ out' = [exists |-> TRUE, by |-> r, read |-> read[r], notes |-> FALSE] - /\ wrote' = Set(wrote, r, TRUE) - /\ Go(r, "clean") - /\ UNCHANGED <> - -MoveAside(r) == - /\ pc[r] = "swap" /\ out.exists /\ ~aside[r].exists - /\ aside' = Set(aside, r, out) - /\ out' = NoFolder - /\ Go(r, "aside") - /\ UNCHANGED <> - -Checked(r) == - /\ pc[r] = "aside" /\ ~aside[r].notes - /\ Go(r, "swap") - /\ UNCHANGED <> - -Supersede(r) == - /\ pc[r] = "swap" /\ out.exists /\ aside[r].exists /\ ~out.notes - /\ aside' = Set(aside, r, NoFolder) - /\ UNCHANGED <> - -Restore(r) == - /\ pc[r] = "aside" /\ aside[r].notes /\ ~out.exists - /\ out' = aside[r] - /\ aside' = Set(aside, r, NoFolder) - /\ Exit(r, "2") - /\ UNCHANGED <> - -KeepAside(r) == - /\ \/ pc[r] = "aside" /\ aside[r].notes /\ out.exists - \/ pc[r] = "swap" /\ aside[r].exists /\ out.notes - /\ Exit(r, "2") - /\ UNCHANGED <> - -RenameFails(r) == - /\ pc[r] \in {"swap", "aside"} - /\ \/ /\ aside[r].exists /\ ~out.exists - /\ out' = aside[r] - /\ aside' = Set(aside, r, NoFolder) - \/ UNCHANGED <> - /\ broke' = Set(broke, r, TRUE) - /\ Exit(r, "2") - /\ UNCHANGED <> - -Clean(r) == - /\ pc[r] = "clean" - /\ aside' = Set(aside, r, NoFolder) - /\ Exit(r, "0") - /\ UNCHANGED <> - -Stop(r) == - /\ Going(r) - /\ Go(r, "stopped") - /\ UNCHANGED <> - -KillNode(r) == - /\ pc[r] = "tlc" - /\ Go(r, "orphanTlc") - /\ UNCHANGED <> - -OrphanEnd(r) == - /\ pc[r] = "orphanTlc" - /\ Go(r, "stopped") - /\ UNCHANGED <> - -Edit == - /\ spec = 1 - /\ spec' = 2 - /\ UNCHANGED <> - -Note == - /\ ~noted - /\ noted' = TRUE - /\ out' = [out EXCEPT !.exists = TRUE, !.notes = TRUE] - /\ UNCHANGED <> - -Next == - \/ Edit \/ Note - \/ \E r \in Runs : - Start(r) \/ Refuse(r) \/ Tlc(r) \/ Draw(r) \/ Stale(r) \/ Foreign(r) \/ Swap(r) - \/ MoveIn(r) \/ MoveAside(r) \/ Checked(r) \/ Supersede(r) \/ Restore(r) - \/ KeepAside(r) \/ RenameFails(r) \/ Clean(r) \/ Stop(r) \/ KillNode(r) \/ OrphanEnd(r) - -Spec == Init /\ [][Next]_vars - -\* The folder never holds a chart drawn from two versions of the spec. -ChartsOneSpec == out.read \in {{}, {1}, {2}} - -\* A file that you put in the folder is never deleted. -KeepsYourFiles == noted => out.notes \/ \E r \in Runs : aside[r].notes - -\* Once a run writes a chart, the folder or a folder that a run moved aside always holds one. -NeverLosesChart == - (\E r \in Runs : wrote[r]) => out.by # NoRun \/ \E r \in Runs : aside[r].by # NoRun - -\* A run that exits 2 did not write the folder. -ExitTwoLeavesFolder == \A r \in Runs : exitCode[r] = "2" => ~wrote[r] - -\* A run exits 2 only for a wrong input, an edited spec, a file that it did not write, or a failed -\* rename, never because another run overlaps it. -OverlapNeverRefuses == - \A r \in Runs : exitCode[r] = "2" => ~inputOk[r] \/ spec # snap[r] \/ noted \/ broke[r] -==== diff --git a/pstack/skills/statechart/scripts/property-loop.mjs b/pstack/skills/statechart/scripts/property-loop.mjs new file mode 100644 index 000000000..47bf0372f --- /dev/null +++ b/pstack/skills/statechart/scripts/property-loop.mjs @@ -0,0 +1,60 @@ +import fs from "node:fs"; +import { joinAnd, parseDot, parseState, stateKey } from "./lib.mjs"; +import { mapView } from "./map-view.mjs"; + +const TEMPORAL_BREAK = "Error: Temporal properties were violated."; + +function lasso(tlcOutput) { + const start = tlcOutput.lastIndexOf(TEMPORAL_BREAK); + if (start < 0) return null; + const lines = tlcOutput.slice(start).split("\n"); + const states = []; + for (let i = 0; i < lines.length; i++) { + const back = /^Back to state (\d+): /.exec(lines[i]); + if (back) return { loop: states.slice(Number(back[1]) - 1), stutters: false }; + const head = /^State \d+: (<|Stuttering)/.exec(lines[i]); + if (!head) continue; + if (head[1] === "Stuttering") return { loop: states.slice(-1), stutters: true }; + const body = []; + while (i + 1 < lines.length && lines[i + 1].trim() !== "") body.push(lines[++i]); + states.push(parseState(body.join("\n"))); + } + return null; +} + +const collapseRepeatsAroundLoop = (names) => { + const runs = names.filter((name, i) => name !== names[i - 1]); + return runs.length > 1 && runs.at(-1) === runs[0] ? runs.slice(0, -1) : runs; +}; + +export function repeatsIn(tlcOutput, nameOf) { + const found = lasso(tlcOutput); + return ( + found && { states: collapseRepeatsAroundLoop(found.loop.map(nameOf)), stutters: found.stutters } + ); +} + +function enabledByState({ nodes, edges, loops }) { + const byId = new Map([...nodes.keys()].map((id) => [id, new Set()])); + for (const e of [...edges, ...loops]) byId.get(e.from).add(e.action); + return new Map([...nodes.values()].map((n) => [stateKey(n.vars), byId.get(n.id)])); +} + +export function readerStateNamer(mapping, dotFile) { + const readerActions = new Set(mapping.readerActions ?? []); + let enabledAt; + return (vars) => { + enabledAt ??= enabledByState(parseDot(fs.readFileSync(dotFile, "utf8"))); + const enabled = enabledAt.get(stateKey(vars)) ?? new Set(); + const settled = [...enabled].every((a) => readerActions.has(a)); + return String(mapping.state(mapView(vars), { enabled, settled })); + }; +} + +export function loopSentence(repeats) { + if (!repeats) return null; + const quoted = repeats.states.map((name) => `"${name}"`); + if (repeats.stutters) return `The run stops in ${quoted[0]} forever.`; + if (quoted.length === 1) return `The run stays in ${quoted[0]} forever.`; + return `The run loops through ${joinAnd(quoted)} forever.`; +} diff --git a/pstack/skills/statechart/scripts/read-check.mjs b/pstack/skills/statechart/scripts/read-check.mjs new file mode 100644 index 000000000..18c52b6d1 --- /dev/null +++ b/pstack/skills/statechart/scripts/read-check.mjs @@ -0,0 +1,79 @@ +import { isRecord } from "./lib.mjs"; +import { SEP } from "./chart-parts.mjs"; + +const kindOf = (v) => (isRecord(v) ? "record" : Array.isArray(v) ? "list" : typeof v); +function shapeOf(graph, traces) { + const shape = new Map(); + const walk = (o, path) => { + for (const [k, v] of Object.entries(o)) { + const p = [...path, k]; + const id = p.join(SEP); + if (!shape.has(id)) shape.set(id, { path: p, kinds: new Set() }); + shape.get(id).kinds.add(kindOf(v)); + if (isRecord(v)) walk(v, p); + } + }; + for (const n of graph.nodes.values()) walk(n.vars, []); + for (const t of traces) for (const s of t.steps) walk(s.vars, []); + return shape; +} +const disjoint = (a, b) => ![...b.kinds].some((k) => a.kinds.has(k)); +const foreign = (a, b) => [...b.kinds].some((k) => !a.kinds.has(k)); +function lacking(a, b) { + const out = []; + for (const [id, x] of a) { + const within = out.some((p) => id.startsWith(`${p.join(SEP)}${SEP}`)); + if (!within && (!b.has(id) || disjoint(x, b.get(id)))) out.push(x.path); + } + return out; +} +const unreadOf = (a, b) => + [...a].flatMap(([id, x]) => + !b.has(id) || foreign(x, b.get(id)) ? [{ ...x, somewhere: b.has(id) }] : [], + ); +function guard(vars, unread, label, at = "v") { + if (!unread.length || !isRecord(vars)) return vars; + const here = new Map(); + const deeper = new Map(); + for (const { path, ...x } of unread) { + const [k, ...rest] = path; + if (!rest.length) here.set(k, x); + else deeper.set(k, [...(deeper.get(k) ?? []), { ...x, path: rest }]); + } + const check = (target, k, listing) => { + const x = here.get(k); + const absent = !Object.hasOwn(target, k); + if (!x || (absent ? x.somewhere : x.kinds.has(kindOf(target[k])))) return; + const name = `${at}.${k}`.slice(2); + const has = absent ? `no ${name}` : `${name} as a value of another type`; + const does = listing ? `lists the fields of ${at}` : `reads ${at}.${k}`; + const hint = listing + ? ", and read the fields it needs by name" + : absent + ? `, or test "${k}" in ${at} first` + : ""; + throw new Error( + `${label} has ${has}, and the mapping ${does} on its states. Label states and steps only from what both models have${hint}.`, + ); + }; + return new Proxy(vars, { + get(target, k) { + check(target, k, false); + return deeper.has(k) ? guard(target[k], deeper.get(k), label, `${at}.${k}`) : target[k]; + }, + ownKeys(target) { + for (const k of here.keys()) check(target, k, true); + return Reflect.ownKeys(target); + }, + }); +} + +export function readCheck({ graph, rawTraces, secondGraph, rawSecondTraces, L }) { + const [a, b] = [shapeOf(graph, rawTraces), shapeOf(secondGraph, rawSecondTraces)]; + const unread = unreadOf(a, b); + return { + lacks: lacking(a, b), + extra: lacking(b, a), + guard: (vars) => guard(vars, unread, L), + }; +} diff --git a/pstack/skills/statechart/scripts/regions-chart.mjs b/pstack/skills/statechart/scripts/regions-chart.mjs new file mode 100644 index 000000000..95d746021 --- /dev/null +++ b/pstack/skills/statechart/scripts/regions-chart.mjs @@ -0,0 +1,307 @@ +import fs from "node:fs"; +import { + NO_LOOPS, + compareCharts, + escapeMermaidQuotes, + parseMermaid, + productOf, + wrap, +} from "./lib.mjs"; +import { SEP, addGuard, frontMatter, guarded } from "./chart-parts.mjs"; +import { RANK, diverges, worst } from "./bug-states.mjs"; +import { edgesAlong } from "./chart-edges.mjs"; + +export function regionsChart({ + states, + starts, + edges, + loops, + classes, + proofPath, + allPath, + mapping, + status, + LEGEND, + groupOf, + drawnState, + groupCounts, + head, +}) { + const norm = (s) => s.replaceAll(";", ",").replace(/\s+/g, " ").trim(); + const split = (k) => { + const at = k.indexOf(": "); + return at > 0 && at + 2 < k.length ? [norm(k.slice(0, at)), norm(k.slice(at + 2))] : null; + }; + const nodes = [...states].sort(); + if (!nodes.every(split)) return null; + const steps = []; + for (const e of edges.values()) { + if (e.labels.size === 0) return { why: "a step has no label" }; + for (const l of e.labels) + steps.push({ + from: split(e.from), + to: split(e.to), + label: norm(l), + words: e.guards?.get(l) ?? new Set([""]), + }); + } + const bar = (p) => p.join(" | "); + const flat = { + Main: { + states: new Set(nodes.map((k) => bar(split(k)))), + initial: new Set(starts.map((k) => bar(split(k)))), + transitions: steps.map((s) => ({ from: bar(s.from), to: bar(s.to), label: s.label })), + }, + }; + const ids = new Map([...flat.Main.states].map((s, i) => [s, `p${i}`])); + const proof = [ + ...frontMatter(), + "stateDiagram-v2", + ...(loops ? [] : [` ${NO_LOOPS}`]), + ...[...ids].map(([s, id]) => ` state "${escapeMermaidQuotes(s)}" as ${id}`), + ...[...flat.Main.initial].map((s) => ` [*] --> ${ids.get(s)}`), + ...steps.map((s) => ` ${ids.get(bar(s.from))} --> ${ids.get(bar(s.to))} : ${s.label}`), + ]; + if (proofPath) fs.writeFileSync(proofPath, proof.join("\n") + "\n"); + + const beside = [new Map(), new Map()]; + for (const k of nodes) { + const p = split(k); + for (const i of [0, 1]) { + if (!beside[i].has(p[i])) beside[i].set(p[i], new Set()); + beside[i].get(p[i]).add(p[1 - i]); + } + } + const size = beside[0].size + beside[1].size; + if (size >= nodes.length) + return { why: `${size} region states are not fewer than the ${nodes.length} flat states` }; + + const regionStatus = [new Map(), new Map()]; + for (const k of nodes) { + const p = split(k); + for (const i of [0, 1]) { + const prev = regionStatus[i].get(p[i]); + if (!prev || RANK.indexOf(status(k)) < RANK.indexOf(prev)) + regionStatus[i].set(p[i], status(k)); + } + } + for (const k of nodes) { + const p = split(k); + const shown = worst(...[0, 1].map((i) => regionStatus[i].get(p[i]))); + if (shown !== status(k)) + return { + why: `"${k}" is "${LEGEND[status(k)][1]}", and its two region states would show "${LEGEND[shown][1]}"`, + }; + } + + const moves = [new Map(), new Map()]; + for (const s of steps) + for (const i of [0, 1]) { + const id = [s.from[i], s.label, s.to[i]].join(SEP); + if (!moves[i].has(id)) + moves[i].set(id, { + from: s.from[i], + label: s.label, + to: s.to[i], + beside: new Set(), + guards: new Map(), + }); + moves[i].get(id).beside.add(s.from[1 - i]); + for (const w of s.words) addGuard(moves[i].get(id).guards, s.label, norm(w)); + } + for (const m of moves) + for (const [id, x] of m) + if ( + x.from === x.to && + ![...m.values()].some((y) => y.from === x.from && y.label === x.label && y.to !== x.from) + ) + m.delete(id); + const rid = beside.map((m, i) => new Map([...m.keys()].sort().map((s, j) => [s, `r${i}s${j}`]))); + const regionGroup = [new Map(), new Map()]; + for (const k of nodes) { + const p = split(k); + for (const i of [0, 1]) { + const prev = regionGroup[i].get(p[i]); + regionGroup[i].set(p[i], prev === undefined || prev === groupOf(k) ? groupOf(k) : ""); + } + } + const drawn = rid.map((m, i) => + [...m.keys()].map((s) => drawnState(regionGroup[i].get(s), regionStatus[i].get(s))), + ); + const draw = (only) => { + const out = head(drawn.flat()); + out.push(` state "At the same time" as regions {`); + let count = 0; + for (const i of [0, 1]) { + if (i) out.push(" --"); + const name = mapping.regionNames?.[i]; + const pad = name ? " " : " "; + if (name) out.push(` state "${escapeMermaidQuotes(wrap(name))}" as region${i} {`); + for (const s of new Set(starts.map((k) => split(k)[i]))) + out.push(`${pad}[*] --> ${rid[i].get(s)}`); + for (const [s, id] of rid[i]) + out.push(`${pad}state "${escapeMermaidQuotes(wrap(s))}" as ${id}`); + const shownMoves = [...moves[i]] + .filter(([id]) => !only || only[i].has(id)) + .map(([, m]) => m) + .sort((a, b) => a.from.localeCompare(b.from) || a.label.localeCompare(b.label)); + for (const m of shownMoves) { + const all = beside[i].get(m.from); + const not = [...all].filter((s) => !m.beside.has(s)).sort(); + const yes = [...m.beside].sort(); + const unless = not.length > 0 && not.length < yes.length; + const guard = not.length === 0 ? null : unless ? not : yes; + const worded = guarded(m.guards, m.label); + const text = + worded !== m.label || !guard + ? worded + : `${m.label} [${unless ? "unless" : "when"} ${guard.join(" or ")}]`; + out.push(`${pad}${rid[i].get(m.from)} --> ${rid[i].get(m.to)} : ${wrap(text, 30)}`); + if (guard) + out.push( + `${pad}%% when ${unless ? "not " : ""}${guard.map((s) => rid[1 - i].get(s)).join(", ")}`, + ); + count++; + } + if (name) out.push(" }"); + } + out.push(" }"); + for (const i of [0, 1]) + [...rid[i].values()].forEach((id, j) => out.push(` class ${id} ${drawn[i][j].cls}`)); + return { mermaid: out.join("\n") + "\n", count }; + }; + + const all = draw(null); + const differences = compareCharts(parseMermaid(all.mermaid).regions, flat, { loops }); + if (differences.length) { + const n = differences.reduce((sum, [, xs]) => sum + xs.length, 0); + return { + why: `the regions do not combine into the flat chart, with ${n} difference(s), such as ${differences[0][0]}: ${differences[0][1][0]}`, + }; + } + if (allPath) fs.writeFileSync(allPath, all.mermaid); + + const parts = new Map(nodes.map((k) => [bar(split(k)), split(k)])); + const key = (from, label, to) => `${from} --> ${to} : ${label}`; + const real = new Set(steps.map((s) => key(bar(s.from), s.label, bar(s.to)))); + const next = new Map(); + for (const s of steps) { + if (!next.has(bar(s.from))) next.set(bar(s.from), []); + next.get(bar(s.from)).push(s); + } + const keep = [new Set(), new Set()]; + const movesOf = (s) => + [0, 1] + .filter((i) => s.from[i] !== s.to[i]) + .map((i) => [i, [s.from[i], s.label, s.to[i]].join(SEP)]); + const price = (s) => movesOf(s).filter(([i, id]) => !keep[i].has(id)).length * nodes.length + 1; + const wanted = []; + const want = (s) => { + wanted.push(s); + for (const [i, id] of movesOf(s)) keep[i].add(id); + }; + const stepsOf = (e, said) => { + const labels = [...e.labels].map(norm); + return (labels.includes(said) ? [said] : labels).map((label) => ({ + from: split(e.from), + to: split(e.to), + label, + })); + }; + for (const c of classes) { + for (const step of edgesAlong(c)) { + const e = edges.get(step.id); + if (e) stepsOf(e, norm(step.label)).forEach(want); + } + if (diverges(c)) stepsOf(edges.get(c.second.edge), "").forEach(want); + } + const cheapest = () => { + const cost = new Map(starts.map((k) => [bar(split(k)), 0])); + const parent = new Map(); + const open = new Set(cost.keys()); + while (open.size) { + const k = [...open].reduce((a, b) => (cost.get(b) < cost.get(a) ? b : a)); + open.delete(k); + for (const s of next.get(k) ?? []) { + const c = cost.get(k) + price(s); + if (c < (cost.get(bar(s.to)) ?? Infinity)) { + cost.set(bar(s.to), c); + parent.set(bar(s.to), s); + open.add(bar(s.to)); + } + } + } + return { cost, parent }; + }; + const targets = [ + ...[0, 1].flatMap((i) => + [...beside[i].keys()].sort().map((r) => ({ at: (k) => parts.get(k)[i] === r })), + ), + ...[...new Set(steps.map((s) => s.label))] + .filter((l) => !wanted.some((s) => s.label === l)) + .sort() + .map((l) => ({ ends: (s) => s.label === l })), + ]; + while (targets.length) { + const { cost, parent } = cheapest(); + let best = null; + targets.forEach((t, n) => { + const ways = t.at + ? [...cost.keys()].filter(t.at).map((k) => ({ k, c: cost.get(k), last: null })) + : steps + .filter(t.ends) + .filter((s) => cost.has(bar(s.from))) + .map((s) => ({ k: bar(s.from), c: cost.get(bar(s.from)) + price(s), last: s })); + for (const w of ways) if (!best || w.c < best.c) best = { ...w, n }; + }); + if (!best) break; + targets.splice(best.n, 1); + const path = best.last ? [best.last] : []; + for (let s = parent.get(best.k); s; s = parent.get(bar(s.from))) path.push(s); + path.forEach(want); + } + const takes = (x, label) => + [0, 1].flatMap((i) => + [...moves[i]] + .filter(([, m]) => m.from === x[i] && m.label === label && m.beside.has(x[1 - i])) + .map(([id]) => [i, id]), + ); + let scoped; + let made; + for (;;) { + scoped = draw(keep); + made = productOf(parseMermaid(scoped.mermaid).regions, { loops }); + const madeKeys = new Set(made.transitions.map((t) => key(t.from, t.label, t.to))); + const redo = [ + ...made.transitions + .filter((t) => parts.has(t.from) && !real.has(key(t.from, t.label, t.to))) + .map((t) => [parts.get(t.from), t.label]), + ...wanted + .filter((s) => made.states.has(bar(s.from))) + .filter((s) => !madeKeys.has(key(bar(s.from), s.label, bar(s.to)))) + .map((s) => [s.from, s.label]), + ]; + const before = keep[0].size + keep[1].size; + for (const [x, label] of redo) for (const [i, id] of takes(x, label)) keep[i].add(id); + if (keep[0].size + keep[1].size === before) break; + } + const sound = + [...made.states].every((s) => parts.has(s)) && + made.transitions.every((t) => real.has(key(t.from, t.label, t.to))); + if (!sound) scoped = all; + + return { + mermaid: scoped.mermaid, + states: [beside[0].size, beside[1].size], + transitions: scoped.count, + all: all.count, + flat: { states: nodes.length, transitions: steps.length }, + status: Object.fromEntries( + RANK.filter((s) => LEGEND[s]).map((s) => [ + s, + regionStatus.reduce((n, m) => n + [...m.values()].filter((x) => x === s).length, 0), + ]), + ), + groups: groupCounts(drawn.flat()), + }; +} diff --git a/pstack/skills/statechart/scripts/render.mjs b/pstack/skills/statechart/scripts/render.mjs new file mode 100644 index 000000000..4c90c5853 --- /dev/null +++ b/pstack/skills/statechart/scripts/render.mjs @@ -0,0 +1,531 @@ +import { execFileSync, spawnSync } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import process from "node:process"; +import { pathToFileURL } from "node:url"; +import { GROUP_PALETTE, PLAIN_FILL } from "./lib.mjs"; + +export const PACKAGES = ["pele@1.8.0", "@resvg/resvg-js@2.6.2"]; +export const CACHE_NAME = PACKAGES.map((p) => p.replace(/^@[^/]+\//, "").replace("@", "-")).join( + "-", +); +const ENTRY = 'export { render } from "pele";\nexport { Resvg } from "@resvg/resvg-js";\n'; + +export const cacheDir = () => + path.join( + process.env.XDG_CACHE_HOME || path.join(os.homedir(), ".cache"), + "statechart", + CACHE_NAME, + ); + +function install(dir) { + fs.mkdirSync(path.dirname(dir), { recursive: true }); + const tmp = fs.mkdtempSync(`${dir}-`); + try { + fs.writeFileSync(path.join(tmp, "package.json"), '{ "private": true }\n'); + console.error(`Installing ${PACKAGES.join(" and ")} with npm in ${dir}. This happens once.`); + const r = spawnSync( + "npm", + ["install", "--no-audit", "--no-fund", "--ignore-scripts", "--save-exact", ...PACKAGES], + { + cwd: tmp, + encoding: "utf8", + env: { + ...process.env, + PATH: `${path.dirname(process.execPath)}${path.delimiter}${process.env.PATH ?? ""}`, + }, + shell: process.platform === "win32", + }, + ); + if (r.status !== 0) { + const said = + r.error?.message ?? + (`${r.stderr ?? ""}${r.stdout ?? ""}`.trim().split("\n").slice(-20).join("\n") || + `npm install exited ${r.status}, with no output`); + throw new Error( + `cannot install ${PACKAGES.join(" and ")} with npm:\n${said.replace(/^/gm, " ")}`, + ); + } + fs.writeFileSync(path.join(tmp, "entry.mjs"), ENTRY); + try { + fs.renameSync(tmp, dir); + } catch (e) { + if (!fs.existsSync(path.join(dir, "entry.mjs"))) throw e; + } + } finally { + fs.rmSync(tmp, { recursive: true, force: true }); + } +} + +export async function loadRenderer() { + const dir = cacheDir(); + const entry = path.join(dir, "entry.mjs"); + if (!fs.existsSync(entry)) install(dir); + try { + return await import(pathToFileURL(entry).href); + } catch (e) { + throw new Error( + `cannot load Pele and resvg from ${dir}: ${e.message}\nDelete that folder to install them again.`, + ); + } +} + +const STYLE = { line: 1.5, text: 0.4, arrow: 1.25, zoom: 2 }; +const MAX_PIXELS = 100_000_000; +const MARK = "2.5px"; +const FONT = "system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif"; + +export const THEMES = { + light: { + background: "#ffffff", + stateFill: "#fcfcfc", + stateBorder: "#c9c9c9", + stateText: "#1c1c1c", + edge: "#808080", + edgeLabel: "#656565", + region: { fill: "#f1f1f1", border: "#c9c9c9", text: "#1c1c1c" }, + groups: ["#c3e0ff", "#f0d3bf", "#e3d5ff", "#f0dcbf", "#bfe3df", "#e5d9ce", "#ffcbe6"], + groupBorder: "#c9c9c9", + wrong: { fill: "#ffecee", border: "#e72e3b", text: "#cd2834" }, + fixed: { fill: "#eaf9f2", border: "#10ab61" }, + mixed: { fill: "#fff3e6", border: "#e8730c", text: "#b85a06" }, + after: { fill: "#ffffff", border: "#9a9a9a", text: "#656565" }, + worse: { fill: "#f7edfd", border: "#9b35d6", text: "#8424b8" }, + path: "#1680f3", + }, + dark: { + background: "#1f1f1f", + stateFill: "#121212", + stateBorder: "#4e4e4e", + stateText: "#f5f5f5", + edge: "#939393", + edgeLabel: "#acacac", + region: { fill: "#2c2c2c", border: "#4e4e4e", text: "#f5f5f5" }, + groups: ["#1b3857", "#573117", "#3c2e57", "#573d17", "#174641", "#3d3127", "#57233e"], + groupBorder: "#4e4e4e", + wrong: { fill: "#2e2021", border: "#cf2b35", text: "#ff6463" }, + fixed: { fill: "#1e2a25", border: "#0e9f5a" }, + mixed: { fill: "#2e2619", border: "#d0700f", text: "#ffa257" }, + after: { fill: "#1f1f1f", border: "#6e6e6e", text: "#acacac" }, + worse: { fill: "#29202e", border: "#9a3fcf", text: "#d38cff" }, + path: "#116dd2", + }, + classic: { + background: "#ffffff", + stateFill: PLAIN_FILL, + stateBorder: "#566573", + stateText: "#1b2631", + edge: "#1f1f1f", + edgeLabel: "#1f1f1f", + region: { fill: "#e6e6e6", border: "#8a8a8a", text: "#1f1f1f" }, + groups: null, + groupBorder: null, + wrong: { fill: "#fadbd8", border: "#922b21", text: "#7b241c" }, + fixed: { fill: "#d5f5e3", border: "#196f3d" }, + mixed: { fill: "#fdebd0", border: "#d35400", text: "#7e5109" }, + after: { fill: "#ffffff", border: "#808b96", text: "#566573" }, + worse: { fill: "#ebdef0", border: "#6c3483", text: "#4a235a" }, + path: "#1f3a93", + }, +}; + +const hex6 = (c) => { + const m = /^#([0-9a-f]{3}|[0-9a-f]{6})$/i.exec(c?.trim() ?? ""); + if (!m) return null; + const h = m[1].toLowerCase(); + return `#${h.length === 3 ? [...h].map((x) => x + x).join("") : h}`; +}; +const rgb = (hex) => [1, 3, 5].map((i) => parseInt(hex.slice(i, i + 2), 16)); +const lab = (hex) => { + const [r, g, b] = rgb(hex).map((c) => + (c /= 255) <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4, + ); + const f = (t) => (t > 0.008856 ? Math.cbrt(t) : 7.787 * t + 16 / 116); + const [x, y, z] = [ + (0.4124 * r + 0.3576 * g + 0.1805 * b) / 0.95047, + 0.2126 * r + 0.7152 * g + 0.0722 * b, + (0.0193 * r + 0.1192 * g + 0.9505 * b) / 1.08883, + ].map(f); + return [116 * y - 16, 500 * (x - y), 200 * (y - z)]; +}; +const deltaE = (a, b) => Math.hypot(...lab(a).map((v, i) => v - lab(b)[i])); + +const STATE_CLASS = + /^(plain|group\d+|works|wrong|fixed|mixed|after|worse)(Wrong|Fixed|Mixed|After|Worse|Path)?$/; +const CLASS_DEF = /^(\s*classDef\s+)(\w+)\s+(\S+?)\s*$/gm; +const rawFill = (body) => /(?:^|,)fill:([^,]+)/.exec(body)?.[1]; +const STATUSES = ["wrong", "fixed", "mixed", "after", "worse"]; + +function markOf(name) { + const m = STATE_CLASS.exec(name); + if (!m) return null; + const [, base, mark = ""] = m; + const own = mark.toLowerCase(); + if (STATUSES.includes(own)) return own; + return STATUSES.includes(base) ? base : own; +} + +function paletteSlots(fills) { + const slot = new Map(); + const taken = new Set(); + for (const f of fills) { + const k = GROUP_PALETTE.indexOf(f); + if (k >= 0) { + slot.set(f, k); + taken.add(k); + } + } + const all = GROUP_PALETTE.map((_, k) => k); + for (const f of fills) { + if (f === PLAIN_FILL || slot.has(f)) continue; + const free = all.filter((k) => !taken.has(k)); + const [k] = (free.length ? free : all).sort( + (a, b) => deltaE(f, GROUP_PALETTE[a]) - deltaE(f, GROUP_PALETTE[b]), + ); + slot.set(f, k); + taken.add(k); + } + return slot; +} + +const marked = ({ fill, border, text }) => + `fill:${fill},stroke:${border},stroke-width:${MARK},color:${text}`; +const STYLES = { + wrong: ({ wrong }) => marked(wrong), + mixed: ({ mixed }) => marked(mixed), + after: ({ after }) => marked(after), + worse: ({ worse }) => marked(worse), + fixed: ({ fixed: f, stateText }) => + `fill:${f.fill},stroke:${f.border},stroke-width:${MARK},color:${stateText}`, + path: ({ path, stateText }, fill) => + `${fill ? `fill:${fill},` : ""}stroke:${path},stroke-width:${MARK},color:${stateText}`, + "": ({ groups, stateFill, stateBorder, groupBorder, stateText }, fill) => + groups && + `${fill ? `fill:${fill},` : ""}stroke:${fill === stateFill ? stateBorder : groupBorder},color:${stateText}`, +}; + +function themed(src, theme) { + const own = new Set(); + for (const [, , name, body] of src.matchAll(CLASS_DEF)) { + const fill = hex6(rawFill(body)); + if (fill && (markOf(name) === "" || markOf(name) === "path")) own.add(fill); + } + const slot = paletteSlots([...own]); + const fillFor = (body) => { + const raw = rawFill(body); + const fill = hex6(raw); + if (!theme.groups || !fill) return raw; + if (fill === PLAIN_FILL) return theme.stateFill; + return slot.has(fill) ? theme.groups[slot.get(fill)] : raw; + }; + return src.replace(CLASS_DEF, (line, head, name, body) => { + const mark = markOf(name); + const style = mark === null ? null : STYLES[mark](theme, fillFor(body)); + return style ? `${head}${name} ${style}` : line; + }); +} + +function withRootDirection(src) { + const lines = src.split("\n"); + let at = -1; + let depth = 0; + let regions = false; + let direction = false; + for (const [i, line] of lines.entries()) { + const code = line + .replace(/"[^"]*"/g, '""') + .replace(/:.*$/, "") + .trim(); + if (at < 0) { + if (/^stateDiagram/.test(code)) at = i; + continue; + } + if (code.startsWith("%%")) continue; + if (code === "--") regions = true; + if (depth === 0 && /^direction\s/.test(code)) direction = true; + depth += (code.match(/\{/g) ?? []).length - (code.match(/\}/g) ?? []).length; + } + if (at < 0 || !regions || direction) return src; + lines.splice(at + 1, 0, " direction LR"); + return lines.join("\n"); +} + +const SYSTEM_KEYWORD = /^(system-ui|-apple-system|BlinkMacSystemFont|ui-sans-serif)$/i; + +export function fcMatch(pattern) { + try { + const [file, name] = execFileSync("fc-match", ["-f", "%{file}\n%{family[0]}", pattern], { + encoding: "utf8", + stdio: ["ignore", "pipe", "ignore"], + }).split("\n"); + return file ? { file, name } : null; + } catch { + return null; + } +} + +let systemFont; +export function systemFamily() { + if (systemFont === undefined) systemFont = fcMatch("system-ui")?.name ?? null; + return systemFont; +} + +function pngStack(sys) { + if (!sys) return FONT; + const names = FONT.split(",") + .map((f) => f.trim().replace(/^'|'$/g, "")) + .map((f) => (SYSTEM_KEYWORD.test(f) ? sys : f)); + return [...new Set(names)].map((f) => (/^[\w-]+$/.test(f) ? f : `'${f}'`)).join(", "); +} + +const xml = (s) => s.replace(/&/g, "&").replace(//g, ">"); + +const measurers = new Map(); +export function fontMeasurer(Resvg, family) { + if (measurers.has(family)) return measurers.get(family); + const ink = (face, text, size, style) => { + const svg = + `${xml(text)}`; + const font = face.file + ? { loadSystemFonts: false, fontFiles: [face.file] } + : { loadSystemFonts: true }; + return new Resvg(svg, { font }).getBBox()?.width; + }; + const draws = (face, style) => ink(face, "x", 16, style) > 0; + const matchFace = (style) => { + const name = style & 4 ? "monospace" : family; + return fcMatch(`${name}${style & 1 ? ":bold" : ""}${style & 2 ? ":italic" : ""}`) ?? { name }; + }; + const regular = matchFace(0); + if (!draws(regular, 0)) { + measurers.set(family, null); + return null; + } + const faces = new Map([[0, regular]]); + const faceFor = (style) => { + if (!faces.has(style)) { + const face = matchFace(style); + faces.set(style, draws(face, style) ? face : regular); + } + return faces.get(style); + }; + const widths = new Map(); + const bars = new Map(); + const measurer = { + width(text, size, style = 0) { + const key = `${size}|${style}|${text}`; + if (!widths.has(key)) { + const face = faceFor(style); + const bar = `${size}|${style}`; + if (!bars.has(bar)) bars.set(bar, ink(face, "||", size, style)); + widths.set(key, text ? ink(face, `|${text}|`, size, style) - bars.get(bar) : 0); + } + return widths.get(key); + }, + }; + measurers.set(family, measurer); + return measurer; +} + +const STATE_PAD = 16; +const DEFAULT_WRAP = 200; +const labelLines = (src) => + [...src.matchAll(/^\s*state "([^"]+)" as|^[^%\n]*-->[^:\n]*:\s*([^\n]+)$/gm)] + .flatMap((m) => (m[1] ?? m[2]).split(//)) + .map((l) => l.trim()) + .filter((l) => l && !/["\\]/.test(l)); + +function widthRatio(render, src, measurer) { + const lines = [...new Set(labelLines(src))]; + if (!lines.length) return 1; + const probe = `---\nconfig:\n state:\n wrappingWidth: 2000\n---\nstateDiagram-v2\n${lines.map((l, i) => ` state "${l}" as p${i}`).join("\n")}\n`; + const builtin = new Map( + [...render(probe).svg.matchAll(/data-id="p(\d+)"[^>]*>]*\swidth="([\d.]+)"/g)].map( + ([, i, w]) => [lines[+i], +w - 2 * STATE_PAD], + ), + ); + const sum = (f) => lines.reduce((s, l) => s + f(l), 0); + return ( + sum((l) => measurer.width(l, 16, 0)) / sum((l) => builtin.get(l) ?? measurer.width(l, 16, 0)) + ); +} + +function setWrap(src, width) { + const fm = /^---\n([\s\S]*?\n)---\n/.exec(src); + if (!fm) return `---\nconfig:\n state:\n wrappingWidth: ${width}\n---\n${src}`; + let body = fm[1]; + if (/^ {4}wrappingWidth:.*$/m.test(body)) + body = body.replace(/^ {4}wrappingWidth:.*$/m, ` wrappingWidth: ${width}`); + else if (/^ {2}state:\n/m.test(body)) + body = body.replace(/^( {2}state:\n)/m, `$1 wrappingWidth: ${width}\n`); + else if (/^config:\n/m.test(body)) + body = body.replace(/^(config:\n)/m, `$1 state:\n wrappingWidth: ${width}\n`); + else body += `config:\n state:\n wrappingWidth: ${width}\n`; + return `---\n${body}---\n${src.slice(fm[0].length)}`; +} +const wrapOf = (src) => + +(/^---\n[\s\S]*?^ {4}wrappingWidth:\s*(\d+)[\s\S]*?^---$/m.exec(src)?.[1] ?? DEFAULT_WRAP); + +const toHex = (c) => `#${c.map((v) => Math.round(v).toString(16).padStart(2, "0")).join("")}`; + +function splitTop(s) { + const out = []; + let depth = 0; + let cur = ""; + for (const ch of s) { + if (ch === "(") depth++; + if (ch === ")") depth--; + if (ch === "," && depth === 0) { + out.push(cur); + cur = ""; + } else cur += ch; + } + out.push(cur); + return out.map((x) => x.trim()); +} + +function resolveCss(svg) { + const defs = new Map(); + for (const m of svg.matchAll(/(--[\w-]+)\s*:\s*([^;"}]+)/g)) + if (!defs.has(m[1])) defs.set(m[1], m[2].trim()); + const resolve = (value) => { + let changed = true; + while (changed) { + changed = false; + value = value.replace( + /var\((--[\w-]+)\s*(?:,\s*((?:[^()]|\([^()]*\))*))?\)/g, + (_, name, fallback) => { + changed = true; + const d = defs.get(name); + if (d !== undefined && d !== `var(${name})`) return resolve(d); + return fallback !== undefined ? fallback : "currentColor"; + }, + ); + value = value.replace(/color-mix\(\s*in srgb\s*,((?:[^()]|\([^()]*\))*)\)/g, (_, args) => { + changed = true; + const [a, b] = splitTop(args); + const pa = /(.*?)\s+([\d.]+)%$/.exec(a); + const ca = hex6(pa ? pa[1] : a); + const cb = hex6(b.replace(/\s+[\d.]+%$/, "")); + const p = pa ? Number(pa[2]) / 100 : 0.5; + if (!ca || !cb) return pa ? pa[1] : a; + const [x, y] = [rgb(ca), rgb(cb)]; + return toHex(x.map((v, i) => v * p + y[i] * (1 - p))); + }); + } + return value; + }; + for (const [k, v] of defs) defs.set(k, resolve(v)); + return resolve(svg.replace(/@import[^;]+;/g, "")); +} + +function themeSvg(svg, theme) { + const vars = + `--pele-bg:${theme.background};--pele-fg:${theme.region.text};--pele-muted:${theme.edgeLabel};--pele-line:${theme.edge};` + + `--pele-surface:${theme.stateFill};--pele-surface-alt:${theme.region.fill};--pele-border:${theme.region.border};`; + return svg + .replace(/(]*?\sstyle=")/, `$1${vars}`) + .replace( + /(]*>]*\/>.pele-edges,.pele-dividers{stroke-width:${STYLE.line}px}` + + `.pele-node rect,.pele-node path,.pele-node circle,.pele-cluster rect,.pele-cluster path,.pele-note rect,.pele-note path{stroke-width:${STYLE.line}px}`; +const round = (v) => +v.toFixed(2); + +function heavier(svg) { + svg = resolveCss(svg) + .replace(/style="([^"]*)"/g, (_, css) => `style="${css.replace(/--[\w-]+:[^;]*;?/g, "")}"`) + .replace(/font:\s*([\d.]+px)\s+([^;"]+);/, "font-size:$1;font-family:$2;"); + const rootFill = /]*\sfill="([^"]+)"/.exec(svg)?.[1] ?? "#1f1f1f"; + return svg + .replace(/(]*>)/, `$1${LINES}`) + .replace(/]*)>/g, (_, attrs) => { + const fill = + /style="[^"]*fill:\s*([^;"]+)/.exec(attrs)?.[1] ?? + /\sfill="([^"]+)"/.exec(attrs)?.[1] ?? + rootFill; + return ``; + }) + .replace( + /(class="pele-marker" d=")M([\d.-]+),([\d.-]+)((?:L[\d.-]+,[\d.-]+)+)Z/g, + (_, head, tx, ty, rest) => { + const [x0, y0] = [+tx, +ty]; + const points = [...rest.matchAll(/L([\d.-]+),([\d.-]+)/g)].map( + ([, x, y]) => + `L${round(x0 + (x - x0) * STYLE.arrow)},${round(y0 + (y - y0) * STYLE.arrow)}`, + ); + return `${head}M${tx},${ty}${points.join("")}Z`; + }, + ); +} + +function maskBands(svg) { + const pad = STYLE.line / 2; + // The holes set their own fill because mask content inherits the root fill, which is not black. + const holes = [ + ...svg.matchAll( + /]*>]*\/>`, + ) + .join(""); + if (!holes) return svg; + return svg.replace( + /${holes}$& mask="url(#pele-bands)"`, + ); +} + +export function renderChart(lib, src, { theme = THEMES.light, systemFont = systemFamily() } = {}) { + let text = withRootDirection(themed(src, theme)); + const measurer = systemFont ? fontMeasurer(lib.Resvg, systemFont) : undefined; + let wrap = wrapOf(text); + if (measurer) { + wrap = Math.ceil(wrap * Math.max(1, widthRatio(lib.render, text, measurer))); + text = setWrap(text, wrap); + } + const r = lib.render(text, { fontFamily: FONT, measurer }); + return { + svg: heavier(maskBands(themeSvg(r.svg, theme))), + width: r.width, + height: r.height, + wrap, + }; +} + +export function toPng( + lib, + { svg, width, height }, + { theme = THEMES.light, systemFont = systemFamily() } = {}, +) { + return new lib.Resvg(svg.split(FONT).join(pngStack(systemFont)), { + background: theme.background, + fitTo: { mode: "zoom", value: Math.min(STYLE.zoom, Math.sqrt(MAX_PIXELS / (width * height))) }, + font: systemFont + ? { loadSystemFonts: true, defaultFontFamily: systemFont, sansSerifFamily: systemFont } + : { loadSystemFonts: true }, + }) + .render() + .asPng(); +} + +export function renderPng(lib, src, options = {}) { + try { + return toPng(lib, renderChart(lib, src, options), options); + } catch (e) { + if (e.code !== "limit") throw e; + throw new Error( + `${e.message} The chart is too big for one PNG, and the .mmd file still has all of it. A chart with fewer states fits. Run node statechart.mjs from-spec SPEC.tla --cover --png to draw a smaller overview of the spec.`, + ); + } +} diff --git a/pstack/skills/statechart/scripts/rule-book.mjs b/pstack/skills/statechart/scripts/rule-book.mjs new file mode 100644 index 000000000..3ea545228 --- /dev/null +++ b/pstack/skills/statechart/scripts/rule-book.mjs @@ -0,0 +1,84 @@ +import fs from "node:fs"; +import path from "node:path"; +import { + blankStrings, + definitions, + ruleSentences, + splitCommentsKeepingOffsets, + tlaVariables, +} from "./lib.mjs"; + +const EXTENDS_STATEMENT = /\bEXTENDS\s+(\w+(?:\s*,\s*\w+)*)/g; +const CONSTANT_STATEMENT = /\bCONSTANTS?\s+(\w+(?:\([^)]*\))?(?:\s*,\s*\w+(?:\([^)]*\))?)*)/g; +const DECLARED_NAME = /(\w+)(?:\([^)]*\))?/g; +const LOCAL_DEFINITION = /^LOCAL\s/; +const INSTANCE_STATEMENT = + /^(LOCAL[ \t]+)?(?:([A-Za-z_]\w*)[ \t]*==\s*)?INSTANCE\s+(\w+)((?:[^\n]|\n(?=\s))*)/gm; +const SUBSTITUTION = /(\w+)\s*<-([\s\S]*?)(?=\w+\s*<-|$)/g; +const ALIAS = /^\s*([A-Za-z_]\w*(?:![A-Za-z_]\w*)*)\s*$/; + +export function ruleBook(spec) { + const modules = new Map(); + const moduleOf = (file) => { + if (modules.has(file)) return modules.get(file); + const module = { rules: new Map(), parameters: [] }; + modules.set(file, module); + const text = fs.readFileSync(file, "utf8"); + const code = blankStrings(splitCommentsKeepingOffsets(text).code); + const fileOf = (name) => path.join(path.dirname(file), `${name}.tla`); + const inherited = new Map(); + const instancedReads = new Map(); + const localImports = new Set(); + const take = (imported, { name, local, substitutions = new Map() }) => { + for (const [rule, entry] of moduleOf(imported).rules) { + if (entry.local) continue; + const key = name ? `${name}!${rule}` : rule; + if (local) localImports.add(key); + if (entry.sentence) inherited.set(key, entry.sentence); + instancedReads.set( + key, + [...entry.reads].map((v) => substitutions.get(v) ?? v), + ); + } + }; + const parameters = []; + for (const [, names] of code.matchAll(EXTENDS_STATEMENT)) + for (const extended of names + .split(/\s*,\s*/) + .map(fileOf) + .filter(fs.existsSync)) { + parameters.push(...moduleOf(extended).parameters); + take(extended, {}); + } + for (const [, names] of code.matchAll(CONSTANT_STATEMENT)) + parameters.push(...[...names.matchAll(DECLARED_NAME)].map(([, constant]) => constant)); + for (const [, local, name, instanced, clause] of code.matchAll(INSTANCE_STATEMENT)) + if (fs.existsSync(fileOf(instanced))) + take(fileOf(instanced), { + name, + local, + substitutions: new Map([...clause.matchAll(SUBSTITUTION)].map(([, v, e]) => [v, e])), + }); + const written = ruleSentences(text); + const { bare, defs, own, through } = definitions(text, instancedReads, parameters); + module.parameters.push(...parameters, ...tlaVariables(bare)); + const sentenceOf = (rule, seen = new Set()) => { + if (written.has(rule)) return written.get(rule); + if (seen.has(rule)) return undefined; + const d = defs.get(rule); + if (!d) return inherited.get(rule); + const alias = ALIAS.exec(bare.slice(d.from, d.to))?.[1]; + return alias ? sentenceOf(alias, seen.add(rule)) : undefined; + }; + for (const [rule, entry] of own) { + const d = defs.get(rule); + module.rules.set(rule, { + sentence: sentenceOf(rule), + reads: through(entry).reads, + local: d ? LOCAL_DEFINITION.test(bare.slice(d.at, d.from)) : localImports.has(rule), + }); + } + return module; + }; + return moduleOf(path.resolve(spec)).rules; +} diff --git a/pstack/skills/statechart/scripts/second-replay.mjs b/pstack/skills/statechart/scripts/second-replay.mjs new file mode 100644 index 000000000..1dba571c2 --- /dev/null +++ b/pstack/skills/statechart/scripts/second-replay.mjs @@ -0,0 +1,122 @@ +import { isRecord, stateKey } from "./lib.mjs"; +import { SEP, edgeId } from "./chart-parts.mjs"; +import { badOf } from "./bug-states.mjs"; + +const holder = (o, p) => p.slice(0, -1).reduce((x, k) => (isRecord(x?.[k]) ? x[k] : null), o); +function omit(vars, paths) { + if (!paths.length) return vars; + const out = structuredClone(vars); + for (const p of paths) { + const o = holder(out, p); + if (o) delete o[p.at(-1)]; + } + return out; +} + +export function replaySecond( + trace, + { main, lacks, extra, second, hiddenStep, readerPath, mainActions, secondRules, rawSecondTraces }, +) { + if (!second) return null; + const shared = (vars) => stateKey(omit(vars, lacks)); + const sharedOf = (n) => stateKey(omit(n.vars, extra)); + const out = new Map(); + for (const e of [...second.graph.edges, ...second.graph.loops]) { + if (!out.has(e.from)) out.set(e.from, []); + out.get(e.from).push(e); + } + const actions = new Set([...second.graph.edges, ...second.graph.loops].map((e) => e.action)); + const nodeOf = (id) => second.graph.nodes.get(id); + const screen = new Map( + [...second.graph.nodes.values()].map((n) => [n.id, second.customer(n.vars)]), + ); + const words = (e) => + second.step(e.action, nodeOf(e.from).vars, nodeOf(e.to).vars, "customer") || e.action; + const settle = (ids) => { + const all = new Set(ids); + for (const id of all) + for (const e of out.get(id) ?? []) { + const sameLabel = screen.get(e.to) === screen.get(id); + if ( + !mainActions.has(e.action) && + hiddenStep(trace.invariant, sameLabel, e.action, nodeOf(id).vars, nodeOf(e.to).vars) + ) + all.add(e.to); + } + return all; + }; + const seen = readerPath(trace, main, main.customer, "customer"); + const shownAt = new Map(seen.map((p, j) => [p.modelStep, j])); + let here = settle( + [...second.graph.nodes.values()] + .filter((n) => n.init && screen.get(n.id) === seen[0].key) + .map((n) => n.id), + ); + if (here.size === 0) return { reason: "starts in another state" }; + const passed = new Set(here); + let j = 0; + for (let i = 1; i < trace.steps.length; i++) { + const { action } = trace.steps[i]; + const hidden = !shownAt.has(i); + if (!hidden) j = shownAt.get(i); + const want = seen[j]; + const [before, after] = [trace.steps[i - 1].vars, trace.steps[i].vars]; + const known = actions.has(action); + if (!known && hidden) continue; + const moved = shared(before) !== shared(after); + const steps = (keep) => [...here].flatMap((id) => (out.get(id) ?? []).filter(keep)); + const tries = steps((e) => + known ? e.action === action : moved && sharedOf(nodeOf(e.to)) === shared(after), + ); + if (hidden && tries.length === 0) continue; + const looksSame = (e) => screen.get(e.to) === want.key && (hidden || words(e) === want.label); + const matched = known ? tries.filter(looksSame) : tries; + const same = matched.length || hidden ? matched : steps(looksSame); + if (same.length) { + here = settle(same.map((e) => e.to)); + for (const id of here) passed.add(id); + continue; + } + const outcomes = new Map(); + for (const e of tries) { + const id = `${screen.get(e.to)}${SEP}${words(e)}`; + outcomes.set(id, [...(outcomes.get(id) ?? []), e]); + } + const taken = [...outcomes.values()].sort((a, b) => b.length - a.length)[0]?.[0]; + const from = main.key(before); + const said = hidden ? main.step(action, before, after, "customer") || action : want.label; + const at = { modelStep: i, said, from, hidden }; + if (!taken) + return { + ...at, + key: from, + label: main.step(action, before, after) || action, + disabled: true, + }; + const [b, a] = [nodeOf(taken.from).vars, nodeOf(taken.to).vars]; + return { + ...at, + key: second.key(a), + label: second.step(taken.action, b, a) || taken.action, + own: words(taken), + }; + } + const bad = badOf(rawSecondTraces, trace.invariant); + const checked = secondRules.has(trace.invariant); + const ruleHolds = checked && ![...passed].some((id) => bad.has(stateKey(nodeOf(id).vars))); + return { sameEnd: true, checked, ruleHolds }; +} + +export function divergence(s, path, n, { inSecond, modelEdges }) { + if (s?.key === undefined) return s; + const step = path.findLastIndex((p) => p.modelStep < s.modelStep) + (s.hidden ? 0 : 1); + const plain = edgeId(s.from, s.key); + if (s.disabled) { + const text = inSecond(`${s.label} cannot happen at step ${step}`); + const edge = `${plain}${SEP}cannot${SEP}second`; + return { ...s, step, text, overview: `${text} of bug ${n}`, edge, apart: true }; + } + const text = inSecond(s.label); + const drawn = [s.label, text].some((l) => modelEdges.get(plain)?.labels.has(l)); + return { ...s, step, text, overview: text, edge: drawn ? plain : `${plain}${SEP}second` }; +} diff --git a/pstack/skills/statechart/scripts/snapshot.mjs b/pstack/skills/statechart/scripts/snapshot.mjs new file mode 100644 index 000000000..ebaf35573 --- /dev/null +++ b/pstack/skills/statechart/scripts/snapshot.mjs @@ -0,0 +1,74 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import process from "node:process"; + +const imports = (text, dir) => + [...text.matchAll(/\b(?:from|import)\s*\(?\s*["'](\.{1,2}\/[^"']+)["']/g)].map((m) => + path.resolve(dir, m[1]), + ); + +const DEPENDENCIES = { + ".tla": (text, dir) => + [ + ...text + .replace(/\(\*[^]*?\*\)/g, " ") + .replace(/\\\*.*/g, " ") + .matchAll(/\b(?:EXTENDS|INSTANCE)\s+(\w+(?:\s*,\s*\w+)*)/g), + ].flatMap((m) => m[1].split(/\s*,\s*/).map((name) => path.join(dir, `${name}.tla`))), + ".mjs": imports, + ".js": imports, +}; + +export const copyPath = (root, file, paths = path) => { + const absolute = paths.resolve(file); + const pathRoot = paths.parse(absolute).root; + return paths.join(root, pathRoot.replace(/\W+/g, ""), paths.relative(pathRoot, absolute)); +}; + +const contentOf = (file) => { + try { + return fs.readFileSync(file, "base64"); + } catch { + return null; + } +}; + +export class Inputs { + #root = fs.mkdtempSync(path.join(os.tmpdir(), "statechart-inputs-")); + #files = new Set(); + + constructor() { + process.on("exit", () => fs.rmSync(this.#root, { recursive: true, force: true })); + } + + add(files) { + for (const file of files) if (file) this.#take(path.resolve(file), true); + return this; + } + + copyOf(file) { + return copyPath(this.#root, file); + } + + changed() { + return [...this.#files].find((file) => contentOf(file) !== contentOf(this.copyOf(file))); + } + + #take(file, listed) { + if (this.#files.has(file)) return; + let content; + try { + content = fs.readFileSync(file); + } catch { + if (listed) this.#files.add(file); + return; + } + this.#files.add(file); + const copy = this.copyOf(file); + fs.mkdirSync(path.dirname(copy), { recursive: true }); + fs.writeFileSync(copy, content); + const found = DEPENDENCIES[path.extname(file)]?.(content.toString("utf8"), path.dirname(file)); + for (const dependency of found ?? []) this.#take(dependency, false); + } +} diff --git a/pstack/skills/statechart/scripts/spec-text.mjs b/pstack/skills/statechart/scripts/spec-text.mjs new file mode 100644 index 000000000..ddfc8d10c --- /dev/null +++ b/pstack/skills/statechart/scripts/spec-text.mjs @@ -0,0 +1,126 @@ +import fs from "node:fs"; +import path from "node:path"; +import { BROKEN_MARK, PRIMES_OR_UNCHANGED, tlaVariables } from "./lib.mjs"; + +export const withoutComments = (spec) => + fs + .readFileSync(spec, "utf8") + .replace(/\(\*[^]*?\*\)/g, " ") + .replace(/\\\*.*/g, " "); + +function specVariables(spec, seen) { + const text = withoutComments(spec); + const extended = /\bEXTENDS\s+(\w+(?:\s*,\s*\w+)*)/.exec(text)?.[1].split(/\s*,\s*/) ?? []; + return [ + ...extended.flatMap((m) => { + const file = path.join(path.dirname(spec), `${m}.tla`); + if (seen.has(m) || !fs.existsSync(file)) return []; + seen.add(m); + return specVariables(file, seen); + }), + ...tlaVariables(text), + ]; +} + +export const RULES_INVARIANT = "StatechartRules"; + +export function writeRulesModule(spec, rules, dir) { + const name = path.basename(spec, ".tla"); + const file = path.join(dir, `${name}_rules.tla`); + const fields = specVariables(spec, new Set([name])).map((v) => `${v} |-> ${v}`); + const broken = rules.map((r) => `(IF ${r} THEN {} ELSE {"${r}"})`).join(" \\cup "); + fs.writeFileSync( + file, + `---- MODULE ${name}_rules ---- +EXTENDS ${name} +StatechartTLC == INSTANCE TLC +StatechartBroken == ${broken} +${RULES_INVARIANT} == StatechartBroken = {} \\/ StatechartTLC!PrintT(<<"${BROKEN_MARK}", [${fields.join(", ")}], StatechartBroken>>) +==== +`, + ); + return file; +} + +export const SCAFFOLD = (name, spec) => { + const text = withoutComments(spec); + const vars = tlaVariables(text); + const primedDefinitions = [ + ...new Set( + text + .split(/\n(?=\S)/) + .map((d) => /^(\w+)\s*(?:\([^)]*\))?\s*==([^]*)/.exec(d)) + .filter((d) => d && PRIMES_OR_UNCHANGED.test(d[2])) + .map((d) => d[1]), + ), + ].sort(); + return `// Chart labels for ${name}, for statechart.mjs. Write labels in the words of the person who reads +// the chart, not in variable or action names. +// v, before, and after map each variable to its value. Strings, numbers, and booleans keep their kind. +// Sequences and tuples, including functions over 1..n, read from 1 as in TLA+, so v.seat[1] is the +// first item. Reading index 0 stops the run. length, array methods, and spread still work. A +// method's index, such as indexOf or the i in map((x, i) => ...), still counts from 0. +// Sets become arrays. Records and other functions become objects. +// Variables: ${vars.join(", ")}. + +// The actions that the reader takes, by action name, when the chart separates what the reader does +// from what happens around the reader. facts.settled is true when only these are enabled. With +// --cover, the overview keeps only the shortest path to each settled state, plus every bug path. +// Leave it empty when every action counts the same. +export const readerActions = []; + +// The label for one model state. facts.enabled is the set of actions the state enables, +// including steps that change no variable. A label "context: progress", such as +// "Primary down: write queued", puts the states of one context in one box, and can draw the +// overview as two parallel regions. Use that form only when the context and the progress change +// independently. A label without ": " draws each state on its own. +// Do not claim a result that a pending step can still change, such as "write lost" while a +// retry is still queued. +export function state(v, facts) { + return JSON.stringify(v); +} + +// The names of the two parallel regions, when the overview draws them: contexts, then progress. +// export const regionNames = ["Primary", "Write"]; + +// The group of one model state, in the reader's words, such as "Order shipped". The chart fills +// the states of one group with one color, and the legend names the groups. TLC still decides +// which states break a rule, and draws them with a thick red border on top of the group color. +// groupColors gives a group a light color. Groups without one take a palette color. +// export function group(v, facts) {} +// export const groupColors = { "Order shipped": "#d1f2eb" }; + +// The label for one step. A step that leaves the reader in the same state is not drawn, +// unless the reader takes it on a bug path. "" draws no label, and a numbered bug step shows the action. +// action is the action's name without its parameters, so compare before and after to tell which +// process or item the step changed. +// STEPS lists every definition that changes a variable. TLC names no step after one that another +// action uses in its /\\ list, such as Log in Save == Log /\\ saved' = TRUE, so delete its entry. +const STEPS = { +${primedDefinitions.map((a) => `\t${a}: ${JSON.stringify(a)},`).join("\n")} +}; +export function step(action, before, after) { + return STEPS[action] ?? action; +} + +// When the step can happen, in plain words, such as "after the lease expires". The chart shows +// "step [words]". "" adds nothing. +// export function guard(action, before, after) {} + +// For a timer step, how many seconds the code waits, from the constant in the code. The chart +// shows "after N s, step". Return nothing for other steps. +// export function timer(action, before, after) {} + +// Why a bad state is bad, in the reader's words, such as "a worker crashed after it took the job, +// and no other worker can take it". No variable, function, or event names. +// Counterexamples that break one rule with one cause are one bug, so two bugs that end in the +// same state stay apart only if their causes differ. Return null to group by the end state. +// export function cause(v, facts, invariant) {} + +// The developer's words for one model state. With it, each bug is also drawn through these states. +// export function detail(v, facts) {} + +// The legend's words for each status, in place of the defaults. Group entries keep their names. +// export const legend = { works: "Works on main", wrong: "Wrong on main", fixed: "Fixed in the open PR" }; +`; +}; diff --git a/pstack/skills/statechart/scripts/statechart-help.mjs b/pstack/skills/statechart/scripts/statechart-help.mjs new file mode 100644 index 000000000..1a900fc51 --- /dev/null +++ b/pstack/skills/statechart/scripts/statechart-help.mjs @@ -0,0 +1,350 @@ +import { CACHE_NAME, PACKAGES } from "./render.mjs"; + +export const USAGE = { + "from-spec": + "node statechart.mjs from-spec SPEC.tla [--config FILE] [--fixed FIXED.tla [--fixed-config FILE] | --against AGREED.tla [--against-config FILE]] [--label TEXT] [--map FILE] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--cover] [--png [--theme NAME]]", + "from-chart": + "node statechart.mjs from-chart CHART.mmd [--module NAME] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png [--theme NAME]]", + render: "node statechart.mjs render CHART.mmd [--theme NAME] [--out FILE]", +}; +export const usage = (lines) => `Usage: ${lines.join("\n ")}`; + +const FROM_SPEC = `from-spec SPEC.tla + Checks each INVARIANT in SPEC.cfg in a separate TLC run. TLC stops at the first rule that a state + breaks, and the separate runs find the rest. Each run also has the rule as a CONSTRAINT, so TLC + stops each path at its first state that breaks the rule, and a rule that stays broken reports + only the step that broke it. SPEC.cfg has SPECIFICATION and one INVARIANT line per rule. Each + PROPERTY, a rule about whole behaviors, gets one TLC run too, and the charts do not draw it. TLC + checks a PROPERTY with no [], <>, or ~> only in the initial states, and the command says so. A + rule listed as both INVARIANT and PROPERTY is checked once, as an INVARIANT. After those runs + print their results, one more run dumps the state graph and checks every INVARIANT on every + state in it, and then the command draws the charts. TLC goes down from the next-state action + through each \\/, \\E, LET, and definition, and names each step after the last definition on that + path. It stops at any other expression, such as /\\, IF, or CASE, so a definition inside one names + no step. Give each kind of step its own definition, and join them with \\/ in the next-state + action. The command warns when steps carry the name of the whole next-state action. The command + prints the config that it checks each model with, and stops when a config does not exist, with + the configs beside the spec. + + --config FILE The config for SPEC.tla. Default: SPEC.cfg next to the spec. + --fixed FIXED.tla The model of the open fix. It is a copy of the spec in a different file, such + as SpecFixed.tla or fix/Spec.tla. The command checks it the same way, with + FIXED.cfg, or with the config of SPEC.tla when there is no FIXED.cfg. The + charts then mark each bad state fixed or still wrong. A rule that the + config of FIXED.tla does not check stays as wrong as it is today. When the + fix shows the same states and steps as a bug of such a rule, the story says + "does not check this rule". + --fixed-config FILE + The config for FIXED.tla. + --against AGREED.tla + The spec that SPEC.tla must follow, when SPEC.tla models the code. It is a + different file from SPEC.tla. The command checks it with AGREED.cfg, and + never with the config of SPEC.tla. SPEC.tla may add variables, record fields, + and actions that AGREED.tla does not have. The comparison leaves them out. + Do not use it with --fixed. + --against-config FILE + The config for AGREED.tla, when it is not AGREED.cfg. Give the config that the + spec's authors say the unchanged spec passes. When the spec shows the same + states and steps as a bug of a rule that this config does not check, the + story says "does not check this rule". + --label TEXT The name of the --fixed or --against model in the charts and the stories. + Default: "the fix", or "the spec". + --map FILE The labels for each model state and step. Default: SPEC.map.mjs next to the + spec. When the file does not exist, the command writes one from the spec's + VARIABLES and actions to fill in, and exits 2 before it runs TLC. Its + comments say what each function gets. The map reads a sequence or tuple + from 1, as in TLA+, so v.seat[1] is its first item. A map that reads from + 0, such as v.pc[r - 1], gets the item before the one it wants, and stops + the run when it reads index 0. + --cover Draw only the shortest path from the start to each settled state, plus + every bug path, in the overview. A state is settled when only the + mapping's readerActions are enabled. The command prints the states and + arrows that the overview leaves out. The counts of states that work or + break a rule still include the states it leaves out. When no step in + TLC's graph is a reader action, the overview draws every state. + + With --fixed or --against, SPEC.tla (and FIXED.tla) models the code, and the command finds each + condition of an action that no comment ties to code with a file and line, as in \\* pay.ts:120. + A condition is a conjunct of the action that changes nothing and reads a variable that the + action does not change. A rule can hold only because the model waits for a condition that the + code does not check, so the command prints these conditions when a rule holds. + With --against, each story says whether a config beside AGREED.tla expects TLC to break the + bug's rule. A config's "\\* expect: RULE" line says what TLC finds, and "\\* module: NAME" names + the module it checks when the folder has more than one .tla file. The command warns when no + config beside AGREED.tla has an expect line for that module.`; + +const FROM_CHART = `from-chart CHART.mmd + Writes NAME.tla, NAME.cfg, and NAME.map.mjs next to the chart. When NAME.tla already exists, it + writes none of them. Otherwise it replaces a NAME.cfg or NAME.map.mjs that is there. Then it + checks the spec as from-spec does, and compares the chart that the spec draws with CHART.mmd. + + --module NAME The TLA+ module name. Default: the chart's file name in CamelCase, such as + NotesApp for notes-app.mmd. + + The chart is a Mermaid stateDiagram-v2. Each transition label is one action. For parallel + regions, split one top-level composite with --, and put each part in a named composite, such as + state Connection { ... }. Each region is one variable. A step moves every region whose current + state has a transition with the step's label, and the other regions stay where they are. Each + rule is one comment line: + %% never NAME: A & B -- the rule in one sentence + Each name after the colon is a state caption or a composite, and each comes from a different + region. A composite stands for every state in it. A "%% when ID, ID" or "%% when not ID, ID" line + right after a transition is its guard, and the transition's label must end with the guard's + words in [ ]. The region takes the transition only while each other region that the line names + is in one of the states that it names, or with not, while none is. A transition with no label is + its own action. NAME.cfg has SPECIFICATION Spec, INVARIANT TypeOK, and one INVARIANT per never + line. The command refuses final, choice, fork, and history states, and regions below the top + level. + + The comparison looks at reachable states, start states, and labeled steps. A chart with regions + compares as its combined machine, with each state named by its region states joined by " | ", so + a guard that blocks a step in only some combinations of regions shows as a difference. The + guard's words in [ ] are not part of the label. A loop compares like any other step. An edge + label "a or b" counts as two labels. An edge with no label counts only when no labeled edge joins + the same two states.`; + +const RENDER = `render CHART.mmd + Draws one chart as a PNG. It does not run TLC, so the chart can come from from-spec, from + from-chart, or from a person. + + --theme NAME The colors of the PNG: light, dark, or classic. Default: light. + --out FILE The PNG to write. Default: the chart's path with .png in place of .mmd. The + command makes the folder when it does not exist. + + Exit codes + 0 The command wrote the PNG. + 2 The command cannot finish: a bad argument, a chart that it cannot read, a renderer that it + cannot install or load, or a PNG that it cannot render or write. It writes no PNG then.`; + +const PNG = `PNG + Pele lays out each chart, and resvg draws it at twice its size. A chart that is more than 100 + million pixels at that size draws smaller, at about 100 million pixels. The first PNG installs + ${PACKAGES.join(" and ")} with npm into a cache folder. The folder is + $XDG_CACHE_HOME/statechart/${CACHE_NAME}, or + ~/.cache/statechart/${CACHE_NAME} when XDG_CACHE_HOME is not set. Delete that folder to + install them again. When npm or the packages fail, the command prints why, exits 2, and writes + no file. + The PNG uses the font that fc-match finds for system-ui. The command measures each label in that + font, so no label goes past its box. When that font is wider than the widths that Pele has built + in, the command makes the wrap width wider by the same amount, so the lines break where the chart + breaks them. Without fc-match, or when resvg cannot read that font, the PNG uses Pele's widths. + Pele puts parallel regions side by side. When a chart with regions has no top-level direction, + the PNG puts the legend above the regions, so the PNG is narrower. + --theme picks light, dark, or classic. light is the default. classic keeps the group fills from + the chart's own classDef lines.`; + +const COMMON = `Both commands + --expect-broken RULE + An INVARIANT or PROPERTY that is known to break. Repeat it, or list rules + with commas, as in A,B. The command then exits 0 when exactly these rules + break, and the --fixed or --against model breaks no other rule. + --title TEXT The chart title. Each bug chart adds ", bug N". + --out DIR The output folder. Default: NAME-statechart next to the input. A run writes its + files in a temporary folder, and then replaces the whole output folder with + it. When runs overlap, the output folder has the files of the run that + finished last. The command does not use an output folder that has files that + it did not write. On a filesystem that cannot rename, the command copies the + files into place, and overlapping runs can mix their files. + --png Also draw each chart as a PNG. The PNG section tells how. + --theme NAME The colors of each PNG: light, dark, or classic. Default: light. It needs + --png. + -h, --help Print this help, and exit 0. + +Output + The command prints whether each rule holds, the chart's counts, and one story per bug. + from-chart also prints the comparison. The command writes these files in the output folder: + chart.mmd The overview. + bug-N.mmd Only the steps of bug N, numbered. Bugs are numbered by fewest reader steps, + then fewest model steps, then rule name, then cause. Bugs that only the + --fixed or --against model has come last. + bug-N-developer.mmd, bug-N-layered.mmd + The path of bug N through the developer's states, when the mapping exports + detail(). The layered chart puts each detail state in a box named by its + state label. A state with a blank detail stays a plain state. + stories.md One entry per bug: the rule, why it breaks, the steps that lead into it, the + shortest path in reader steps, and what the --fixed or --against model does + instead. The rule's sentence is the \\* comment lines above the rule in the + spec, or in a module in its folder that the spec EXTENDS, directly or + through another module. With --fixed or --against, the other spec and its + modules count too, after SPEC.tla and its modules. The entry numbers each + step that changes the state label, each reader action, each step that + changes a variable that the rule reads, and the step into the bug. Between + them, it counts the hidden steps and names each different step once. Above + the steps, a line names up to five different steps that lead into the bug, + most common first, and counts the rest. When more than one step or start + state leads into the bug, the entry says that the path is the shortest that + TLC found. A step leads into the bug when it goes there from a state that + keeps the bug's own rule. That state can already break other rules. "(the + state stays the same)" marks a step on that line that keeps the state's + label and is not on the bug's path. from-spec does not draw such a step. + When TLC names the steps only after the whole next-state action, the entry + cannot count them. Then one line per broken PROPERTY with its sentence, + which comes only from its own spec file, the reader states that repeat + forever, if any, and the TLC output that has its counterexample. + summary.json The counts behind the chart. Each bug has counterexamples, how many model + states end it, each with its own TLC trace; lastSteps, the steps that lead + into it, most common first; and startsHere, whether a start state is in it. + brokenProperties has each broken PROPERTY, its rule sentence, the reader + states that repeat forever as repeats, and its TLC output. mixedStatus + lists the states that break a rule in some cases. chartGroups counts the + drawn states of each group, and mixedGroups lists the states in more than + one group. With --cover, cover lists the states and arrows that the + overview leaves out. With --fixed or --against, uncited has the conditions + with no file and line for each code model. With --against, second.listed + has what each config beside AGREED.tla expects, and each bug has the + configs that list it. + tlc/ The config and the TLC output of each run. When the overview has regions, + from-spec also writes tlc/regions-all.mmd, the regions with every arrow, and + tlc/regions-proof.mmd, every state and step of TLC's graph in one region. + compare-charts.mjs tlc/regions-all.mmd tlc/regions-proof.mmd repeats the + proof. It prints "same" and exits 0, or prints each difference and exits 1. + *.png Each chart as a PNG, with --png. A run without --png deletes them. + +Reading a chart + from-spec draws the overview as two parallel regions when every state reads "context: progress": + the contexts on one side, the progress on the other, both at the same time. It does so only when + that has fewer states, when the regions combine back into exactly every state and step of TLC's + graph, and when each state's color is the worse color of its two region states. Otherwise it puts + the states of one context in a box named context, and prints why. Each state in a box shows only + its progress, and a context with one state stays a plain state. + Regions draw only the arrows of the shortest way to each state, of each bug path, and of the + nearest step with each label. The script checks that these arrows make no step that TLC's graph + does not have. At a step that they make wrongly, it puts back every arrow that the full regions + take there. + An arrow from a box's border leaves from every state in the box. The chart draws it only when + every state in the box takes the step with the same labels to the same state, and no state + outside the box has the box's name. + In regions, a step leaves every region that has it. "[when A or B]" or "[unless A or B]", + whichever is shorter, says which states of the other region the step needs, and a "%% when" or + "%% when not" line under it lists their ids. A loop says the region may also stay. + Other words in [ ] come from the mapping's guard(), and "after N s" from its timer(). An edge + shows guard() words only when every model step behind it has them. In regions, they replace + "when A or B" and "unless A or B". TLC does not check them, and has no clock, so a reviewer + checks them against the code. + Gray states work. Red states break a rule. An orange state breaks a rule in some cases: a model + state under its label breaks a rule, and a path without a bug reaches another one. A white state + is reached only after a bug: every path from the start to its model states passes a state that + breaks a rule, and none of them breaks one. With --fixed, red, orange, and white describe the + fix, and green states are fixed by it or exist only with it. A purple state breaks a rule only + with the fix: today it works, is reached only after a bug, or does not exist. With --against, a + state takes the worse of its two models' colors, red, then orange, then white, and green states + exist only in the spec. + When the mapping exports group(), the fill is the state's group, which the legend names. A chart + state is in a group when all its model states are in it, and a region state when every state + that it is part of is in it. A state in no group, or in more than one, is gray. The legend lists + the groups that the chart draws: first the groups in groupColors, in that order, then the others + from A to Z, and then each status border that the chart uses. groupColors gives a group a light + "#rgb" or "#rrggbb" color. A group with no color takes the next free palette color: light blue, + orange, purple, yellow, teal, tan, then pink. A group cannot have the name of a status, such as + "Breaks a rule". Without group(), the legend shows only the statuses that the chart uses. In the + .mmd file, the status is a thick border of its color, gray for a white state, on top of the group + fill. In a PNG, a red, orange, purple, or white state is a box of that color with a border and + text of its color in place of its group fill, and a fixed state is a green box, so each box shows + one signal. + A thick blue border and numbered labels mark the bug. With group(), and in a PNG, the blue border + marks only the states on the bug's path that work, and the numbered labels show the rest. + Mermaid state diagrams cannot color an edge. + Each bug replays on the other model by its state labels and step words, which are what the + reader sees. The replay takes the bug's own steps. Between them, it also takes the other model's + steps of actions that SPEC.tla's state graph does not have, when they keep the state label, are + not reader actions, and change no variable that the broken rule reads. A bug step that the story + hides must keep the state label on the other model too, unless that model cannot take it. A step + that the reader sees takes a step with the same words between the same two state labels, and + prefers a step of the same action. A hidden step of an action that the other model does not have + is skipped. A step that the reader sees of such an action takes any step to the same shared state + when it changes a shared variable, or else a step with the same words between the same two state + labels. + A label "with the fix, STEP" or "in the spec, STEP" is a step that only the other model takes. On + a bug's path it leaves the state before the first step that the other model shows the reader + differently, and ends where that model goes instead. The story quotes the other model's words + when they differ. "with the fix, STEP cannot happen at step N" or "in the spec, STEP cannot + happen at step N" loops on the state where that model stops the step, on an arrow of its own. On + the overview, the label ends with "of bug M". Outside regions, two states otherwise have at most + one arrow between them. Its lines are the numbered steps, then SPEC.tla's steps, then the other + model's steps, with a blank line between each group. + The other model may have other variables, record fields, and actions. The comparison leaves out a + variable or field that no state of one model has, in its state graph or its traces, and one whose + values never have the same type in the two models. One mapping draws both models, so it may read + only what both have. The command stops when the mapping reads, on a state of the other model, a + variable or field that no state of that model has, or one whose value has a type that SPEC.tla + never has there. It also stops when the mapping lists the fields of such a state, as a spread, + Object.keys, or JSON.stringify does. Test "name" in v before the mapping reads a field that the + other model lacks. The command also stops when a label, guard, group, or cause that the mapping + makes for the other model has the word "undefined" in it. + A chart with another model has the line "%% second model (fix|against): LABEL" right after + stateDiagram-v2. Each chart starts with front matter that keeps Mermaid from shrinking it to the + page or wrapping its labels again. + An edge lists every step that takes it, most common first. Words that the steps share at the + start are written once, as in "you tap Pay or Back". + +Regenerate + List every rule that is known to break with --expect-broken, the goals and the known bugs. Each + run then exits 0 until a rule changes, so one script rebuilds every chart and stops on any change: + set -e + node statechart.mjs from-spec Code.tla --against Agreed.tla --expect-broken NoLostUpload --png + A run that leaves a broken rule out of --expect-broken exits 1, and set -e stops the script there. + +TLC + TLA2TOOLS is the path to tla2tools.jar, and JAVA is the path to the java binary. With only + TLA2TOOLS set, the command runs java from PATH. Otherwise it uses TLC 1.7.4 and Java temurin-25 + from mise, or JAVA when it is set, and installs what is missing with mise. The first install + downloads about 305 MB and needs the network. It checks the checksum of tla2tools.jar. mise runs + in an empty temporary folder, so the mise files around the spec, in a repo, or above TMPDIR do + not apply, do not change, and need no trust. When mise cannot start or fails, the command prints + why, the install command, and how to set TLA2TOOLS and JAVA instead. + When the command starts, it copies its input files, which Exit codes lists. TLC checks the + copies, so an edit during the run does not change what TLC checks. + TLC checks each rule with one worker per CPU. When a rule is broken, TLC checks it again with one + worker, which finds the shortest bug paths, the same on every run. The command then prints how + many times TLC ran and how long that took. + TLC can find that a rule is broken and then stop on an error, such as a Java exception. The + command then keeps the paths that TLC printed, marks the rule broken, and warns that the charts + can miss states that break it. A smaller constant can let TLC finish. + +Exit codes + 0 Every rule holds, or exactly the --expect-broken rules break. from-chart also draws CHART.mmd + back unchanged. + 1 Another rule is broken, an expected rule holds, or the chart that the spec draws is not + CHART.mmd. The output says which. + 2 The command cannot finish: a bad argument, a missing file, a chart or TLC output that it + cannot read, no TLC, a helper script that cannot start, an output folder with files that the + command did not write, an input file whose content changed during the run, a new mapping to + fill in, a mapping that it cannot use, a renderer that --png cannot install or load, or a + chart that --png cannot render. + When a chart does not render, the command keeps the files it wrote and exits 2, whatever the + rules give. Otherwise, exit 2 does not change the output folder, unless the command moved it + aside and cannot put it back. Then the message names the folder's new path. When the command + stops after TLC has run, it keeps the files that it wrote, TLC's output among them, in a + temporary folder, and the message names it. When TLC's state graph is too large to draw, the + command prints whether each rule holds and exits 2, whatever the rules give. It stops TLC as + soon as the graph passes that size. The input files are the spec and config, the --fixed or + --against spec and config, the chart, and the map. Each .tla file in the same folder that an + input .tla file EXTENDS or INSTANCEs is an input file too, and so is each file that an input + .mjs or .js file imports by a relative path. Before --png renders, and again before it writes + the output folder, the command compares the content of each input file with its content at + the start. A save that does not change the content does not stop the run.`; + +export const HELP = { + "": `${usage([...Object.values(USAGE), "node statechart.mjs [from-spec|from-chart|render] --help"])} + +Draws a Mermaid statechart from TLC's state graph of a TLA+ spec. The spec can model any system, +such as a product flow, a protocol, a service, a job queue, or an algorithm. A mapping labels each +state and step in the words of the person who reads the chart, and the help calls that person the +reader. The reader can be a user, an operator, an engineer, or a reviewer. + +${FROM_SPEC} + +${FROM_CHART} + +${COMMON} + +${RENDER} + +${PNG}`, + "from-spec": `${usage([USAGE["from-spec"]])}\n\n${FROM_SPEC}\n\n${COMMON}\n\n${PNG}`, + "from-chart": `${usage([USAGE["from-chart"]])}\n\n${FROM_CHART}\n\n${COMMON}\n\n${PNG}`, + render: `${usage([USAGE.render])}\n\n${RENDER}\n\n${PNG}`, +}; + +export const more = (cmd) => + `Run node statechart.mjs ${cmd ? `${cmd} ` : ""}--help for the flags, the output, and the exit codes.`; diff --git a/pstack/skills/statechart/scripts/statechart.mjs b/pstack/skills/statechart/scripts/statechart.mjs index 0ff4255c2..555f10b8c 100644 --- a/pstack/skills/statechart/scripts/statechart.mjs +++ b/pstack/skills/statechart/scripts/statechart.mjs @@ -1,217 +1,29 @@ #!/usr/bin/env node -import { spawnSync } from "node:child_process"; +import { constants } from "node:buffer"; +import { spawn, spawnSync } from "node:child_process"; import fs from "node:fs"; import os from "node:os"; import path from "node:path"; import process from "node:process"; import { fileURLToPath, pathToFileURL } from "node:url"; +import { HELP, more, USAGE, usage } from "./statechart-help.mjs"; import { BROKEN_PROPERTY, - findTlc, + checkedOnlyInInitialStates, joinAnd, parseDot, - PRIMES_OR_UNCHANGED, - tlaVariables, tlcOutputWarning, uncitedConditions, } from "./lib.mjs"; +import { loopSentence, readerStateNamer, repeatsIn } from "./property-loop.mjs"; +import { loadRenderer, renderPng, THEMES } from "./render.mjs"; +import { ruleBook } from "./rule-book.mjs"; +import { Inputs } from "./snapshot.mjs"; +import { RULES_INVARIANT, SCAFFOLD, withoutComments, writeRulesModule } from "./spec-text.mjs"; +import { cannotRunTlc, findTlc } from "./tlc-tools.mjs"; const here = path.dirname(fileURLToPath(import.meta.url)); - -const USAGE = { - "from-spec": - "node statechart.mjs from-spec SPEC.tla [--config FILE] [--fixed FIXED.tla [--fixed-config FILE] | --against AGREED.tla [--against-config FILE]] [--label TEXT] [--map FILE] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png]", - "from-chart": - "node statechart.mjs from-chart CHART.mmd [--module NAME] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png]", -}; -const usage = (lines) => `Usage: ${lines.join("\n ")}`; - -const FROM_SPEC = `from-spec SPEC.tla - Checks each INVARIANT in SPEC.cfg in a separate TLC run. TLC stops at the first rule that a state - breaks, and the separate runs find the rest. Each run also has the rule as a CONSTRAINT, so TLC - stops each path at its first state that breaks the rule, and a rule that stays broken reports - only the step that broke it. SPEC.cfg has SPECIFICATION and one INVARIANT line per rule. Each - PROPERTY, a rule about whole behaviors, gets one TLC run too, and the charts do not draw it. A - rule listed as both INVARIANT and PROPERTY is checked once, as an INVARIANT. One - more run dumps the state graph, and then the command draws the charts. TLC names each step after - the innermost named action that takes it, so give each kind of step its own named action. The - command warns when steps carry the name of the whole next-state action. The command prints the - config that it checks each model with, and stops when a config does not exist, with the configs - beside the spec. - - --config FILE The config for SPEC.tla. Default: SPEC.cfg next to the spec. - --fixed FIXED.tla The model of the open fix. It is a copy of the spec with its own file and - module name, such as SpecFixed.tla with MODULE SpecFixed. The command checks - it the same way, with FIXED.cfg, or with the config of SPEC.tla when there is - no FIXED.cfg. The charts then mark each bad state fixed or still wrong. - --fixed-config FILE - The config for FIXED.tla. - --against AGREED.tla - The spec that SPEC.tla must follow, when SPEC.tla models the code. It needs - its own file and module name. The command checks it with AGREED.cfg, and - never with the config of SPEC.tla. SPEC.tla may add variables, record fields, - and actions that AGREED.tla does not have. The comparison leaves them out. - Do not use it with --fixed. - --against-config FILE - The config for AGREED.tla, when it is not AGREED.cfg. Give the config that the - spec's authors say the unchanged spec passes. A rule that this config does not - check is "not checked" in the stories. - --label TEXT The name of the --fixed or --against model in the charts and the stories. - Default: "the fix", or "the spec". - --map FILE The labels for each model state and step. Default: SPEC.map.mjs next to the - spec. When the file does not exist, the command writes one from the spec's - VARIABLES and actions to fill in, and exits 2 before it runs TLC. Its - comments say what each function gets. - - With --fixed or --against, SPEC.tla (and FIXED.tla) models the code, and the command finds each - condition of an action that no comment ties to code with a file and line, as in \\* pay.ts:120. - A condition is a conjunct of the action that changes nothing and reads a variable that the - action does not change. A rule can hold only because the model waits for a condition that the - code does not check, so the command prints these conditions when a rule holds. - With --against, each story says whether a config beside AGREED.tla expects TLC to break the - bug's rule. A config's "\\* expect: RULE" line says what TLC finds, and "\\* module: NAME" names - the module it checks when the folder has more than one .tla file. The command warns when no - config beside AGREED.tla has an expect line for that module.`; - -const FROM_CHART = `from-chart CHART.mmd - Writes NAME.tla, NAME.cfg, and NAME.map.mjs next to the chart. When NAME.tla already exists, it - writes none of them. Then it checks the spec as from-spec does, and compares the chart that the - spec draws with CHART.mmd. - - --module NAME The TLA+ module name. Default: the chart's file name in CamelCase, such as - NotesApp for notes-app.mmd. - - The chart is a Mermaid stateDiagram-v2. Each transition label is one action. For parallel - regions, split one top-level composite with --, and put each part in a named composite, such as - state Connection { ... }. Each region is one variable. A step moves every region whose current - state has a transition with the step's label, and the other regions stay where they are. Each - rule is one comment line: - %% never NAME: A & B -- the rule in one sentence - Each name after the colon is a state caption or a composite, and each comes from a different - region. A composite stands for every state in it. The command refuses final, choice, fork, and - history states, and regions below the top level.`; - -const COMMON = `Both commands - --expect-broken RULE - An INVARIANT or PROPERTY that is known to break. Repeat it, or list rules - with commas, as in A,B. The command then exits 0 when exactly these rules - break, and the --fixed or --against model breaks no other rule. - --title TEXT The chart title. Each bug chart adds ", bug N". - --out DIR The output folder. Default: NAME-statechart next to the input. A run writes its - files in a temporary folder, and then replaces the whole output folder with - it. When runs overlap, the output folder has the files of the run that - finished last. The command does not use an output folder that has files that - it did not write. On a filesystem that cannot rename, the command copies the - files into place, and overlapping runs can mix their files. - --png Also render each chart to PNG with npx @mermaid-js/mermaid-cli. - -h, --help Print this help, and exit 0. - -Output - The command prints whether each rule holds, the chart's counts, and one story per bug. - from-chart also prints the comparison. The command writes these files in the output folder: - chart.mmd The overview. - bug-N.mmd Only the steps of bug N, numbered. Bugs are numbered by fewest reader steps, - then fewest model steps, then rule name, then cause. Bugs that only the - --fixed or --against model has come last. - bug-N-developer.mmd, bug-N-layered.mmd - The path of bug N through the developer's states, when the mapping exports - detail(). - stories.md One entry per bug: the rule, why it breaks, the shortest path in reader - steps, and what the --fixed or --against model does instead. - summary.json The counts behind the chart. With --fixed or --against, uncited has the - conditions with no file and line for each code model. With --against, - second.listed has what each config beside AGREED.tla expects, and each bug - has the configs that list it. - tlc/ The config and the TLC output of each run. When the overview has regions, - from-spec also writes tlc/regions-all.mmd, the regions with every arrow, and - tlc/regions-proof.mmd, every state and step of TLC's graph in one region. - compare-charts.mjs tlc/regions-all.mmd tlc/regions-proof.mmd repeats the - proof. - *.png Each chart as a PNG, with --png. A run without --png deletes them. - -Reading a chart - from-spec draws the overview as two parallel regions when every state reads "context: progress": - the contexts on one side, the progress on the other, both at the same time. It does so only when - that has fewer states, and when the regions combine back into exactly every state and step of - TLC's graph. Otherwise it puts the states of one context in a box named context, and prints why. - Regions draw only the arrows of the shortest way to each state, of each bug path, and of the - nearest step with each label. The script checks that these arrows make no step that TLC's graph - does not have. - An arrow from a box's border leaves from every state in the box. - In regions, a step leaves every region that has it. "[when A]" or "[unless A]" says which states - of the other region the step needs. A loop says the region may also stay. - Other words in [ ] come from the mapping's guard(), and "after N s" from its timer(). TLC does - not check them, and has no clock, so a reviewer checks them against the code. - Gray states work. Red states break a rule. With --fixed, red states stay wrong with the fix, and - green states are fixed by it or exist only with it. With --against, red states break a rule in - either model, and green states exist only in the spec. - When the mapping exports group(), the fill is the state's group, which the legend names, and the - status is a thick red or green border on top. tlc-to-chart.mjs --help tells more. - A state is bad when it breaks a rule, or when every path from the start to it passes a state - that breaks one. TLC stops each path at the first state that breaks the rule, so a rule that - stays broken colors only the states where it first breaks and the states after them that no - path without a bug reaches. - A thick blue border and numbered labels mark the bug. Mermaid state diagrams cannot color an - edge. - A label "with the fix, STEP" or "in the spec, STEP" is a step that only the other model takes. On - a bug's path it leaves the state before the first step that the other model shows the reader - differently, and ends where that model goes instead. "with the fix, STEP cannot happen" or "in - the spec, STEP cannot happen" loops on the state where that model stops the step. Outside - regions, two states have at most one arrow between them. Its lines are the numbered steps, then - SPEC.tla's steps, then the other model's steps, with a blank line between each group. - An edge lists every step that takes it, most common first. Words that the steps share at the - start are written once, as in "you tap Pay or Back". - -Regenerate - With --expect-broken, one script rebuilds every chart and stops on any change: - set -e - node statechart.mjs from-spec Code.tla --against Agreed.tla --expect-broken NoLostUpload --png - -TLC - TLA2TOOLS is the path to tla2tools.jar, and JAVA is the java binary (default: java, Java 11 or - later). Download tla2tools.jar from https://github.com/tlaplus/tlaplus/releases. When TLA2TOOLS - is not set, the command says where to get TLC and exits 2. - TLC checks each rule with one worker per CPU. When a rule is broken, TLC checks it again with one - worker, which finds the shortest bug paths, the same on every run. The command then prints how - many times TLC ran and how long that took. - TLC can find that a rule is broken and then fail with a Java exception while it reports the rule. - The command then keeps the paths that TLC printed, marks the rule broken, and warns that the - charts can miss states that break it. The failure is in TLC, so do not weaken the rule because of - it. A smaller constant can let TLC finish. - -Exit codes - 0 Every rule holds, or exactly the --expect-broken rules break. from-chart also draws CHART.mmd - back unchanged. - 1 Another rule is broken, an expected rule holds, or the chart that the spec draws is not - CHART.mmd. The output says which. - 2 The command cannot finish: a bad argument, a missing file, a chart or TLC output that it - cannot read, no TLC, a helper script that cannot start, an output folder with files that the - command did not write, an input file that changed during the run, a new mapping to fill in, - or a chart that --png cannot render. When a chart does not render, the command keeps the - files it wrote and exits 2, whatever the rules give. Otherwise, exit 2 does not change the - output folder, unless the command moved it aside and cannot put it back. Then the message - names the folder's new path. When TLC stops on an error, the message names TLC's output, in a - temporary folder that the command keeps. When the command cannot put its files in the output - folder, it keeps them in a temporary folder, and the message names it. The command checks - its input files again before --png renders, so a changed input stops the run before that.`; - -const HELP = { - "": `${usage([...Object.values(USAGE), "node statechart.mjs [from-spec|from-chart] --help"])} - -Draws a Mermaid statechart of what an app's user sees, from TLC's state graph of a TLA+ spec. The -help calls that user the reader. - -${FROM_SPEC} - -${FROM_CHART} - -${COMMON}`, - "from-spec": `${usage([USAGE["from-spec"]])}\n\n${FROM_SPEC}\n\n${COMMON}`, - "from-chart": `${usage([USAGE["from-chart"]])}\n\n${FROM_CHART}\n\n${COMMON}`, -}; - -const more = (cmd) => - `Run node statechart.mjs ${cmd ? `${cmd} ` : ""}--help for the flags, the output, and the exit codes.`; +const { MAX_STRING_LENGTH } = constants; class CannotFinish extends Error {} class TlcStopped extends CannotFinish {} @@ -219,7 +31,7 @@ class NotPublished extends CannotFinish {} const COMMANDS = { "from-spec": { - flags: [ + values: [ "--map", "--config", "--fixed", @@ -230,23 +42,33 @@ const COMMANDS = { "--expect-broken", "--title", "--out", + "--theme", ], + switches: ["--cover", "--png"], + renders: (flags) => flags.png, run: fromSpec, }, - "from-chart": { flags: ["--module", "--expect-broken", "--title", "--out"], run: fromChart }, + "from-chart": { + values: ["--module", "--expect-broken", "--title", "--out", "--theme"], + switches: ["--png"], + renders: (flags) => flags.png, + run: fromChart, + }, + render: { values: ["--theme", "--out"], switches: [], renders: () => true, run: renderFile }, }; function parseArgs([command, ...rest]) { if (command === "-h" || command === "--help") return { help: HELP[""] }; if (!Object.hasOwn(COMMANDS, command)) throw new CannotFinish(`${usage(Object.values(USAGE))}\n${more()}`); + const { values, switches, renders } = COMMANDS[command]; const flags = { "expect-broken": new Set() }; let input; for (let i = 0; i < rest.length; i++) { const a = rest[i]; if (a === "-h" || a === "--help") return { help: HELP[command] }; - if (a === "--png") flags.png = true; - else if (COMMANDS[command].flags.includes(a)) { + if (switches.includes(a)) flags[a.slice(2)] = true; + else if (values.includes(a)) { if (i + 1 === rest.length) throw new CannotFinish(`${a} needs a value. ${more(command)}`); const name = a.slice(2); if (flags[name] instanceof Set) for (const v of rest[++i].split(",")) flags[name].add(v); @@ -255,9 +77,25 @@ function parseArgs([command, ...rest]) { else throw new CannotFinish(`unknown argument ${a}. ${more(command)}`); } if (input === undefined) throw new CannotFinish(`${usage([USAGE[command]])}\n${more(command)}`); + const themes = Object.keys(THEMES); + if (flags.theme !== undefined && !themes.includes(flags.theme)) + throw new CannotFinish( + `--theme takes ${themes.slice(0, -1).join(", ")}, or ${themes.at(-1)}. ${more(command)}`, + ); + if (flags.theme !== undefined && !renders(flags)) + throw new CannotFinish(`--theme needs --png. ${more(command)}`); return { command, input, flags }; } +async function rendererFor(command, flags) { + if (!COMMANDS[command].renders(flags)) return null; + try { + return { lib: await loadRenderer(), theme: THEMES[flags.theme ?? "light"] }; + } catch (e) { + throw new CannotFinish(e.message); + } +} + const rel = (file) => { const r = path.relative(process.cwd(), file) || "."; return r === ".." || r.startsWith(`..${path.sep}`) ? path.resolve(file) : r; @@ -314,19 +152,48 @@ function splitRules(cfg) { }; } -function tlc(tools, spec, cfg, base, { dump, workers = "1" } = {}) { +function runJava(file, args, options, dot) { + return new Promise((resolve) => { + const r = { stdout: "", stderr: "", stopped: false, overflowed: false }; + const child = spawn(file, args, options); + for (const stream of ["stdout", "stderr"]) + child[stream].setEncoding("utf8").on("data", (text) => { + if (r.overflowed) return; + if (r[stream].length + text.length <= MAX_STRING_LENGTH) r[stream] += text; + else { + r.overflowed = true; + child.kill("SIGKILL"); + } + }); + child.on("error", (e) => (r.error = e)); + const watch = + dot && + setInterval(() => { + if ((fs.statSync(dot, { throwIfNoEntry: false })?.size ?? 0) <= MAX_STRING_LENGTH) return; + r.stopped = true; + child.kill("SIGKILL"); + }, 500); + child.on("close", (status, signal) => { + clearInterval(watch); + resolve({ ...r, status, signal }); + }); + }); +} + +async function tlc(tools, spec, cfg, base, { dump, module, workers = "1" } = {}) { fs.writeFileSync(`${base}.cfg`, cfg); const metadir = fs.mkdtempSync(path.join(os.tmpdir(), "tlc-")); // TLC writes the standard modules, such as Naturals.tla, to java.io.tmpdir under fixed names, // so runs that share that folder can read a module while another run rewrites it. const javaTmp = path.join(metadir, "tmp"); fs.mkdirSync(javaTmp); - // -deadlock: a UI at rest has nothing left to do, which TLC would report as a deadlock. - const r = spawnSync( + // -deadlock: a system at rest has nothing left to do, which TLC would report as a deadlock. + const r = await runJava( tools.java, [ "-XX:+UseParallelGC", `-Djava.io.tmpdir=${javaTmp}`, + ...(module ? [`-DTLA-Library=${path.dirname(path.resolve(spec))}`] : []), "-cp", tools.jar, "tlc2.TLC", @@ -339,12 +206,23 @@ function tlc(tools, spec, cfg, base, { dump, workers = "1" } = {}) { ...(dump ? ["-dump", "dot,actionlabels", path.resolve(dump)] : []), "-config", path.resolve(`${base}.cfg`), - path.basename(spec), + module ? path.resolve(module) : path.basename(spec), ], - { cwd: path.dirname(path.resolve(spec)), encoding: "utf8", maxBuffer: 1 << 30 }, + { cwd: path.dirname(path.resolve(spec)) }, + dump, ); fs.rmSync(metadir, { recursive: true, force: true }); - if (r.error) throw new CannotFinish(`cannot run TLC: ${r.error.message}`); + if (r.overflowed) { + fs.writeFileSync(`${base}.out`, r.stdout); + throw new TlcStopped( + `${base}.out: TLC printed more than ${MAX_STRING_LENGTH.toLocaleString("en-US")} characters, the most that the command can read, so the command stopped TLC. Smaller constants make TLC print less.`, + ); + } + if (r.stopped) { + fs.writeFileSync(`${base}.out`, r.stdout); + return { stdout: r.stdout, stopped: true }; + } + if (r.error || !/^TLC2 Version/m.test(r.stdout)) throw new CannotFinish(cannotRunTlc(tools, r)); fs.writeFileSync(`${base}.out`, r.stdout); try { return { stdout: r.stdout, warning: tlcOutputWarning(r.stdout, `${base}.out`) }; @@ -353,14 +231,8 @@ function tlc(tools, spec, cfg, base, { dump, workers = "1" } = {}) { } } -const withoutComments = (spec) => - fs - .readFileSync(spec, "utf8") - .replace(/\(\*[^]*?\*\)/g, " ") - .replace(/\\\*.*/g, " "); - -function warnUnnamedSteps(spec, cfg, dot) { - const text = withoutComments(spec); +function warnUnnamedSteps({ spec, copy }, cfg, dot) { + const text = withoutComments(copy); const specName = /^SPECIFICATION\s+(\w+)/m.exec(cfg)?.[1]; const specBody = specName && new RegExp(`^${specName}\\s*==(.*(\\n[ \\t]+.*)*)`, "m").exec(text); const next = @@ -369,13 +241,15 @@ function warnUnnamedSteps(spec, cfg, dot) { const unnamed = [...graph.edges, ...graph.loops].filter((e) => e.action === next).length; if (unnamed > 0) console.error( - `${rel(spec)}: ${unnamed} ${unnamed === 1 ? "step has" : "steps have"} the name ${next}, the whole next-state action, because TLC names a step after the innermost named action that takes it. If the reader must tell these steps apart, give each part of ${next} its own name, as in Pay == ... and ${next} == Pay \\/ Charge.`, + `${rel(spec)}: ${unnamed} ${unnamed === 1 ? "step has" : "steps have"} the name ${next}, the whole next-state action. TLC goes down from ${next} through each \\/, \\E, LET, and definition, and names each step after the last definition on that path. It stops at any other expression, such as /\\, IF, or CASE. If the reader must tell these steps apart, join a definition for each kind of step with \\/ in ${next}, as in ${next} == Pay \\/ Charge, and move each conjunct of ${next} into those definitions.`, ); + return next; } -function readModel(spec, cfgFile, flag) { - if (!fs.existsSync(spec)) throw new CannotFinish(`${rel(spec)} does not exist`); - if (!fs.existsSync(cfgFile)) { +function readModel(spec, cfgFile, flag, inputs) { + const [copy, cfgCopy] = [spec, cfgFile].map((f) => inputs.copyOf(f)); + if (!fs.existsSync(copy)) throw new CannotFinish(`${rel(spec)} does not exist`); + if (!fs.existsSync(cfgCopy)) { const beside = fs .readdirSync(path.dirname(spec)) .filter((f) => f.endsWith(".cfg")) @@ -388,11 +262,11 @@ function readModel(spec, cfgFile, flag) { }`, ); } - return { spec, cfgFile, ...splitRules(fs.readFileSync(cfgFile, "utf8")) }; + return { spec, copy, cfgFile, ...splitRules(fs.readFileSync(cfgCopy, "utf8")) }; } -function readInput(spec, cfgFile, expected) { - const model = readModel(spec, cfgFile, "--config"); +function readInput(spec, cfgFile, expected, inputs) { + const model = readModel(spec, cfgFile, "--config", inputs); const names = [...model.rules, ...model.properties]; const unknown = [...expected].filter((r) => !names.includes(r)); if (unknown.length) @@ -402,53 +276,79 @@ function readInput(spec, cfgFile, expected) { return model; } -function check(tools, model, { work, out }, expected) { - const { spec, cfgFile, cfgWithoutRules, rules, properties } = model; +async function check(tools, model, { work, out, role }, expected) { + const { spec, copy, cfgFile, cfgWithoutRules, rules, properties } = model; const started = Date.now(); let runs = 0; const run = (cfg, file, options) => { runs++; - return tlc(tools, spec, cfg, file, options); + return tlc(tools, copy, cfg, file, options); }; const name = path.basename(spec, ".tla"); - const tlcDir = path.join(work, "tlc"); - const dot = path.join(tlcDir, `${name}.dot`); - // One worker writes the graph in a fixed order, which breaks ties between equally short paths. - run(cfgWithoutRules, path.join(tlcDir, name), { dump: dot }); - warnUnnamedSteps(spec, cfgWithoutRules, dot); - console.log(`TLC on ${path.basename(spec)} with ${rel(cfgFile)}:`); + const tlcDir = path.join(work, "tlc", ...(role ? [role] : [])); + fs.mkdirSync(tlcDir, { recursive: true }); + console.log(`TLC on ${rel(spec)} with ${rel(cfgFile)}:`); if (rules.length === 0) console.log( ` ${path.basename(cfgFile)} has no INVARIANT, so ${properties.length ? "the chart marks no state broken" : "no rule is checked"}.`, ); const said = (rule, broken) => ` ${rule} ${broken ? (expected.has(rule) ? "is broken (expected)" : "is broken") : "holds"}.`; - const shown = (file) => rel(path.join(out, "tlc", path.basename(file))); - const traces = rules.map((rule) => { + const shown = published(work, out); + const traces = []; + for (const rule of rules) { const file = path.join(tlcDir, `${name}-${rule}`); const cfg = `${cfgWithoutRules}INVARIANT ${rule}\nCONSTRAINT ${rule}\n`; - let result = run(cfg, file, { workers: "auto" }); + let result = await run(cfg, file, { workers: "auto" }); // Several workers can reach a bad state by a longer path first. One worker searches breadth // first in a fixed order, so the chart's bug paths are the shortest, and the same every run. - if (/^Error: Invariant /m.test(result.stdout)) result = run(cfg, file); + if (/^Error: Invariant /m.test(result.stdout)) result = await run(cfg, file); const broken = /^Error: Invariant /m.test(result.stdout); console.log(said(rule, broken)); if (result.warning) console.error(`${shown(`${file}.out`)}: ${result.warning}`); - return { file: `${file}.out`, rule, broken }; - }); - const temporal = properties.map((property) => { + traces.push({ file: `${file}.out`, rule, broken }); + } + const temporal = []; + for (const property of properties) { const file = path.join(tlcDir, `${name}-${property}`); - const { stdout } = run(`${cfgWithoutRules}PROPERTY ${property}\n`, file, { workers: "auto" }); + const { stdout } = await run(`${cfgWithoutRules}PROPERTY ${property}\n`, file, { + workers: "auto", + }); const broken = BROKEN_PROPERTY.test(stdout); - console.log( - `${said(property, broken)}${broken ? ` The chart draws only INVARIANT rules, so read TLC's counterexample in ${shown(`${file}.out`)}.` : ""}`, - ); - return { file: `${file}.out`, rule: property, broken }; + const notes = [ + broken && + `The chart draws only INVARIANT rules, so read TLC's counterexample in ${shown(`${file}.out`)}.`, + checkedOnlyInInitialStates(spec, property) && + `TLC checked ${property} only in the initial states, because it has no [], <>, or ~>. List it as an INVARIANT to check every state.`, + ]; + console.log([said(property, broken), ...notes.filter(Boolean)].join(" ")); + temporal.push({ file: `${file}.out`, rule: property, broken }); + } + const dot = path.join(tlcDir, `${name}.dot`); + const module = rules.length ? writeRulesModule(copy, rules, tlcDir) : undefined; + const dumpCfg = module ? `${cfgWithoutRules}INVARIANT ${RULES_INVARIANT}\n` : cfgWithoutRules; + const { stopped } = await run(dumpCfg, path.join(tlcDir, name), { + dump: dot, + module, + workers: "1", }); + const dotBytes = fs.statSync(dot).size; + const next = dotBytes <= MAX_STRING_LENGTH && warnUnnamedSteps(model, cfgWithoutRules, dot); const seconds = ((Date.now() - started) / 1000).toFixed(1); console.log(` TLC ran ${runs} ${runs === 1 ? "time" : "times"} in ${seconds} s.`); const broken = [...traces, ...temporal].some((t) => t.broken); - return { spec, dot, traces, properties: temporal, broken }; + return { + spec, + copy, + dot, + dotBytes, + stopped, + next, + dumpOut: path.join(tlcDir, `${name}.out`), + traces, + properties: temporal, + broken, + }; } function rulesAsExpected(now, other, expected) { @@ -466,93 +366,28 @@ function rulesAsExpected(now, other, expected) { return !unexpected.length && !holds.length; } -const SCAFFOLD = (name, spec) => { - const text = withoutComments(spec); - const vars = tlaVariables(text); - // A definition that only another action calls is listed too, though TLC names no step after it. - const actions = [ - ...new Set( - text - .split(/\n(?=\S)/) - .map((d) => /^(\w+)\s*(?:\([^)]*\))?\s*==([^]*)/.exec(d)) - .filter((d) => d && PRIMES_OR_UNCHANGED.test(d[2])) - .map((d) => d[1]), - ), - ].sort(); - return `// Chart labels for ${name}, for statechart.mjs. Write labels in plain words, not variable or action names. -// v, before, and after map each variable to its value. Strings, numbers, and booleans keep their kind. -// Sequences, tuples, and sets become arrays, indexed from 0. Records and functions become objects. -// Variables: ${vars.join(", ")}. - -// The actions the reader takes. facts.settled is true when only these are enabled, and the overview -// keeps the shortest path to each settled state, plus every bug path. -export const readerActions = []; - -// The label for one model state. facts.enabled is the set of actions the state enables, -// including steps that change no variable. Write it as "context: progress", such as -// "Offline: upload queued": the chart puts the states of one context in one box. -// Do not claim a result that a pending step can still change, such as "upload failed" while a -// retry is still queued. -export function state(v, facts) { - return JSON.stringify(v); -} - -// The names of the two parallel regions, when the overview draws them: contexts, then progress. -// export const regionNames = ["Network", "What happened to the file"]; - -// The group of one model state, in words the reader sees, such as "Upload done". The chart fills -// the states of one group with one color, and the legend names the groups. TLC still decides -// which states break a rule, and draws them with a thick red border on top of the group color. -// groupColors gives a group a light color. Groups without one take a palette color. -// export function group(v, facts) {} -// export const groupColors = { "Upload done": "#d1f2eb" }; - -// The label for one step. A step that leaves the reader in the same state is not drawn, -// unless the reader takes it on a bug path. "" draws no label, and a numbered bug step shows the action. -const STEPS = { -${actions.map((a) => `\t${a}: ${JSON.stringify(a)},`).join("\n")} -}; -export function step(action, before, after) { - return STEPS[action] ?? action; -} - -// When the step can happen, in plain words, such as "offline or disk full". The chart shows -// "step [words]". "" adds nothing. -// export function guard(action, before, after) {} - -// For a timer step, how many seconds the code waits, from the constant in the code. The chart -// shows "after N s, step". Return nothing for other steps. -// export function timer(action, before, after) {} - -// Why a bad state is bad, in words a product manager reads, such as "the laptop went offline, and -// the app drops an upload it cannot send". No variable, function, or event names. -// Counterexamples that break one rule with one cause are one bug, so two bugs that end on the -// same screen stay apart only if their causes differ. Return null to group by the end state. -// export function cause(v, facts, invariant) {} - -// The developer's words for one model state. With it, each bug is also drawn through these states. -// export function detail(v, facts) {} - -// The legend's words for each status, in place of the defaults. Group entries keep their names. -// export const legend = { works: "Works on main", wrong: "Wrong on main", fixed: "Fixed in the open PR" }; -`; -}; - -async function draw({ now, other, uncited, map, work, out, title, comparing }) { +async function draw({ now, other, uncited, map, work, out, title, comparing, cover }) { + const huge = [now, other].find((m) => m?.dotBytes > MAX_STRING_LENGTH); + const limit = MAX_STRING_LENGTH.toLocaleString("en-US"); + if (huge) + throw new NotPublished( + `${rel(huge.spec)}: TLC's state graph ${huge.stopped ? `passed ${limit} bytes, the most that the command can read, so the command stopped TLC and draws no chart` : `has ${huge.dotBytes.toLocaleString("en-US")} bytes, more than the ${limit} that the command can read, so it draws no chart`}. The rule results above stand. Smaller constants make a smaller graph.`, + ); const shown = published(work, out); const mapping = await import(pathToFileURL(path.resolve(map)).href); - const from = ["--dot", now.dot, "--map", map, "--spec", now.spec]; + const from = ["--dot", now.dot, "--broken", now.dumpOut, "--map", map, "--spec", now.copy]; + for (const m of [now, other]) if (m?.next) from.push("--next", m.next); from.push(...now.traces.flatMap((t) => ["--traces", t.file])); if (other) { - from.push(`--${other.role}-dot`, other.dot, "--spec", other.spec); + from.push(`--${other.role}-dot`, other.dot, `--${other.role}-broken`, other.dumpOut); + from.push("--spec", other.copy); from.push(...other.traces.flatMap((t) => [`--${other.role}-traces`, t.file])); if (other.traces.length) from.push(`--${other.role}-rules`, other.traces.map((t) => t.rule).join(",")); if (other.label !== undefined) from.push("--label", other.label); if (other.listed) from.push("--listed", JSON.stringify(other.listed)); } - if (comparing) from.push("--full-labels", "--flat"); - else if (mapping.readerActions?.length) from.push("--cover"); + if (comparing) from.push("--full-labels", "--flat", "--loops"); const titled = (suffix) => (title ? ["--title", `${title}${suffix}`] : []); const files = { @@ -567,6 +402,7 @@ async function draw({ now, other, uncited, map, work, out, title, comparing }) { [ ...from, ...(comparing ? [] : ["--regions", "--proof", proof, "--regions-all", everyArrow]), + ...(cover ? ["--cover"] : []), ...titled(""), "--highlight", "0", @@ -580,8 +416,46 @@ async function draw({ now, other, uncited, map, work, out, title, comparing }) { map, ); const summary = JSON.parse(fs.readFileSync(files.summary, "utf8")); - if (uncited) - fs.writeFileSync(files.summary, `${JSON.stringify({ ...summary, uncited }, null, 2)}\n`); + const L = summary.second?.label; + const broken = [now, other].filter(Boolean).flatMap((m) => { + const book = ruleBook(m.copy); + const where = m === now ? "" : ` ${other.role === "fixed" ? "with" : "in"} ${L}`; + const nameOf = readerStateNamer(mapping, m.dot); + return m.properties + .filter((p) => p.broken) + .map((p) => ({ + where, + property: { + spec: rel(m.spec), + property: p.rule, + rule: book.get(p.rule)?.sentence ?? null, + repeats: repeatsIn(fs.readFileSync(p.file, "utf8"), nameOf), + counterexample: path.relative(work, p.file), + }, + })); + }); + const brokenProperties = broken.map((b) => b.property); + fs.writeFileSync( + files.summary, + `${JSON.stringify({ ...summary, ...(uncited && { uncited }), brokenProperties }, null, 2)}\n`, + ); + if (brokenProperties.length) { + const lines = broken.map( + ({ where, property: b }) => + `Rule broken${where}: ${[ + (b.rule ?? `\`${b.property}\``).replace(/\.?$/, "."), + loopSentence(b.repeats), + `TLC's counterexample is in ${b.counterexample}.`, + ] + .filter(Boolean) + .join(" ")}`, + ); + const bugs = fs.readFileSync(files.stories, "utf8"); + fs.writeFileSync( + files.stories, + `${bugs}${bugs && "\n"}## Broken rules that the chart does not draw\n\n${lines.join("\n")}\n`, + ); + } const charts = [files.overview]; for (const c of summary.classes) { const views = typeof mapping.detail === "function" ? ["", "-developer", "-layered"] : [""]; @@ -597,8 +471,7 @@ async function draw({ now, other, uncited, map, work, out, title, comparing }) { } } - const { works, wrong, fixed: fixedCount } = summary.chartStatus; - const L = summary.second?.label; + const { works, wrong, fixed: fixedCount, mixed, after, worse } = summary.chartStatus; console.log( `\nChart: ${summary.chartStates} states and ${summary.chartTransitions} transitions from ${summary.modelStates} model states.`, ); @@ -608,58 +481,101 @@ async function draw({ now, other, uncited, map, work, out, title, comparing }) { `It has 2 parallel regions, of ${r.states[0]} and ${r.states[1]} states. Proof: their ${r.all} arrows together make exactly the ${r.flat.states} states and ${r.flat.transitions} labeled steps of ${shown(proof)}, and the ${r.transitions} drawn arrows make only steps among them. ${shown(everyArrow)} has all ${r.all} arrows.`, ); else if (r) console.log(`It has no parallel regions, because ${r.why}.`); - const breaks = `${wrong} ${wrong === 1 ? "breaks" : "break"} a rule`; + const c = summary.cover; + if (c?.drawn === false) console.log(`--cover draws every state, because ${c.why}.`); + else if (c) { + const list = (head, xs) => [ + ...(xs.length ? [head] : []), + ...xs.slice(0, 10).map((x) => ` ${x}`), + ...(xs.length > 10 ? [` and ${xs.length - 10} more in summary.json`] : []), + ]; + const left = [ + ...list("States left out:", c.leftOutStates), + ...list("Transitions left out:", c.leftOutTransitions), + ]; + console.log( + left.length + ? `--cover leaves out ${c.leftOutStates.length} of ${c.states} states and ${c.leftOutTransitions.length} of ${c.transitions} transitions. The counts of states that work or break a rule include the states it leaves out.\n${left.join("\n")}` + : "--cover leaves out no state and no transition.", + ); + } + const verb = (n) => (n === 1 ? "breaks" : "break"); + const breaks = `${wrong} ${verb(wrong)} a rule`; + const withFix = other?.role === "fixed" ? ` with ${L}` : ""; + const partly = + (mixed ? `, ${mixed} ${verb(mixed)} a rule in some cases${withFix}` : "") + + (after ? `, ${after} reached only after a bug${withFix}` : "") + + (worse ? `, ${worse} ${verb(worse)} a rule only${withFix}` : ""); console.log( other?.role === "fixed" - ? `${works} work today, ${wrong} still wrong with ${L}, ${fixedCount} fixed or new with ${L}.\n` + ? `${works} work today, ${wrong} still wrong with ${L}, ${fixedCount} fixed or new with ${L}${partly}.\n` : other?.role === "against" - ? `${works} work, ${breaks}, ${fixedCount} only in ${L}.\n` - : `${works} work, ${breaks}.\n`, + ? `${works} work, ${breaks}, ${fixedCount} only in ${L}${partly}.\n` + : `${works} work, ${breaks}${partly}.\n`, ); + if (summary.mixedStatus.length) + console.log( + `${summary.mixedStatus.length === 1 ? "This state breaks" : "These states break"} a rule in some cases and not in others:\n${summary.mixedStatus.map((t) => ` ${t}`).join("\n")}\n`, + ); if (summary.codeLike.length) console.log( `These labels or causes read like code. Write them in the reader's words in ${rel(map)}:\n${summary.codeLike.map((t) => ` ${t}`).join("\n")}\n`, ); + const stories = fs.readFileSync(files.stories, "utf8").trimEnd(); console.log( summary.classes.length - ? fs.readFileSync(files.stories, "utf8").trimEnd() - : "No state breaks a rule.", + ? stories + : ["No state breaks an INVARIANT rule.", stories].filter(Boolean).join("\n\n"), ); return { files, charts }; } -function render(charts, shown) { +const pngOf = (chart) => chart.replace(/(\.mmd)?$/, ".png"); + +function render(charts, { lib, theme }, shown) { const pngs = []; for (const chart of charts) { - const png = chart.replace(/\.mmd$/, ".png"); - const r = spawnSync( - "npx", - ["-y", "@mermaid-js/mermaid-cli", "-i", chart, "-o", png, "-s", "2"], - { - encoding: "utf8", - }, - ); - if (r.status === 0) pngs.push(png); - else - console.error( - `could not render ${shown(chart)}: ${(r.stderr || r.error?.message || "").trim()}`, - ); + try { + fs.writeFileSync(pngOf(chart), renderPng(lib, fs.readFileSync(chart, "utf8"), { theme })); + pngs.push(pngOf(chart)); + } catch (e) { + console.error(`could not render ${shown(chart)}: ${e.message}`); + } } return pngs; } -function finish({ work, out, inputs }, { files, charts }, ok, png) { +function finish({ work, out, inputs }, { files, charts }, ok, renderer) { unchanged(inputs, out); - const pngs = png ? render(charts, published(work, out)) : []; + const pngs = renderer ? render(charts, renderer, published(work, out)) : []; publish(work, out, inputs); const written = [...charts, ...pngs, files.stories, files.summary] .map((f) => path.basename(f)) .join(", "); console.log(`\nWrote ${written} in ${rel(out)}.`); - if (png && pngs.length < charts.length) return 2; + if (renderer && pngs.length < charts.length) return 2; return ok ? 0 : 1; } +function renderFile(input, flags, { lib, theme }) { + const out = flags.out ?? pngOf(input); + const step = (failed, f) => { + try { + return f(); + } catch (e) { + throw new CannotFinish(`${failed}: ${e.message}`); + } + }; + const src = step(`cannot read ${rel(input)}`, () => fs.readFileSync(input, "utf8")); + const png = step(`could not render ${rel(input)}`, () => renderPng(lib, src, { theme })); + step(`cannot write ${rel(out)}`, () => { + fs.mkdirSync(path.dirname(path.resolve(out)), { recursive: true }); + fs.writeFileSync(out, png); + }); + console.log(`Wrote ${rel(out)}.`); + return 0; +} + const published = (work, out) => (file) => rel(path.join(out, path.relative(work, file))); const OWN = @@ -691,20 +607,6 @@ function outDir(flagged, input, name) { return out; } -const fingerprint = (file) => { - try { - const { ino, ctimeNs } = fs.statSync(file, { bigint: true }); - return `${ino} ${ctimeNs} ${fs.readFileSync(file, "base64")}`; - } catch { - return null; - } -}; - -function snapshot(files, inputs = new Map()) { - for (const f of files) if (f && !inputs.has(f)) inputs.set(f, fingerprint(f)); - return inputs; -} - async function inWorkFolder(body) { const work = fs.mkdtempSync(path.join(os.tmpdir(), "statechart-")); fs.mkdirSync(path.join(work, "tlc")); @@ -712,8 +614,8 @@ async function inWorkFolder(body) { try { return await body(work); } catch (e) { - keep = e instanceof TlcStopped || e instanceof NotPublished; - if (e instanceof NotPublished) + keep = fs.readdirSync(path.join(work, "tlc")).some((f) => f.endsWith(".out")); + if (keep && !(e instanceof TlcStopped)) e.message += ` The files that this run wrote are in ${rel(work)}.`; throw e; } finally { @@ -730,24 +632,61 @@ const moved = (from, to, ...missed) => { if (!["ENOSYS", "EXDEV", "EISDIR"].includes(e.code)) throw e; } // Some filesystems, such as FUSE mounts, cannot rename a folder that has files. They fail with - // ENOSYS, or with EISDIR when the target exists. + // ENOSYS, or with EISDIR when the target exists. They can rename an empty folder. if (!fs.existsSync(from) && missed.includes("ENOENT")) return false; - if (fs.existsSync(to)) { - if (fs.readdirSync(to).length && missed.includes("ENOTEMPTY")) return false; - fs.rmdirSync(to); - } + const claimed = claimedFolder(to, missed); + if (claimed === null) return false; try { - fs.cpSync(from, to, { recursive: true, errorOnExist: true, force: false }); + copyExclusive(from, to); + if (!sameFolder(to, claimed)) + throw new Error("another run replaced the folder during the copy"); } catch (e) { - fs.rmSync(to, { recursive: true, force: true }); + if (sameFolder(to, claimed)) fs.rmSync(to, { recursive: true, force: true }); throw e; + } finally { + fs.closeSync(claimed); } fs.rmSync(from, { recursive: true, force: true }); return true; }; +function claimedFolder(to, missed) { + // Making a folder and opening it take two calls. Another run can replace the folder at `to` + // between them. So the run opens its folder before it moves the folder to `to`. + // A filesystem can reuse a removed folder's inode number. Holding this folder open keeps its + // number, so a folder that another run makes in its place cannot match it. + const mine = fs.mkdtempSync(path.join(path.dirname(to), `.${path.basename(to)}-`)); + const claimed = fs.openSync(mine, "r"); + try { + fs.renameSync(mine, to); + return claimed; + } catch (e) { + fs.closeSync(claimed); + fs.rmdirSync(mine); + if (missed.includes(e.code)) return null; + throw e; + } +} + +function sameFolder(at, claimed) { + const now = fs.statSync(at, { throwIfNoEntry: false }); + const made = fs.fstatSync(claimed); + return now?.dev === made.dev && now?.ino === made.ino; +} + +function copyExclusive(from, to) { + for (const entry of fs.readdirSync(from, { withFileTypes: true })) { + const source = path.join(from, entry.name); + const target = path.join(to, entry.name); + if (entry.isDirectory()) { + fs.mkdirSync(target); + copyExclusive(source, target); + } else fs.copyFileSync(source, target, fs.constants.COPYFILE_EXCL); + } +} + function unchanged(inputs, out) { - const changed = [...inputs].find(([f, was]) => fingerprint(f) !== was)?.[0]; + const changed = inputs.changed(); if (changed) throw new NotPublished( `${rel(changed)} changed while this run read it, so the run left ${rel(out)} as it was. Run the command again.`, @@ -806,7 +745,7 @@ function secondCfg(cfg, flags) { return flags.against && (flags["against-config"] ?? cfgOf(flags.against)); } -function secondModel(input, cfg, { fixed, against, label }) { +function secondModel(input, cfg, { fixed, against, label }, inputs) { if (fixed && against) throw new CannotFinish(`use --fixed or --against, not both. ${more("from-spec")}`); if (!fixed && !against) { @@ -816,19 +755,16 @@ function secondModel(input, cfg, { fixed, against, label }) { ); } const [role, spec] = fixed ? ["fixed", fixed] : ["against", against]; - const name = path.basename(input, ".tla"); - if (path.basename(spec, ".tla") === name) - throw new CannotFinish( - `${rel(spec)} needs its own file and module name, such as ${name}${fixed ? "Fixed" : "Agreed"}.tla.`, - ); - return { role, label, model: readModel(spec, cfg, `--${role}-config`) }; + if (path.resolve(spec) === path.resolve(input)) + throw new CannotFinish(`--${role} needs a file other than the spec, ${rel(input)}.`); + return { role, label, model: readModel(spec, cfg, `--${role}-config`, inputs) }; } function warnUnchecked(now, other) { const unchecked = now.rules.filter((r) => !other.rules.includes(r)); if (unchecked.length) console.error( - `${rel(other.cfgFile)} does not check ${unchecked.join(", ")}, so the stories cannot say whether ${path.basename(other.spec)} keeps ${unchecked.length === 1 ? "it" : "them"}.`, + `${rel(other.cfgFile)} does not check ${unchecked.join(", ")}, so the stories cannot say whether ${rel(other.spec)} keeps ${unchecked.length === 1 ? "it" : "them"}.`, ); } @@ -860,27 +796,28 @@ function warnUnlisted(now, other) { function warnUncited(models) { return Object.fromEntries( - models.map(({ spec, traces, properties }) => { - const found = uncitedConditions(fs.readFileSync(spec, "utf8")); + models.map(({ spec, copy, traces, properties }) => { + const found = uncitedConditions(fs.readFileSync(copy, "utf8")); const held = [...traces, ...properties].filter((t) => !t.broken).map((t) => t.rule); const short = (s) => (s.length > 90 ? `${s.slice(0, 87)}...` : s); if (found.length && held.length) console.error( `${rel(spec)}: ${joinAnd(held)} ${held.length === 1 ? "holds" : "hold"}, but a rule can also hold because the model waits for a condition that the code does not check. No comment gives the file and line of the code that checks ${found.length === 1 ? "this condition" : "these conditions"}:\n${found.map((c) => ` ${rel(spec)}:${c.line} ${c.action}: ${short(c.condition)}`).join("\n")}\nPut that file and line in a comment on each condition, as in \\* pay.ts:120, or remove the condition when no code checks it.`, ); - return [path.basename(spec), found]; + return [rel(spec), found]; }), ); } -async function fromSpec(input, { "expect-broken": expected, ...flags }) { +async function fromSpec(input, { "expect-broken": expected, title, cover, ...flags }, renderer) { const name = path.basename(input, ".tla"); const map = flags.map ?? path.join(path.dirname(input), `${name}.map.mjs`); const cfg = flags.config ?? cfgOf(input); const otherCfg = secondCfg(cfg, flags); - const inputs = snapshot([input, cfg, map, flags.fixed ?? flags.against, otherCfg]); - const second = secondModel(input, otherCfg, flags); - const model = readInput(input, cfg, expected); + const inputs = new Inputs().add([input, cfg, map, flags.fixed ?? flags.against, otherCfg]); + const second = secondModel(input, otherCfg, flags, inputs); + const listed = second?.role === "against" ? listedBreaks(second.model.spec) : undefined; + const model = readInput(input, cfg, expected, inputs); const out = outDir(flags.out, input, name); if (!fs.existsSync(map)) { fs.writeFileSync(map, SCAFFOLD(name, input)); @@ -888,12 +825,12 @@ async function fromSpec(input, { "expect-broken": expected, ...flags }) { } const tools = findTools(); return inWorkFolder(async (work) => { - const now = check(tools, model, { work, out }, expected); + const now = await check(tools, model, { work, out }, expected); const other = second && { role: second.role, label: second.label, - listed: second.role === "against" ? listedBreaks(second.model.spec) : undefined, - ...check(tools, second.model, { work, out }, expected), + listed, + ...(await check(tools, second.model, { work, out, role: second.role }, expected)), }; let uncited; if (second) { @@ -901,12 +838,12 @@ async function fromSpec(input, { "expect-broken": expected, ...flags }) { if (second.role === "against") warnUnlisted(now, other); uncited = warnUncited(second.role === "fixed" ? [now, other] : [now]); } - const drawn = await draw({ now, other, uncited, map, work, out, title: flags.title }); - return finish({ work, out, inputs }, drawn, rulesAsExpected(now, other, expected), flags.png); + const drawn = await draw({ now, other, uncited, map, work, out, title, cover }); + return finish({ work, out, inputs }, drawn, rulesAsExpected(now, other, expected), renderer); }); } -async function fromChart(input, { "expect-broken": expected, ...flags }) { +async function fromChart(input, { "expect-broken": expected, ...flags }, renderer) { const name = flags.module ?? path @@ -916,23 +853,28 @@ async function fromChart(input, { "expect-broken": expected, ...flags }) { throw new CannotFinish(`cannot make a TLA+ module name from ${input}; add --module NAME`); const spec = path.join(path.dirname(input), `${name}.tla`); const map = spec.replace(/\.tla$/, ".map.mjs"); - const inputs = snapshot([input]); + const inputs = new Inputs().add([input]); if (fs.existsSync(spec)) { console.log(`Kept ${rel(spec)}. Delete it to write the spec again from ${rel(input)}.`); } else { script("chart-to-tla.mjs", [input, "--module", name], input); console.log(`Wrote ${name}.tla, ${name}.cfg, and ${name}.map.mjs from ${rel(input)}.`); } - snapshot([spec, cfgOf(spec), map], inputs); + inputs.add([spec, cfgOf(spec), map]); const tools = findTools(); - const model = readInput(spec, cfgOf(spec), expected); + const model = readInput(spec, cfgOf(spec), expected, inputs); const out = outDir(flags.out, input, name); return inWorkFolder(async (work) => { - const now = check(tools, model, { work, out }, expected); + const now = await check(tools, model, { work, out }, expected); const drawn = await draw({ now, map, work, out, title: flags.title, comparing: true }); - const compared = script("compare-charts.mjs", [input, drawn.files.overview], input, { - okExits: [0, 1], - }); + const compared = script( + "compare-charts.mjs", + [inputs.copyOf(input), drawn.files.overview], + input, + { + okExits: [0, 1], + }, + ); if (compared.status === 0) console.log(`\nThe spec draws ${rel(input)} back unchanged.`); else console.log( @@ -942,7 +884,7 @@ async function fromChart(input, { "expect-broken": expected, ...flags }) { { work, out, inputs }, drawn, rulesAsExpected(now, null, expected) && compared.status === 0, - flags.png, + renderer, ); }); } @@ -953,7 +895,7 @@ try { console.log(help); process.exit(0); } - process.exit(await COMMANDS[command].run(input, flags)); + process.exit(await COMMANDS[command].run(input, flags, await rendererFor(command, flags))); } catch (e) { if (!(e instanceof CannotFinish)) throw e; console.error(e.message); diff --git a/pstack/skills/statechart/scripts/test/band.mmd b/pstack/skills/statechart/scripts/test/band.mmd new file mode 100644 index 000000000..83af58601 --- /dev/null +++ b/pstack/skills/statechart/scripts/test/band.mmd @@ -0,0 +1,15 @@ +--- +title: "An edge into a group from above" +--- +stateDiagram-v2 + state "Outside" as a + state "A group" as G { + state "Inside one" as i1 + state "Inside two" as i2 + i1 --> i2 : go + } + state "After" as b + [*] --> a + a --> G : into the group + G --> b : out of the group + a --> i2 : straight to a child diff --git a/pstack/skills/statechart/scripts/test/regions.mmd b/pstack/skills/statechart/scripts/test/regions.mmd new file mode 100644 index 000000000..d241e24ff --- /dev/null +++ b/pstack/skills/statechart/scripts/test/regions.mmd @@ -0,0 +1,62 @@ +--- +title: "Parcel and payment" +config: + markdownAutoWrap: false + state: + useMaxWidth: false +--- +stateDiagram-v2 + classDef plain fill:#f4f6f7,stroke:#566573,color:#1b2631 + classDef plainWrong fill:#f4f6f7,stroke:#c0392b,stroke-width:4px,color:#7b241c + classDef group0 fill:#d6eaf8,stroke:#6b757c,color:#1b2631 + classDef group0Wrong fill:#d6eaf8,stroke:#c0392b,stroke-width:4px,color:#7b241c + classDef group1 fill:#fdebd0,stroke:#7f7668,color:#1b2631 + state "Legend" as legend { + state "Card on file" as legendGroup0 + state "No card yet" as legendGroup1 + state "Breaks a rule" as legendWrong + } + class legendGroup0 group0 + class legendGroup1 group1 + class legendWrong plainWrong + state "At the same time" as regions { + state "The parcel" as region0 { + [*] --> r0s0 + state "Label printed" as r0s0 + state "In transit" as r0s1 + state "Out for delivery" as r0s2 + state "Left at the front door
without a signature" as r0s3 + state "Held at the depot after
a missed delivery" as r0s4 + state "Sent back to the shop" as r0s5 + r0s0 --> r0s1 : the courier scans the label
at the depot + r0s1 --> r0s2 : the van leaves the depot with
the parcel on board + r0s2 --> r0s3 : the courier leaves the parcel
at the door [when charged
once] + r0s2 --> r0s4 : nobody answers the door
before the van moves on + r0s4 --> r0s2 : the depot books a second
delivery for the next day + r0s4 --> r0s5 : the parcel waits at the depot
for more than fourteen days + } + -- + state "Payment" as region1 { + [*] --> r1s0 + state "No card on file" as r1s0 + state "Card saved, waiting
for the charge" as r1s1 + state "Charged once" as r1s2 + state "Charged twice for
one parcel" as r1s3 + state "Refund sent while the
parcel was on the doorstep" as r1s4 + r1s0 --> r1s1 : you save a card on the
checkout page + r1s1 --> r1s2 : the bank approves the charge
for the parcel + r1s2 --> r1s3 : the payment service retries
a charge that went through + r1s2 --> r1s4 : the shop sends a refund before
the parcel comes back [when
left at the front door] + } + } + class r0s0 group1 + class r0s1 plain + class r0s2 plain + class r0s3 plainWrong + class r0s4 plain + class r0s5 plain + class r1s0 group1 + class r1s1 group0 + class r1s2 group0 + class r1s3 group0Wrong + class r1s4 group0Wrong diff --git a/pstack/skills/statechart/scripts/test/statechart.test.mjs b/pstack/skills/statechart/scripts/test/statechart.test.mjs index 72782e679..543fe8d67 100644 --- a/pstack/skills/statechart/scripts/test/statechart.test.mjs +++ b/pstack/skills/statechart/scripts/test/statechart.test.mjs @@ -1,37 +1,58 @@ import assert from "node:assert/strict"; +import { constants } from "node:buffer"; import { spawn, spawnSync } from "node:child_process"; import { + cpSync, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, + renameSync, rmSync, + statSync, + symlinkSync, + utimesSync, writeFileSync, } from "node:fs"; import { tmpdir } from "node:os"; -import { delimiter, dirname, join } from "node:path"; +import { basename, delimiter, dirname, join, posix, win32 } from "node:path"; import { test } from "node:test"; import { setTimeout as sleep } from "node:timers/promises"; -import { fileURLToPath } from "node:url"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { crc32, inflateSync } from "node:zlib"; import { - findTlc, + checkedOnlyInInitialStates, + groupClassDef, parseDot, parseMermaid, parseTraces, parseValue, ruleSentences, stateKey, + STATUS_CLASSES, tlcOutputWarning, uncitedConditions, } from "../lib.mjs"; +import { + CACHE_NAME, + cacheDir, + fcMatch, + fontMeasurer, + loadRenderer, + renderChart, + THEMES, +} from "../render.mjs"; +import { ruleBook } from "../rule-book.mjs"; +import { copyPath } from "../snapshot.mjs"; +import { findTlc, JAVA_TOOL, TLC_TOOL } from "../tlc-tools.mjs"; const here = dirname(fileURLToPath(import.meta.url)); const scripts = dirname(here); const fixture = (name) => readFileSync(join(here, name), "utf8"); const edges = (r) => r.transitions.map((t) => `${t.from} -> ${t.to} : ${t.label}`).sort(); -const node = (script, args, cwd, env = {}) => { - const r = spawnSync(process.execPath, [join(scripts, script), ...args], { +const node = (script, args, cwd, env = {}, at = scripts) => { + const r = spawnSync(process.execPath, [join(at, script), ...args], { cwd, encoding: "utf8", env: { ...process.env, ...env }, @@ -40,6 +61,17 @@ const node = (script, args, cwd, env = {}) => { r.stdout = r.stdout?.replace(/^( {2}TLC ran \d+ times? in )\d+\.\d s\.$/gm, "$1N s."); return r; }; +const statusOf = (chart) => { + const names = new Map( + [...chart.matchAll(/^ {2}state "([^"]+)" as (s\d+)$/gm)].map((m) => [ + m[2], + m[1].replace(//g, " "), + ]), + ); + return Object.fromEntries( + [...chart.matchAll(/^ {2}class (s\d+) (\w+)$/gm)].map((m) => [names.get(m[1]), m[2]]), + ); +}; test("a transition on a composite leaves from every state inside it", () => { const { regions } = parseMermaid(fixture("upload.mmd")); @@ -113,6 +145,283 @@ Bare == TRUE assert.equal(stateKey({ b: 1, a: { d: 2, c: [3] } }), stateKey({ a: { c: [3], d: 2 }, b: 1 })); }); +test("a rule defined in a module that the spec extends, directly or through another module, keeps its sentence and the step that breaks it", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "LampVars.tla"), + `---- MODULE LampVars ---- +VARIABLES light, wire +==== +`, + ); + writeFileSync( + join(dir, "LampRules.tla"), + `---- MODULE LampRules ---- +EXTENDS LampVars +\\* The light never comes on over a shorted wire. +SafeLight == wire = "Short" => light # "On" +==== +`, + ); + writeFileSync( + join(dir, "LampSteps.tla"), + `---- MODULE LampSteps ---- +EXTENDS LampRules +Short == wire = "Ok" /\\ wire' = "Short" /\\ UNCHANGED light +SwitchOn == light = "Off" /\\ light' = "On" /\\ UNCHANGED wire +==== +`, + ); + writeFileSync( + join(dir, "Lamp.tla"), + `---- MODULE Lamp ---- +EXTENDS Naturals, + LampSteps +Init == light = "Off" /\\ wire = "Ok" +Spec == Init /\\ [][Short \\/ SwitchOn]_<> +==== +`, + ); + const dotNode = (id, light, wire, init) => + `${id} [label="/\\\\ light = \\"${light}\\"\\n/\\\\ wire = \\"${wire}\\""${init ? ",style = filled" : ""}]`; + writeFileSync( + join(dir, "Lamp.dot"), + `strict digraph DiskGraph { +${dotNode(1, "Off", "Ok", true)} +${dotNode(2, "Off", "Short")} +${dotNode(3, "On", "Short")} +${dotNode(4, "On", "Ok")} +1 -> 2 [label="Short"]; +2 -> 3 [label="SwitchOn"]; +1 -> 4 [label="SwitchOn"]; +4 -> 3 [label="Short"]; +}`, + ); + writeFileSync( + join(dir, "Lamp.out"), + `Error: Invariant SafeLight is violated. +Error: The behavior up to this point is: +State 1: +/\\ light = "Off" +/\\ wire = "Ok" + +State 2: +/\\ light = "Off" +/\\ wire = "Short" + +State 3: +/\\ light = "On" +/\\ wire = "Short" + +${done}`, + ); + writeFileSync( + join(dir, "lamp.map.mjs"), + 'export const state = (v) => v.light;\nexport const step = (a) => ({ Short: "the wire shorts", SwitchOn: "you switch it on" })[a];\n', + ); + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "Lamp.dot", "--map", "lamp.map.mjs", "--traces", "Lamp.out"], + ...["--spec", "Lamp.tla", "--story", "story.md", "--out", "chart.mmd"], + ], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const story = readFileSync(join(dir, "story.md"), "utf8"); + assert.match(story, /^Rule broken: The light never comes on over a shorted wire\.$/m); + assert.match(story, /^1\. The wire shorts\. Now: Off\n2\. You switch it on\. Now: On$/m); +}); + +const lampRules = `---- MODULE LampRules ---- +VARIABLES light, wire +\\* The light never comes on over a shorted wire. +SafeLight == wire = "Short" => light # "On" +\\* The wire never shorts. +NeverShorts == [](wire # "Short") +==== +`; +const lampSteps = `Init == lamp = "Off" /\\ cable = "Ok" +Short == cable = "Ok" /\\ cable' = "Short" /\\ UNCHANGED lamp +SwitchOn == lamp = "Off" /\\ lamp' = "On" /\\ UNCHANGED cable +Spec == Init /\\ [][Short \\/ SwitchOn]_<> +==== +`; +const lampMap = + 'export const state = (v) => v.lamp;\nexport const step = (a) => ({ Short: "the wire shorts", SwitchOn: "you switch it on" })[a];\n'; + +test("a rule that the spec pulls in through an unnamed INSTANCE with substitutions keeps its sentence and the step that breaks it", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "LampRules.tla"), lampRules); + writeFileSync( + join(dir, "Lamp.tla"), + `---- MODULE Lamp ---- +VARIABLES lamp, cable +INSTANCE LampRules + WITH light <- lamp, + wire <- cable +${lampSteps}`, + ); + const dotNode = (id, lamp, cable, init) => + `${id} [label="/\\\\ cable = \\"${cable}\\"\\n/\\\\ lamp = \\"${lamp}\\""${init ? ",style = filled" : ""}]`; + writeFileSync( + join(dir, "Lamp.dot"), + `strict digraph DiskGraph { +${dotNode(1, "Off", "Ok", true)} +${dotNode(2, "Off", "Short")} +${dotNode(3, "On", "Short")} +${dotNode(4, "On", "Ok")} +1 -> 2 [label="Short"]; +2 -> 3 [label="SwitchOn"]; +1 -> 4 [label="SwitchOn"]; +4 -> 3 [label="Short"]; +}`, + ); + writeFileSync( + join(dir, "Lamp.out"), + `Error: Invariant SafeLight is violated. +Error: The behavior up to this point is: +State 1: +/\\ cable = "Ok" +/\\ lamp = "Off" + +State 2: +/\\ cable = "Short" +/\\ lamp = "Off" + +State 3: +/\\ cable = "Short" +/\\ lamp = "On" + +${done}`, + ); + writeFileSync(join(dir, "lamp.map.mjs"), lampMap); + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "Lamp.dot", "--map", "lamp.map.mjs", "--traces", "Lamp.out"], + ...["--spec", "Lamp.tla", "--story", "story.md", "--out", "chart.mmd"], + ], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const story = readFileSync(join(dir, "story.md"), "utf8"); + assert.match(story, /^Rule broken: The light never comes on over a shorted wire\.$/m); + assert.match(story, /^1\. The wire shorts\. Now: Off\n2\. You switch it on\. Now: On$/m); +}); + +test("rules reached through nested EXTENDS and INSTANCE keep their sentences and read the spec's variables behind each substitution", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const module = (name, body) => + writeFileSync(join(dir, `${name}.tla`), `---- MODULE ${name} ----\n${body}\n====\n`); + module("Wire", `VARIABLE wire\n\\* The wire is never cut.\nIntact == wire # "Cut"`); + module("Socket", `VARIABLE power\n\\* The socket always has power.\nLive == power = "On"`); + module("Count", `VARIABLE v\n\\* The count never goes negative.\nOk == v >= 0`); + module( + "LampRules", + `EXTENDS Naturals, Wire\nVARIABLE light\nPlug == INSTANCE Socket WITH power <- light\nLOCAL INSTANCE Count WITH v <- light\n\\* The light never comes on over a shorted wire.\nSafeLight == wire = "Short" => light # "On"`, + ); + module( + "Lamp", + `EXTENDS TLC +VARIABLES lamp, wire +shown == lamp +INSTANCE LampRules WITH light <- shown +Rules == INSTANCE + LampRules WITH light <- lamp, wire <- lamp +Quoted == INSTANCE Wire WITH wire <- IF lamp = "Cut<-" THEN wire ELSE "Cut" +Plain == Intact +Chain == Plain +Safe == Rules!SafeLight +Ok == wire # "Short" +\\* Its own words. +Worded == Rules!SafeLight`, + ); + const book = ruleBook(join(dir, "Lamp.tla")); + const reads = (rule) => [...book.get(rule).reads].sort(); + assert.deepEqual(reads("SafeLight"), ["lamp", "wire"]); + assert.deepEqual(reads("Intact"), ["wire"]); + assert.deepEqual(reads("Plug!Live"), ["lamp"]); + assert.deepEqual(reads("Rules!SafeLight"), ["lamp"]); + assert.deepEqual(reads("Worded"), ["lamp"]); + assert.deepEqual(reads("Quoted!Intact"), ["lamp", "wire"]); + assert.deepEqual(reads("Ok"), ["wire"]); + const rules = [ + "SafeLight", + "Rules!SafeLight", + "Plug!Live", + "Plain", + "Chain", + "Safe", + "Ok", + "Worded", + ]; + assert.deepEqual( + rules.map((rule) => book.get(rule).sentence), + [ + "The light never comes on over a shorted wire.", + "The light never comes on over a shorted wire.", + "The socket always has power.", + "The wire is never cut.", + "The wire is never cut.", + "The light never comes on over a shorted wire.", + undefined, + "Its own words.", + ], + ); +}); + +test("a LOCAL INSTANCE or LOCAL definition stays inside the module that declares it, and a public name that starts with LOCAL does not", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const module = (name, body) => + writeFileSync(join(dir, `${name}.tla`), `---- MODULE ${name} ----\n${body}\n====\n`); + module("Glow", `VARIABLE g\n\\* The glow is bright.\nOk == g = "Bright"`); + module("Dim", `VARIABLE d\n\\* The glow is dim.\nOk == d = "Dim"`); + module( + "Base", + `VARIABLE b\nLOCAL INSTANCE Dim WITH d <- b\nLOCAL Lit == b = "On"\n\\* The base is lit or dim.\nShown == Lit \\/ Ok\n\\* The base is never off.\nLOCALOn == b # "Off"`, + ); + module("Room", `EXTENDS Base\nVARIABLE r\nINSTANCE Glow WITH g <- r\nHall == INSTANCE Base`); + const book = ruleBook(join(dir, "Room.tla")); + assert.deepEqual(book.get("Ok").sentence, "The glow is bright."); + assert.deepEqual([...book.get("Ok").reads], ["r"]); + assert.deepEqual(book.get("Shown").sentence, "The base is lit or dim."); + assert.deepEqual([...book.get("Shown").reads], ["b"]); + assert.deepEqual([...book.get("Hall!Shown").reads], ["b"]); + assert.deepEqual(book.get("LOCALOn")?.sentence, "The base is never off."); + assert.deepEqual(book.get("Hall!LOCALOn")?.sentence, "The base is never off."); + assert.deepEqual( + ["Lit", "Hall!Lit", "Hall!Ok"].filter((rule) => book.has(rule)), + [], + ); +}); + +test("a rule that reads a CONSTANT reads the spec's variables that WITH puts in its place", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const module = (name, body) => + writeFileSync(join(dir, `${name}.tla`), `---- MODULE ${name} ----\n${body}\n====\n`); + module( + "Limit", + `EXTENDS Naturals\nCONSTANTS Max, Fits(_, _)\nVARIABLE level\n\\* The level stays under the cap.\nUnder == Fits(level, Max)`, + ); + module("Capped", `EXTENDS Limit\nVARIABLE spare\nRoom == spare + level <= Max`); + module( + "Tank", + `EXTENDS Naturals +VARIABLES tank, cap, gap +Gauge == INSTANCE Limit WITH level <- tank, Max <- cap, Fits <- LAMBDA a, b : a <= b +Fixed == INSTANCE Limit WITH level <- tank, Max <- 3, Fits <- LAMBDA a, b : a <= b +Shell == INSTANCE Capped WITH level <- tank, spare <- gap, Max <- cap + 1, Fits <- LAMBDA a, b : a <= b`, + ); + const book = ruleBook(join(dir, "Tank.tla")); + const reads = (rule) => [...book.get(rule).reads].sort(); + assert.deepEqual(reads("Gauge!Under"), ["cap", "tank"]); + assert.deepEqual(reads("Fixed!Under"), ["tank"]); + assert.deepEqual(reads("Shell!Under"), ["cap", "tank"]); + assert.deepEqual(reads("Shell!Room"), ["cap", "gap", "tank"]); + assert.deepEqual(book.get("Gauge!Under").sentence, "The level stays under the cap."); +}); + test("an action's condition on a variable that the action does not change needs a file and line", () => { const found = uncitedConditions(`---- MODULE Pay ---- CONSTANT MaxAge @@ -145,6 +454,78 @@ Next == Tick \\/ Pay \\/ Fill ]); }); +test("a rule reads the variables its helpers read, and not the ones its actions only change or leave unchanged", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Door.tla"), + `---- MODULE Door ---- +VARIABLES door, lock, bell +rest == <> +Locked == lock = "on" +NeverOpenWhileLocked == ~(door = "open" /\\ Locked) +Open == door = "shut" /\\ door' = "open" /\\ bell' = TRUE /\\ UNCHANGED lock +Close == door = "open" /\\ door' = "shut" /\\ UNCHANGED rest +System == Open \\/ Close +Settled == ~ENABLED System +====`, + ); + const book = ruleBook(join(dir, "Door.tla")); + assert.deepEqual([...book.get("NeverOpenWhileLocked").reads].sort(), ["door", "lock"]); + assert.deepEqual([...book.get("Settled").reads], ["door"]); +}); + +test("a property is checked only in the initial states when no definition it reaches has a temporal operator", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "LampBase.tla"), + `---- MODULE LampBase ---- +VARIABLE on +Safe == [](on \\in BOOLEAN) +==== +`, + ); + writeFileSync( + join(dir, "Lamp.tla"), + `---- MODULE Lamp ---- +VARIABLE on +vars == <> +Base == INSTANCE LampBase +StaysOff == on = FALSE +NotEmpty == vars /= <<>> +Always == [](on \\in BOOLEAN) +Eventually == <>on +LeadsTo == on ~> ~on +WhilePlus == StaysOff -+-> StaysOff +Weak == WF_vars(on' = ~on) +Strong == SF_vars(on' = ~on) +Some == \\EE x : StaysOff +Every == \\AA x : StaysOff +ByHelper == Always +FromBase == Base!Safe +==== +`, + ); + const names = [ + "StaysOff", + "NotEmpty", + "Always", + "Eventually", + "LeadsTo", + "WhilePlus", + "Weak", + "Strong", + "Some", + "Every", + "ByHelper", + "FromBase", + "Missing", + ]; + assert.deepEqual( + names.filter((name) => checkedOnlyInInitialStates(join(dir, "Lamp.tla"), name)), + ["StaysOff", "NotEmpty"], + ); +}); + test("the legend is not part of the machine", () => { const { regions } = parseMermaid(`stateDiagram-v2 state "Legend" as legend { @@ -308,10 +689,6 @@ test("TLC output from a run that failed or stopped early is refused", () => { () => tlcOutputWarning("Error: Parsing or semantic analysis failed.\n", "parse.out"), /not a broken rule/, ); - assert.throws( - () => tlcOutputWarning(evaluationError, "eval.out"), - /TLC threw an unexpected exception/, - ); assert.throws( () => tlcOutputWarning( @@ -320,10 +697,6 @@ test("TLC output from a run that failed or stopped early is refused", () => { ), /did not finish/, ); - assert.deepEqual( - parseTraces(evaluationError).map((t) => t.invariant), - ["NoLost", null], - ); }); const tlcThrew = (cls, message) => `Error: Evaluating invariant Small failed. @@ -341,32 +714,50 @@ const smallBroken = "Error: Invariant Small is violated.\nError: The behavior up to this point is:\nState 1: \nx = 0\n\n"; const stoppedEarly = "9 states generated, 9 distinct states found, 4 states left on queue.\n"; -test("a rule that TLC finds broken before TLC throws a Java exception keeps its paths, with a warning", () => { +test("a rule that TLC finds broken keeps its paths after any error that TLC then stops on, with a warning", () => { + const warned = (rule, error) => + `TLC found that ${rule} is broken, and then stopped on an error: ${error}. The paths that TLC printed before it stopped are complete, but TLC did not check every state, so the charts can miss states that break ${rule}. A smaller constant can let TLC finish.`; const aioobe = "java.lang.ArrayIndexOutOfBoundsException: Index 123 out of bounds for length 4"; const oneWorker = `${smallBroken}Error: Invariant Small is violated.\n${tlcThrew( "java.lang.ArrayIndexOutOfBoundsException", "Index 123 out of bounds for length 4", )}\n${stoppedEarly}`; - assert.equal( - tlcOutputWarning(oneWorker, "tlc.out"), - `TLC found that Small is broken, and then failed with ${aioobe}. TLC stopped before it checked every state, so the charts can miss states that break Small. The failure is in TLC, not in the rule. Do not weaken Small because of it. A smaller constant can let TLC finish.`, - ); + assert.equal(tlcOutputWarning(oneWorker, "tlc.out"), warned("Small", aioobe)); const workersInterleaved = `${tlcThrew("java.lang.RuntimeException", aioobe)}${smallBroken}${stoppedEarly}`; - assert.match( - tlcOutputWarning(workersInterleaved, "tlc.out"), - /failed with java\.lang\.ArrayIndexOutOfBoundsException: Index 123/, - ); + assert.equal(tlcOutputWarning(workersInterleaved, "tlc.out"), warned("Small", aioobe)); assert.deepEqual( parseTraces(workersInterleaved).map((t) => t.invariant), ["Small"], ); - assert.throws( - () => - tlcOutputWarning( - `${smallBroken}${tlcThrew("java.lang.RuntimeException", "util.Assert$TLCRuntimeException: Attempted to apply tuple")}${stoppedEarly}`, - "formula.out", - ), - /formula\.out: TLC stopped on an error that is not a broken rule: Evaluating invariant Small failed\./, + const arraycopy = (message) => + `${smallBroken}${tlcThrew("java.lang.ArrayIndexOutOfBoundsException", message)}${stoppedEarly}`; + const negative = "java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -3 is negative"; + assert.equal( + tlcOutputWarning(arraycopy("arraycopy: length -3 is negative"), "tlc.out"), + warned("Small", negative), + ); + assert.equal(tlcOutputWarning(arraycopy(negative), "tlc.out"), warned("Small", negative)); + assert.equal( + tlcOutputWarning( + `${smallBroken}${tlcThrew("java.lang.RuntimeException", "util.Assert$TLCRuntimeException: Attempted to apply tuple")}${stoppedEarly}`, + "formula.out", + ), + warned("Small", "util.Assert$TLCRuntimeException: Attempted to apply tuple"), + ); + assert.equal( + tlcOutputWarning( + `${smallBroken}Error: Invariant Small is violated.\nError: Failed to recover the initial state from its fingerprint.\nThis is probably a TLC bug(4).\n${stoppedEarly}`, + "fingerprint.out", + ), + warned("Small", "Failed to recover the initial state from its fingerprint"), + ); + assert.equal( + tlcOutputWarning(evaluationError, "eval.out"), + warned("NoLost", "TLC threw an unexpected exception"), + ); + assert.deepEqual( + parseTraces(evaluationError).map((t) => t.invariant), + ["NoLost"], ); assert.throws( () => @@ -407,6 +798,220 @@ export const step = (action) => ({ Tap: "you tap Save" })[action] ?? ""; ]); }); +test("a map keys on the bare action name when TLC 1.8.0 writes each step with its parameters", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "seats.dot"), + `strict digraph DiskGraph { +node [shape=box,style=rounded] +nodesep=0.35; +subgraph cluster_graph { +color="white"; +1 [label="seat = <<\\"free\\", \\"free\\">>",style = filled] +1 -> 2 [label="Take(1)",color="black",fontcolor="black"]; +2 [label="seat = <<\\"taken\\", \\"free\\">>",tooltip="seat = <<\\"taken\\", \\"free\\">>"]; +1 -> 3 [label="Take(2)",color="black",fontcolor="black"]; +3 [label="seat = <<\\"free\\", \\"taken\\">>",tooltip="seat = <<\\"free\\", \\"taken\\">>"]; +2 -> 1 [label="Leave(1)",color="black",fontcolor="black"]; +2 -> 4 [label="Take(2)",color="black",fontcolor="black"]; +4 [label="seat = <<\\"taken\\", \\"taken\\">>",tooltip="seat = <<\\"taken\\", \\"taken\\">>"]; +3 -> 4 [label="Take(1)",color="black",fontcolor="black"]; +3 -> 1 [label="Leave(2)",color="black",fontcolor="black"]; +4 -> 3 [label="Leave(1)",color="black",fontcolor="black"]; +4 -> 2 [label="Leave(2)",color="black",fontcolor="black"]; +{rank = same; 1;} +{rank = same; 3;2;} +{rank = same; 4;} +} +}`, + ); + writeFileSync( + join(dir, "seats.map.mjs"), + `export const readerActions = ["Take", "Leave"]; +const taken = (v) => v.seat.filter((s) => s === "taken").length; +export const state = (v) => ["Both free", "One taken", "Both taken"][taken(v)]; +export const step = (action) => ({ Take: "you sit down", Leave: "you get up" })[action] ?? action; +`, + ); + const draw = (...extra) => { + const r = node( + "tlc-to-chart.mjs", + ["--dot", "seats.dot", "--map", "seats.map.mjs", "--out", "chart.mmd", ...extra], + dir, + ); + assert.equal(r.status, 0, r.stderr); + return parseMermaid(readFileSync(join(dir, "chart.mmd"), "utf8")).regions.Main; + }; + assert.deepEqual(edges(draw()), [ + "Both free -> One taken : you sit down", + "Both taken -> One taken : you get up", + "One taken -> Both free : you get up", + "One taken -> Both taken : you sit down", + ]); + assert.deepEqual([...draw("--cover").states].sort(), ["Both free", "Both taken", "One taken"]); +}); + +function layeredHouse({ + extra = [], + edges = "", + state = '(v.x === "attic" ? "Upstairs" : "Downstairs")', + detail = '(v.x === "porch" || v.x === "hall" ? v.x : "")', +} = {}) { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "house.dot"), + `strict digraph DiskGraph { +1 [label="x = \\"start\\"",style = filled] +1 -> 2 [label="Side"]; +2 [label="x = \\"porch\\""]; +1 -> 3 [label="Front"]; +3 [label="x = \\"hall\\""]; +2 -> 4 [label="Arrive"]; +4 [label="x = \\"attic\\""]; +4 -> 1 [label="Leave"]; +${edges} +}`, + ); + writeFileSync( + join(dir, "house.map.mjs"), + `export const readerActions = ["Side", "Front", "Arrive"]; +export const state = (v) => ${state}; +export const detail = (v) => ${detail}; +export const step = (a) => ({ Side: "you take the side door", Front: "you take the front door", Arrive: "you climb the stairs", Leave: "the lights go out" })[a]; +`, + ); + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "house.dot", "--map", "house.map.mjs", "--view", "layered"], + ...["--out", "chart.mmd", ...extra], + ], + dir, + ); + assert.equal(r.status, 0, r.stderr); + return dir; +} + +test("--cover names a layered state with an empty detail by its outer label alone", () => { + const dir = layeredHouse({ extra: ["--cover", "--summary", "summary.json"] }); + const { cover } = JSON.parse(readFileSync(join(dir, "summary.json"), "utf8")); + assert.deepEqual(cover.leftOutStates, ["Upstairs"]); + assert.deepEqual(cover.leftOutTransitions, [ + "Downstairs: porch --> Upstairs: you climb the stairs [Arrive]", + "Upstairs --> Downstairs: the lights go out [Leave]", + ]); +}); + +test("a layered chart draws a state with an empty detail by its outer label alone, outside any box", () => { + const chart = readFileSync(join(layeredHouse(), "chart.mmd"), "utf8"); + assert.doesNotMatch(chart, /state ""/); + assert.match( + chart, + /^ {2}state "Downstairs" as g\d+ \{\n {4}state "hall" as s\d+\n {4}state "porch" as s\d+\n {2}\}$/m, + ); + const plain = /^ {2}state "Downstairs" as (s\d+)$/m.exec(chart)?.[1]; + assert.ok(plain, chart); + assert.match(chart, new RegExp(`^ {2}\\[\\*\\] --> ${plain}$`, "m")); + assert.match(chart, /^ {2}state "Upstairs" as s\d+$/m); +}); + +test("a box's shared step stays on each state when a plain state outside the box has the box's name", () => { + const chart = readFileSync( + join(layeredHouse({ edges: '3 -> 4 [label="Arrive"];' }), "chart.mmd"), + "utf8", + ); + assert.doesNotMatch(chart, /^ {2}g\d+ --> /m); + assert.deepEqual( + flatEdges(chart).map((e) => e.replaceAll("
", " ")), + [ + "Downstairs --> Downstairs: hall : you take the front door [Front]", + "Downstairs --> Downstairs: porch : you take the side door [Side]", + "Downstairs: hall --> Upstairs : you climb the stairs [Arrive]", + "Downstairs: porch --> Upstairs : you climb the stairs [Arrive]", + "Upstairs --> Downstairs : the lights go out [Leave]", + ], + ); +}); + +test("--flat draws a layered chart with no boxes, naming each state by its label and detail", () => { + const chart = readFileSync(join(layeredHouse({ extra: ["--flat"] }), "chart.mmd"), "utf8"); + assert.doesNotMatch(chart, / as g\d+ \{$/m); + assert.deepEqual([...chart.matchAll(/^ {2}state "([^"]+)" as s\d+$/gm)].map((m) => m[1]).sort(), [ + "Downstairs", + "Downstairs: hall", + "Downstairs: porch", + "Upstairs", + ]); +}); + +test("a layered chart and its --cover lists treat a blank state label or detail as absent", () => { + const dir = layeredHouse({ + extra: ["--cover", "--summary", "summary.json"], + state: '({ attic: "Upstairs", hall: " " })[v.x] ?? "Downstairs"', + detail: '(v.x === "porch" || v.x === "hall" ? v.x : " ")', + }); + const chart = readFileSync(join(dir, "chart.mmd"), "utf8"); + assert.doesNotMatch(chart, /state ""/); + assert.match(chart, /^ {2}state "Downstairs" as g\d+ \{\n {4}state "porch" as s\d+\n {2}\}$/m); + assert.deepEqual([...chart.matchAll(/^ {2}state "([^"]+)" as s\d+$/gm)].map((m) => m[1]).sort(), [ + "Downstairs", + "hall", + ]); + const { cover } = JSON.parse(readFileSync(join(dir, "summary.json"), "utf8")); + assert.deepEqual(cover.leftOutStates, ["Upstairs"]); + assert.deepEqual(cover.leftOutTransitions, [ + "Downstairs: porch --> Upstairs: you climb the stairs [Arrive]", + "Upstairs --> Downstairs: the lights go out [Leave]", + ]); +}); + +test("a large graph with a thousand bugs draws its overview and a bug chart in seconds", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const [states, bugs, bound] = [20_000, 1000, 10_000]; + const label = (n, broken) => `/\\\\ n = ${n}\\n/\\\\ broken = ${broken ? "TRUE" : "FALSE"}`; + const dot = []; + for (let i = 1; i <= states; i++) { + dot.push(`${i} [label="${label(i, false)}"${i === 1 ? ",style = filled" : ""}];`); + if (i < states) dot.push(`${i} -> ${i + 1} [label="Tick"];`); + dot.push(`${i} -> ${((i * 7) % states) + 1} [label="Jump"];`); + } + const traces = []; + for (let k = 1; k <= bugs; k++) { + dot.push(`${states + k} [label="${label(k, true)}"];`, `1 -> ${states + k} [label="Break"];`); + traces.push( + `Error: Invariant ${k % 2 ? "Safe" : "Calm"} is violated.\nError: The behavior up to this point is:\nState 1: \n/\\ n = 1\n/\\ broken = FALSE\n\nState 2: \n/\\ n = ${k}\n/\\ broken = TRUE\n`, + ); + } + writeFileSync(join(dir, "big.dot"), `strict digraph DiskGraph {\n${dot.join("\n")}\n}\n`); + writeFileSync(join(dir, "big.out"), `${traces.join("\n")}\n${done}`); + writeFileSync( + join(dir, "big.map.mjs"), + `export const state = (v) => (v.broken ? "Broken" : "Counting"); +export const step = (a) => ({ Tick: "a second passes", Jump: "it jumps", Break: "it breaks" })[a]; +export const cause = (v) => (v.broken ? \`counter \${v.n} broke\` : null); +`, + ); + for (const view of [ + "--highlight 0 --out chart.mmd --story stories.md --summary summary.json", + "--highlight 1 --path-only --out bug-1.mmd", + ]) { + const input = `--dot big.dot --map big.map.mjs --traces big.out ${view}`.split(" "); + const r = spawnSync(process.execPath, [join(scripts, "tlc-to-chart.mjs"), ...input], { + cwd: dir, + encoding: "utf8", + timeout: bound, + }); + assert.equal(r.signal, null, `${view} ran past ${bound / 1000} s`); + assert.equal(r.status, 0, r.stderr); + } + const { classes } = JSON.parse(readFileSync(join(dir, "summary.json"), "utf8")); + assert.equal(classes.length, bugs); + assert.deepEqual( + new Set(classes.map((c) => JSON.stringify([c.lastSteps, c.startsHere]))), + new Set([JSON.stringify([["it breaks"], false])]), + ); +}); + test("chart-to-tla writes one action per label and one invariant per never line", () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); @@ -637,16 +1242,19 @@ FlipTheSwitch == ); }); -test("compare reads a one-region chart as the machine its steps reach, without steps that change nothing", () => { +test("compare reads a one-region chart as the machine its steps reach, with the steps that keep the state", () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); const drawn = "stateDiagram-v2\n [*] --> Idle\n Idle --> Busy : start\n Busy --> Idle : finish\n"; writeFileSync(join(dir, "drawn.mmd"), drawn); writeFileSync(join(dir, "loop.mmd"), `${drawn} Idle --> Idle : refresh\n`); writeFileSync(join(dir, "orphan.mmd"), `${drawn} Lost --> Idle : come back\n`); - const same = node("compare-charts.mjs", ["loop.mmd", "drawn.mmd"], dir); - assert.equal(same.status, 0); - assert.equal(same.stdout, "same\n"); + const loop = node("compare-charts.mjs", ["loop.mmd", "drawn.mmd"], dir); + assert.equal(loop.status, 1); + assert.equal( + loop.stdout, + "transitions only in EXPECTED (a guard in the spec blocks them, or the spec renamed the step):\n Idle --> Idle : refresh\n1 difference(s)\n", + ); const orphan = node("compare-charts.mjs", ["orphan.mmd", "drawn.mmd"], dir); assert.equal(orphan.status, 1); assert.equal( @@ -796,15 +1404,14 @@ export const step = (a) => ({ TapPay: "you tap Pay", TapBack: "you tap Back", Ta assert.equal(node("compare-charts.mjs", ["agreed.mmd", "chart.mmd"], dir).stdout, "same\n"); }); -const NO_TLC = `Cannot find TLC. Download tla2tools.jar from a TLA+ release: - https://github.com/tlaplus/tlaplus/releases -Then set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later). -`; +const NO_TLC = + "Cannot find TLC. It installs with mise, which cannot start: spawnSync mise ENOENT. Install mise (https://mise.jdx.dev), or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later).\n"; const FROM_SPEC_USAGE = - "node statechart.mjs from-spec SPEC.tla [--config FILE] [--fixed FIXED.tla [--fixed-config FILE] | --against AGREED.tla [--against-config FILE]] [--label TEXT] [--map FILE] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png]"; + "node statechart.mjs from-spec SPEC.tla [--config FILE] [--fixed FIXED.tla [--fixed-config FILE] | --against AGREED.tla [--against-config FILE]] [--label TEXT] [--map FILE] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--cover] [--png [--theme NAME]]"; const FROM_CHART_USAGE = - "node statechart.mjs from-chart CHART.mmd [--module NAME] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png]"; -const noTlc = { TLA2TOOLS: "", JAVA: "" }; + "node statechart.mjs from-chart CHART.mmd [--module NAME] [--expect-broken RULE]... [--title TEXT] [--out DIR] [--png [--theme NAME]]"; +const RENDER_USAGE = "node statechart.mjs render CHART.mmd [--theme NAME] [--out FILE]"; +const noTlc = { PATH: mkdtempSync(join(tmpdir(), "empty-path-")), TLA2TOOLS: "", JAVA: "" }; test("statechart.mjs writes the spec next to the chart, keeps it on the next run, and stops without TLC", () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); @@ -836,6 +1443,43 @@ test("statechart.mjs writes the spec next to the chart, keeps it on the next run missingJar.stderr, "TLA2TOOLS is /nonexistent/tla2tools.jar, which does not exist.\n", ); + const missingJava = node("statechart.mjs", ["from-spec", "NotesApp.tla"], dir, { + ...noTlc, + JAVA: "/nonexistent/java", + }); + assert.equal(missingJava.status, 2); + assert.equal(missingJava.stderr, "JAVA is /nonexistent/java, which does not exist.\n"); +}); + +const installCommand = (...tools) => + `cd "$(mktemp -d)" && MISE_CEILING_PATHS="$PWD" mise install ${tools.map((t) => `'${t}'`).join(" ")}`; + +test("statechart.mjs shows mise's own error when mise fails before it can look for TLC", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const bin = mkdtempSync(join(tmpdir(), "fake-mise-")); + const said = + "mise ERROR failed to read the global config\nmise ERROR Run with --verbose for more"; + writeFileSync(join(bin, "mise"), `#!/bin/sh\nprintf '%s\\n' "${said}" >&2\nexit 1\n`, { + mode: 0o755, + }); + writeFileSync(join(dir, "Pay.tla"), "---- MODULE Pay ----\n====\n"); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\n"); + writeFileSync(join(dir, "Pay.map.mjs"), "export default {};\n"); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, { + PATH: bin, + TLA2TOOLS: "", + JAVA: "", + }); + assert.equal(r.status, 2); + assert.equal( + r.stderr, + `Cannot find TLC, because mise failed: + mise ERROR failed to read the global config + mise ERROR Run with --verbose for more +The install command: ${installCommand(JAVA_TOOL, TLC_TOOL)} +Fix that, or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later). +`, + ); }); test("statechart.mjs writes no output folder when it refuses the input", () => { @@ -919,17 +1563,11 @@ test("statechart.mjs names the chart it refuses and the argument it does not kno unknown.stderr, "unknown argument --fixed. Run node statechart.mjs from-chart --help for the flags, the output, and the exit codes.\n", ); - const sameName = node("statechart.mjs", ["from-spec", "C.tla", "--fixed", "fix/C.tla"], dir); - assert.equal(sameName.status, 2); - assert.equal( - sameName.stderr, - "fix/C.tla needs its own file and module name, such as CFixed.tla.\n", - ); - const sameAgreed = node("statechart.mjs", ["from-spec", "C.tla", "--against", "a/C.tla"], dir); - assert.equal( - sameAgreed.stderr, - "a/C.tla needs its own file and module name, such as CAgreed.tla.\n", - ); + const sameFile = node("statechart.mjs", ["from-spec", "C.tla", "--fixed", "./C.tla"], dir); + assert.equal(sameFile.status, 2); + assert.equal(sameFile.stderr, "--fixed needs a file other than the spec, C.tla.\n"); + const sameAgreed = node("statechart.mjs", ["from-spec", "C.tla", "--against", "C.tla"], dir); + assert.equal(sameAgreed.stderr, "--against needs a file other than the spec, C.tla.\n"); for (const [extra, stderr] of [ [ ["--fixed", "F.tla", "--against", "A.tla"], @@ -950,25 +1588,45 @@ test("statechart.mjs names the chart it refuses and the argument it does not kno valueless.stderr, "--out needs a value. Run node statechart.mjs from-chart --help for the flags, the output, and the exit codes.\n", ); + for (const [args, stderr] of [ + [ + ["render", "c.mmd", "--theme", "blue"], + "--theme takes light, dark, or classic. Run node statechart.mjs render --help for the flags, the output, and the exit codes.\n", + ], + [ + ["from-chart", "c.mmd", "--theme", "dark"], + "--theme needs --png. Run node statechart.mjs from-chart --help for the flags, the output, and the exit codes.\n", + ], + [ + ["render", "c.mmd", "--png"], + "unknown argument --png. Run node statechart.mjs render --help for the flags, the output, and the exit codes.\n", + ], + [ + ["render"], + `Usage: ${RENDER_USAGE}\nRun node statechart.mjs render --help for the flags, the output, and the exit codes.\n`, + ], + ]) { + const r = node("statechart.mjs", args, dir); + assert.equal(r.status, 2, args.join(" ")); + assert.equal(r.stderr, stderr); + } const bare = node("statechart.mjs", [], dir); assert.equal(bare.status, 2); assert.equal( bare.stderr, `Usage: ${FROM_SPEC_USAGE} ${FROM_CHART_USAGE} + ${RENDER_USAGE} Run node statechart.mjs --help for the flags, the output, and the exit codes. `, ); }); -test("every script prints its help with -h or --help, and points at it for an argument it does not know", () => { +test("statechart.mjs prints its help with -h or --help and points at it for an argument it does not know, and each inner script refuses that argument", () => { const help = { "statechart.mjs": [[], `Usage: ${FROM_SPEC_USAGE}`], "statechart.mjs from-spec": [["from-spec"], `Usage: ${FROM_SPEC_USAGE}`], "statechart.mjs from-chart": [["from-chart", "notes-app.mmd"], `Usage: ${FROM_CHART_USAGE}`], - "chart-to-tla.mjs": [[], "Usage: node chart-to-tla.mjs CHART.mmd --module NAME [--dir OUT]"], - "tlc-to-chart.mjs": [[], "Usage: node tlc-to-chart.mjs --dot FILE --map FILE [options]"], - "compare-charts.mjs": [[], "Usage: node compare-charts.mjs EXPECTED.mmd ACTUAL.mmd"], }; for (const [name, [before, usage]] of Object.entries(help)) { const script = name.split(" ")[0]; @@ -982,7 +1640,8 @@ test("every script prints its help with -h or --help, and points at it for an ar } const top = node("statechart.mjs", ["--help"], here).stdout.split("\n"); assert.equal(top[1], ` ${FROM_CHART_USAGE}`); - assert.equal(top[2], " node statechart.mjs [from-spec|from-chart] --help"); + assert.equal(top[2], ` ${RENDER_USAGE}`); + assert.equal(top[3], " node statechart.mjs [from-spec|from-chart|render] --help"); for (const flag of [ "--fixed FIXED.tla", "--label TEXT", @@ -1003,31 +1662,9 @@ test("every script prints its help with -h or --help, and points at it for an ar ["from-spec", "--nope"], "unknown argument --nope. Run node statechart.mjs from-spec --help for the flags, the output, and the exit codes.\n", ], - [ - "chart-to-tla.mjs", - ["--nope"], - "unknown argument --nope. Run node chart-to-tla.mjs --help for the flags.\n", - ], - [ - "tlc-to-chart.mjs", - ["--nope"], - "unknown argument --nope. Run node tlc-to-chart.mjs --help for the flags.\n", - ], - [ - "tlc-to-chart.mjs", - ["--dot", "a.dot", "--map", "a.map.mjs", "--regions"], - "--regions draws the whole customer chart: use --highlight 0, without --flat or --path-only. Run node tlc-to-chart.mjs --help for the flags.\n", - ], - [ - "tlc-to-chart.mjs", - ["--dot", "a.dot", "--map", "a.map.mjs", "--proof", "p.mmd"], - "--proof writes the regions proof; it needs --regions. Run node tlc-to-chart.mjs --help for the flags.\n", - ], - [ - "compare-charts.mjs", - ["--nope"], - "unknown argument --nope. Run node compare-charts.mjs --help for the flags.\n", - ], + ["chart-to-tla.mjs", ["--nope"], "unknown argument --nope.\n"], + ["tlc-to-chart.mjs", ["--nope"], "unknown argument --nope.\n"], + ["compare-charts.mjs", ["--nope"], "unknown argument --nope.\n"], ]) { const r = node(script, args, here); assert.equal(r.status, 2, script); @@ -1045,6 +1682,124 @@ const tools = (() => { })(); const needsTlc = { skip: tools ? false : "cannot find TLC" }; +const realEntry = pathToFileURL(join(cacheDir(), "entry.mjs")).href; +const lib = await loadRenderer().catch(() => null); +const needsPng = { skip: lib ? false : "cannot install Pele and resvg" }; + +function fakeRenderer() { + const xdg = mkdtempSync(join(tmpdir(), "xdg-")); + const dir = join(xdg, "statechart", CACHE_NAME); + mkdirSync(dir, { recursive: true }); + writeFileSync( + join(dir, "entry.mjs"), + `import { appendFileSync } from "node:fs"; +import { render as real } from ${JSON.stringify(realEntry)}; +export { Resvg } from ${JSON.stringify(realEntry)}; +export function render(text, options) { + const title = JSON.parse(/^title: (.*)$/m.exec(text)?.[1] ?? '""'); + if (process.env.RENDERED) appendFileSync(process.env.RENDERED, \`\${title}\\n\`); + if (title && process.env.UNRENDERED?.split("|").includes(title)) + throw new Error("Pele cannot lay out this chart"); + return real(text, options); +} +`, + ); + return xdg; +} + +const paeth = (a, b, c) => { + const [pa, pb, pc] = [a, b, c].map((v) => Math.abs(a + b - c - v)); + return pa <= pb && pa <= pc ? a : pb <= pc ? b : c; +}; +const FILTERS = [() => 0, (a) => a, (a, b) => b, (a, b) => (a + b) >> 1, paeth]; + +function decodePng(png) { + assert.deepEqual([...png.subarray(0, 8)], [137, 80, 78, 71, 13, 10, 26, 10], "PNG signature"); + const chunks = {}; + for (let at = 8; at < png.length;) { + const size = png.readUInt32BE(at); + const type = png.toString("latin1", at + 4, at + 8); + const crc = crc32(png.subarray(at + 4, at + 8 + size)); + assert.equal(png.readUInt32BE(at + 8 + size), crc, `${type} chunk CRC`); + (chunks[type] ??= []).push(png.subarray(at + 8, at + 8 + size)); + at += 12 + size; + } + const [ihdr] = chunks.IHDR; + const [width, height] = [ihdr.readUInt32BE(0), ihdr.readUInt32BE(4)]; + assert.deepEqual([ihdr[8], ihdr[9], ihdr[12]], [8, 6, 0], "8-bit RGBA, not interlaced"); + const raw = inflateSync(Buffer.concat(chunks.IDAT)); + const stride = width * 4; + const px = Buffer.alloc(stride * height); + for (let y = 0; y < height; y++) { + const row = y * (stride + 1); + const filter = FILTERS[raw[row]]; + assert.ok(filter, `row ${y} has PNG filter ${raw[row]}`); + for (let x = 0; x < stride; x++) { + const a = x < 4 ? 0 : px[y * stride + x - 4]; + const b = y ? px[(y - 1) * stride + x] : 0; + const c = x < 4 || !y ? 0 : px[(y - 1) * stride + x - 4]; + px[y * stride + x] = raw[row + 1 + x] + filter(a, b, c); + } + } + const at = (x, y) => (Math.floor(y) * width + Math.floor(x)) * 4; + return { + width, + height, + pixel: (x, y) => `#${px.subarray(at(x, y), at(x, y) + 3).toString("hex")}`, + }; +} + +const xmlText = (s) => + s + .replace(/</g, "<") + .replace(/>/g, ">") + .replace(/"/g, '"') + .replace(/'/g, "'") + .replace(/&/g, "&"); +const textLines = (text) => { + const spans = [...text.matchAll(/]*>([^<]*)<\/tspan>/g)].map((m) => m[1]); + return (spans.length ? spans : [text.replace(/<[^>]+>/g, "")]) + .map(xmlText) + .filter((l) => l.trim()); +}; +const width = (rect) => +/\swidth="([\d.]+)"/.exec(rect)[1]; + +function labeledBoxes(svg) { + const states = [ + ...svg.matchAll( + /]*>(]*\/>)([\s\S]*?)<\/g>/g, + ), + ].flatMap(([, id, rect, inside]) => { + const text = /]*>[\s\S]*?<\/text>/.exec(inside); + return text + ? [{ kind: "state", id, box: width(rect), size: 16, lines: textLines(text[0]) }] + : []; + }); + const edges = [ + ...svg.matchAll( + /(]*\/>)()/g, + ), + ].map(([, id, rect, text]) => ({ + kind: "edge label", + id, + box: width(rect), + size: 14, + lines: textLines(text), + })); + const regions = [ + ...svg.matchAll( + /(]*\/>)[\s\S]*?()/g, + ), + ].map(([, id, rect, text]) => ({ + kind: "region title", + id, + box: width(rect), + size: 16, + lines: textLines(text), + })); + return [...states, ...edges, ...regions]; +} + function tlc(dir, name) { const r = spawnSync( tools.java, @@ -1069,6 +1824,246 @@ function tlc(dir, name) { return r; } +const FAKE_MISE = `#!/bin/sh +echo "$PWD $*" >> "$MISE_LOG" +d=$PWD +while :; do + grep -qs '^locked = true' "$d/mise.toml" && locked=1 + case ":$MISE_CEILING_PATHS:" in *":$d:"*) break;; esac + [ "$d" = / ] && break + d=$(dirname "$d") +done +case $1 in +install) + if [ -n "$FAIL" ]; then echo "mise ERROR cannot download tla2tools.jar" >&2; exit 1; fi + if [ -n "$locked" ]; then echo "mise ERROR No lockfile URL found for $3 (--locked mode)" >&2; exit 1; fi + touch "$MISE_STATE/installed" + if [ -n "$FAIL_AFTER_INSTALL" ]; then echo "mise ERROR failed to rebuild runtime symlinks: File exists (os error 17)" >&2; exit 1; fi + exit 0;; +where) if [ -e "$MISE_STATE/installed" ]; then echo "$TLAPLUS_HOME"; exit 0; fi;; +which) if [ -e "$MISE_STATE/installed" ]; then echo "$TLC_JAVA"; exit 0; fi;; +esac +echo "mise ERROR $2 is not installed" >&2 +exit 1 +`; + +function outsideRepoWithoutTlc() { + const root = mkdtempSync(join(tmpdir(), "statechart-outside-")); + const at = join(root, "skill", "scripts"); + cpSync(scripts, at, { recursive: true, filter: (f) => !f.startsWith(here) }); + const bin = join(root, "bin"); + const state = join(root, "state"); + const dir = join(root, "work"); + for (const d of [bin, state, dir]) mkdirSync(d); + writeFileSync(join(bin, "mise"), FAKE_MISE, { mode: 0o755 }); + writeFileSync(join(dir, "mise.toml"), '[tools]\njava = "17"\n'); + const log = join(root, "mise.log"); + const env = { + PATH: `${bin}${delimiter}${process.env.PATH}`, + TLA2TOOLS: "", + JAVA: "", + MISE_STATE: state, + MISE_LOG: log, + }; + const calls = () => + existsSync(log) + ? readFileSync(log, "utf8") + .trim() + .split("\n") + .map((line) => { + const [folder, ...args] = line.split(" "); + return { folder, args }; + }) + : []; + return { at, dir, env, calls }; +} + +function assertMiseRanInARemovedEmptyFolder(dir, calls) { + const folders = [...new Set(calls.map((c) => c.folder))]; + assert.equal(folders.length > 0, true); + for (const folder of folders) { + assert.equal(folder.startsWith(dir), false, folder); + assert.equal(existsSync(folder), false, folder); + } +} + +const INSTALLING = "Installing TLC and Java with mise. This happens once.\n"; + +test("statechart.mjs outside the repo installs TLC with mise in an empty folder once and shows mise's error", () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + writeFileSync(join(dir, "Pay.tla"), "---- MODULE Pay ----\n====\n"); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\n"); + writeFileSync(join(dir, "Pay.map.mjs"), "export default {};\n"); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, { ...env, FAIL: "1" }, at); + assert.equal(r.status, 2); + assert.equal( + r.stderr, + `${INSTALLING}Cannot find TLC, and mise could not install it: + mise ERROR cannot download tla2tools.jar +The install command: ${installCommand(JAVA_TOOL, TLC_TOOL)} +Or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later). +`, + ); + const install = ["install", "--quiet", JAVA_TOOL, TLC_TOOL]; + assert.deepEqual( + calls().map((c) => c.args), + [ + ["where", TLC_TOOL], + ["which", "java", "--tool", JAVA_TOOL], + install, + ["where", TLC_TOOL], + ["which", "java", "--tool", JAVA_TOOL], + ], + ); + assertMiseRanInARemovedEmptyFolder(dir, calls()); +}); + +test("statechart.mjs with JAVA set installs only TLC with mise", () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + writeFileSync(join(dir, "Pay.tla"), "---- MODULE Pay ----\n====\n"); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\n"); + writeFileSync(join(dir, "Pay.map.mjs"), "export default {};\n"); + const javaEnv = { ...env, JAVA: process.execPath, FAIL: "1" }; + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, javaEnv, at); + assert.equal(r.status, 2); + assert.equal( + r.stderr, + `Installing TLC with mise. This happens once. +Cannot find TLC, and mise could not install it: + mise ERROR cannot download tla2tools.jar +The install command: ${installCommand(TLC_TOOL)} +Or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later). +`, + ); + assert.deepEqual( + calls().map((c) => c.args[0]), + ["where", "install", "where"], + ); + assertMiseRanInARemovedEmptyFolder(dir, calls()); +}); + +test( + "statechart.mjs outside the repo installs a missing TLC with mise once, and leaves the mise files next to the spec alone", + needsTlc, + () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + const tlcEnv = { ...env, TLAPLUS_HOME: dirname(tools.jar), TLC_JAVA: tools.java }; + writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); + const first = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, tlcEnv, at); + assert.equal(first.status, 0, first.stderr); + assert.equal(first.stderr, INSTALLING); + assert.match(first.stdout, /\nThe spec draws upload\.mmd back unchanged\.\n/); + const again = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, tlcEnv, at); + assert.equal(again.status, 0, again.stderr); + assert.equal(again.stderr, ""); + assert.equal(calls().filter((c) => c.args[0] === "install").length, 1); + assertMiseRanInARemovedEmptyFolder(dir, calls()); + }, +); + +test( + "statechart.mjs uses TLC that mise installed even when mise install then exits non-zero", + needsTlc, + () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + const tlcEnv = { + ...env, + TLAPLUS_HOME: dirname(tools.jar), + TLC_JAVA: tools.java, + FAIL_AFTER_INSTALL: "1", + }; + writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); + const r = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, tlcEnv, at); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, INSTALLING); + assert.match(r.stdout, /\nThe spec draws upload\.mmd back unchanged\.\n/); + assert.equal(calls().filter((c) => c.args[0] === "install").length, 1); + }, +); + +test( + "statechart.mjs installs TLC with mise inside a repo whose mise.toml sets locked = true", + needsTlc, + () => { + const { at, dir, env, calls } = outsideRepoWithoutTlc(); + writeFileSync(join(dir, "mise.toml"), "[settings]\nlocked = true\n"); + const specs = join(dir, "specs"); + mkdirSync(specs); + writeFileSync(join(specs, "upload.mmd"), fixture("upload.mmd")); + const tlcEnv = { ...env, TLAPLUS_HOME: dirname(tools.jar), TLC_JAVA: tools.java }; + const r = node("statechart.mjs", ["from-chart", "upload.mmd"], specs, tlcEnv, at); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, INSTALLING); + assert.match(r.stdout, /\nThe spec draws upload\.mmd back unchanged\.\n/); + assertMiseRanInARemovedEmptyFolder(dir, calls()); + }, +); + +test( + "statechart.mjs finds TLC with the real mise when an untrusted mise.toml is next to the spec and above TMPDIR", + { + skip: + !tools || process.env.TLA2TOOLS + ? "needs TLC from mise, not from TLA2TOOLS" + : spawnSync("mise", ["--version"]).error + ? "needs mise" + : false, + }, + () => { + const root = mkdtempSync(join(tmpdir(), "statechart-outside-")); + const at = join(root, "scripts"); + cpSync(scripts, at, { recursive: true, filter: (f) => !f.startsWith(here) }); + const dir = join(root, "work"); + const tmp = join(dir, "tmp"); + mkdirSync(tmp, { recursive: true }); + writeFileSync(join(dir, "mise.toml"), '[tools]\njava = "17"\n\n[env]\nSTATECHART = "1"\n'); + writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); + const env = { JAVA: "", TMPDIR: tmp }; + const r = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, env, at); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, ""); + assert.match(r.stdout, /\nThe spec draws upload\.mmd back unchanged\.\n/); + assert.deepEqual(readdirSync(dir).sort(), [ + "Upload-statechart", + "Upload.cfg", + "Upload.map.mjs", + "Upload.tla", + "mise.toml", + "tmp", + "upload.mmd", + ]); + }, +); + +test("statechart.mjs shows why java cannot start TLC", needsTlc, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); + const setVars = + "Fix that, or set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later).\n"; + const folder = dirname(tools.jar); + const notJar = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, { + TLA2TOOLS: folder, + JAVA: tools.java, + }); + assert.equal(notJar.status, 2); + assert.equal( + notJar.stderr, + `cannot run TLC with ${tools.java} -cp ${folder}: java exited 1: + Error: Could not find or load main class tlc2.TLC + Caused by: java.lang.ClassNotFoundException: tlc2.TLC +${setVars}`, + ); + const noJava = node("statechart.mjs", ["from-chart", "upload.mmd"], dir, { + ...noTlc, + TLA2TOOLS: tools.jar, + }); + assert.equal(noJava.status, 2); + assert.equal( + noJava.stderr, + `cannot run TLC with java -cp ${tools.jar}: spawn java ENOENT\n${setVars}`, + ); +}); + test("from-chart checks the agreed chart and draws it back unchanged", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeFileSync(join(dir, "upload.mmd"), fixture("upload.mmd")); @@ -1084,7 +2079,7 @@ TLC on Upload.tla with Upload.cfg: Chart: 5 states and 8 transitions from 5 model states. 5 work, 0 break a rule. -No state breaks a rule. +No state breaks an INVARIANT rule. The spec draws upload.mmd back unchanged. @@ -1094,6 +2089,45 @@ Wrote chart.mmd, stories.md, summary.json in Upload-statechart. assert.equal(r.stderr, ""); }); +const ringing = `stateDiagram-v2 + [*] --> Idle + Idle --> Ringing : a call comes in + Ringing --> Ringing : it rings again + Ringing --> Idle : the caller hangs up +`; + +test( + "from-chart counts a self-loop, so a spec that drops it does not draw the chart back", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "call.mmd"), ringing); + const drawn = node("statechart.mjs", ["from-chart", "call.mmd"], dir); + assert.equal(drawn.status, 0, drawn.stdout + drawn.stderr); + assert.match(drawn.stdout, /\nThe spec draws call\.mmd back unchanged\.\n/); + + const spec = readFileSync(join(dir, "Call.tla"), "utf8"); + const dropped = spec.replace(" \\/ ItRingsAgain", ""); + assert.notEqual(dropped, spec); + writeFileSync(join(dir, "Call.tla"), dropped); + const r = node("statechart.mjs", ["from-chart", "call.mmd"], dir); + assert.equal(r.status, 1, r.stdout + r.stderr); + assert.match( + r.stdout, + /\nThe spec does not draw call\.mmd:\n {2}transitions only in EXPECTED [^\n]*\n {4}Ringing --> Ringing : it rings again\n {2}1 difference\(s\)\n/, + ); + }, +); + +test("from-chart draws a chart with no transitions back unchanged", needsTlc, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "lone.mmd"), "stateDiagram-v2\n [*] --> Idle\n"); + const r = node("statechart.mjs", ["from-chart", "lone.mmd"], dir); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match(r.stdout, /\nThe spec draws lone\.mmd back unchanged\.\n/); + assert.equal(r.stderr, ""); +}); + test("TLC reads a label with a guarded and an unguarded move in one region", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeFileSync(join(dir, "switch.mmd"), switchLamp); @@ -1105,7 +2139,7 @@ test("TLC reads a label with a guarded and an unguarded move in one region", nee const syncStory = `## 1. Offline | Synced Rule broken: You never see "Synced" while you are offline. -This is the only way to get here in the model. +In the model, one step leads into this bug: "the server confirms". 0. Start: Online | Synced 1. The network drops. Now: Offline | Waiting to sync @@ -1168,7 +2202,14 @@ Wrote chart.mmd, bug-1.mmd, stories.md, summary.json in Sync-statechart. `, ); const summary = JSON.parse(readFileSync(join(dir, "Sync-statechart", "summary.json"), "utf8")); - assert.deepEqual(summary.chartStatus, { works: 3, wrong: 0, fixed: 1 }); + assert.deepEqual(summary.chartStatus, { + works: 3, + wrong: 0, + fixed: 1, + mixed: 0, + after: 0, + worse: 0, + }); assert.equal(summary.classes[0].fix, "Offline | Waiting to sync"); const map = readFileSync(join(dir, "Sync.map.mjs"), "utf8"); @@ -1213,7 +2254,7 @@ TLC on Sync.tla with Sync.cfg: Chart: 3 states and 5 transitions from 3 model states. 3 work, 0 break a rule. -No state breaks a rule. +No state breaks an INVARIANT rule. The spec does not draw sync.mmd: states only in EXPECTED (no path from the start reaches them in the spec, or the spec renamed them): @@ -1237,155 +2278,845 @@ Wrote chart.mmd, stories.md, summary.json in Sync-statechart. }); test( - "--png exits 2 unless every chart renders, even with --expect-broken, and keeps the files it wrote", + "a second model with the module name of the spec, in its own folder, keeps its own TLC files", needsTlc, () => { - const args = ["from-chart", "sync.mmd", "--expect-broken", "NeverSyncedWhileOffline", "--png"]; - const bin = mkdtempSync(join(tmpdir(), "fake-npx-")); - const launch = "Error: Failed to launch the browser process!"; - writeFileSync( - join(bin, "npx"), - `#!/bin/sh -while [ $# -gt 0 ]; do - case "$1" in -i) chart=$2 ;; -o) png=$2 ;; esac - shift -done -for c in $UNRENDERED; do - case "$chart" in */$c.mmd) echo "${launch}" >&2; exit 1 ;; esac -done -echo png > "$png" -`, - { mode: 0o755 }, + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); + assert.equal(node("statechart.mjs", ["from-chart", "sync.mmd"], dir).status, 1); + const guarded = readFileSync(join(dir, "Sync.tla"), "utf8").replace( + "TheServerConfirms ==\n", + 'TheServerConfirms ==\n /\\ connection = "Online"\n', ); - for (const [unrendered, status, written] of [ - ["chart bug-1", 2, ["chart.mmd", "bug-1.mmd"]], - ["bug-1", 2, ["chart.mmd", "bug-1.mmd", "chart.png"]], - ["", 0, ["chart.mmd", "bug-1.mmd", "chart.png", "bug-1.png"]], - ]) { - const dir = mkdtempSync(join(tmpdir(), "statechart-")); - writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); - const r = node("statechart.mjs", args, dir, { - PATH: `${bin}${delimiter}${process.env.PATH}`, - UNRENDERED: unrendered, - }); - assert.equal(r.status, status, `${unrendered}: ${r.stdout}${r.stderr}`); - assert.equal( - r.stderr, - unrendered - .split(" ") - .filter(Boolean) - .map((c) => `could not render Sync-statechart/${c}.mmd: ${launch}\n`) - .join(""), - ); - assert.match(r.stdout, /\nBroken as expected: NeverSyncedWhileOffline\.\n/); + mkdirSync(join(dir, "fix")); + writeFileSync(join(dir, "fix", "Sync.tla"), guarded); + writeFileSync(join(dir, "SyncFixed.tla"), guarded.replace("MODULE Sync", "MODULE SyncFixed")); + const run = (...args) => node("statechart.mjs", ["from-spec", "Sync.tla", ...args], dir); + const broken = /^Error: Invariant NeverSyncedWhileOffline is violated/m; + + const named = run("--fixed", "SyncFixed.tla", "--out", "named"); + const same = run("--fixed", "fix/Sync.tla", "--out", "same"); + assert.equal(same.status, 1, same.stderr); + assert.equal(same.stderr, named.stderr.replaceAll("SyncFixed.tla", "fix/Sync.tla")); + assert.equal( + same.stdout, + named.stdout + .replace("TLC on SyncFixed.tla with", "TLC on fix/Sync.tla with") + .replace(" in named.\n", " in same.\n"), + ); + for (const file of ["chart.mmd", "bug-1.mmd", "stories.md"]) assert.equal( - r.stdout.split("\n").at(-2), - `Wrote ${[...written, "stories.md", "summary.json"].join(", ")} in Sync-statechart.`, + readFileSync(join(dir, "same", file), "utf8"), + readFileSync(join(dir, "named", file), "utf8"), ); - assert.deepEqual( - readdirSync(join(dir, "Sync-statechart")).sort(), - [...written, "stories.md", "summary.json", "tlc"].sort(), - ); - } + const tlcOut = (...parts) => readFileSync(join(dir, "same", "tlc", ...parts), "utf8"); + assert.match(tlcOut("Sync-NeverSyncedWhileOffline.out"), broken); + assert.doesNotMatch(tlcOut("fixed", "Sync-NeverSyncedWhileOffline.out"), broken); + + cpSync(join(dir, "Sync.cfg"), join(dir, "fix", "Sync.cfg")); + const agreed = run("--against", "fix/Sync.tla", "--out", "agreed"); + assert.equal(agreed.status, 1, agreed.stderr); + assert.match(agreed.stdout, /^TLC on fix\/Sync\.tla with fix\/Sync\.cfg:\n/m); + const agreedOut = (...parts) => readFileSync(join(dir, "agreed", "tlc", ...parts), "utf8"); + assert.match(agreedOut("Sync-NeverSyncedWhileOffline.out"), broken); + assert.doesNotMatch(agreedOut("against", "Sync-NeverSyncedWhileOffline.out"), broken); }, ); -const pay = (name, charge) => `---- MODULE ${name} ---- -EXTENDS Naturals -VARIABLES screen, charges -vars == <> -Init == screen = "Cart" /\\ charges = 0 -Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED charges -Charge == ${charge} -Refund == charges = 2 /\\ charges' = 1 /\\ screen' = "Refunded" -Next == Pay \\/ Charge \\/ Refund -Spec == Init /\\ [][Next]_vars -NoDoubleCharge == charges <= 1 +const lockSpec = `---- MODULE Lock ---- +VARIABLES a, b, locked +vars == <> +Init == a = "idle" /\\ b = "idle" /\\ locked = FALSE +CheckA == a = "idle" /\\ ~locked /\\ a' = "checked" /\\ UNCHANGED <> +TakeA == a = "checked" /\\ a' = "critical" /\\ locked' = TRUE /\\ UNCHANGED b +ReleaseA == a = "critical" /\\ a' = "idle" /\\ locked' = FALSE /\\ UNCHANGED b +CheckB == b = "idle" /\\ ~locked /\\ b' = "checked" /\\ UNCHANGED <> +TakeB == b = "checked" /\\ b' = "critical" /\\ locked' = TRUE /\\ UNCHANGED a +ReleaseB == b = "critical" /\\ b' = "idle" /\\ locked' = FALSE /\\ UNCHANGED a +Spec == Init /\\ [][CheckA \\/ TakeA \\/ ReleaseA \\/ CheckB \\/ TakeB \\/ ReleaseB]_vars +\\* Two processes are never in the critical section at once. +MutualExclusion == ~(a = "critical" /\\ b = "critical") ==== `; -const payMap = `export const readerActions = ["Pay"]; -export const state = (v) => v.screen + ["", ", charged", ", charged twice"][v.charges]; -export const step = (action) => ({ Pay: "you pay", Charge: "the bank charges you", Refund: "the bank refunds you" })[action] ?? ""; -export const cause = () => "each charge request charges again"; +const lockMap = `const S = { idle: "outside", checked: "saw the lock free", critical: "in the critical section" }; +export const state = (v) => \`A \${S[v.a]}, B \${S[v.b]}\`; +export const step = (a) => ({ + CheckA: "A checks the lock", TakeA: "A takes the lock", ReleaseA: "A releases the lock", + CheckB: "B checks the lock", TakeB: "B takes the lock", ReleaseB: "B releases the lock", +})[a]; `; test( - "from-spec checks each PROPERTY in its own TLC run, and names the counterexample of a broken one", + "from-spec on a lock protocol names every step that leads into the bug, not only the shortest trace", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); - writeFileSync( - join(dir, "Pay.tla"), - pay("Pay", `screen = "Paying" /\\ charges = 0 /\\ charges' = 1 /\\ UNCHANGED screen`).replace( - "====", - 'ChargedAtMostOnce == [](charges <= 1)\nEventuallyPaying == <>(screen = "Paying")\n====', - ), - ); - writeFileSync( - join(dir, "Pay.cfg"), - "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\nPROPERTIES ChargedAtMostOnce EventuallyPaying\n", + writeFileSync(join(dir, "Lock.tla"), lockSpec); + writeFileSync(join(dir, "Lock.cfg"), "SPECIFICATION Spec\nINVARIANT MutualExclusion\n"); + writeFileSync(join(dir, "Lock.map.mjs"), lockMap); + const r = node( + "statechart.mjs", + ["from-spec", "Lock.tla", "--expect-broken", "MutualExclusion"], + dir, ); - writeFileSync(join(dir, "Pay.map.mjs"), payMap); - const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); - assert.equal(r.status, 1, r.stderr); + assert.equal(r.status, 0, r.stdout + r.stderr); + const story = `## 1. A in the critical section, B in the critical section + +Rule broken: Two processes are never in the critical section at once. +In the model, 2 different steps lead into this bug: "A takes the lock" and "B takes the lock". The steps below are the shortest path that TLC found. + +0. Start: A outside, B outside +1. A checks the lock. Now: A saw the lock free, B outside +2. B checks the lock. Now: A saw the lock free, B saw the lock free +3. A takes the lock. Now: A in the critical section, B saw the lock free +4. B takes the lock. Now: A in the critical section, B in the critical section +`; assert.equal( - r.stdout.split("\n\n")[0], - `TLC on Pay.tla with Pay.cfg: - NoDoubleCharge holds. - ChargedAtMostOnce holds. - EventuallyPaying is broken. The chart draws only INVARIANT rules, so read TLC's counterexample in ${join("Pay-statechart", "tlc", "Pay-EventuallyPaying.out")}. - TLC ran 4 times in N s.`, + r.stdout, + `TLC on Lock.tla with Lock.cfg: + MutualExclusion is broken (expected). + TLC ran 3 times in N s. + +Chart: 9 states and 18 transitions from 13 model states. +8 work, 1 breaks a rule. + +${story} +Broken as expected: MutualExclusion. + +Wrote chart.mmd, bug-1.mmd, stories.md, summary.json in Lock-statechart. +`, ); - assert.match(r.stdout, /\nNo state breaks a rule\.\n/); - const tlcDir = join(dir, "Pay-statechart", "tlc"); - assert.match( - readFileSync(join(tlcDir, "Pay-EventuallyPaying.out"), "utf8"), - /^Error: Temporal properties were violated\./m, + const out = join(dir, "Lock-statechart"); + assert.equal(readFileSync(join(out, "stories.md"), "utf8"), story); + const [bug] = JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).classes; + assert.equal(bug.counterexamples, 1); + assert.deepEqual(bug.lastSteps, ["A takes the lock", "B takes the lock"]); + assert.equal(bug.startsHere, false); + }, +); + +const counterMap = (steps) => + `export const state = (v) => \`The counter shows \${v.x}\`;\nexport const step = (a) => (${JSON.stringify(steps)})[a] ?? "the counter moves";\n`; + +test( + "a story does not count the steps into a bug when TLC names them only after the whole next-state action", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Jump.tla"), + `---- MODULE Jump ---- +VARIABLE x +Init == x = 0 +Next == + \\/ x = 0 /\\ x' = 1 + \\/ x = 1 /\\ x' = 3 + \\/ x = 0 /\\ x' = 3 +Spec == Init /\\ [][Next]_x +\\* The counter never shows 3. +NotThree == x # 3 +==== +`, ); - assert.deepEqual( - readdirSync(tlcDir) - .filter((f) => f.endsWith(".cfg") && /PROPERT/.test(readFileSync(join(tlcDir, f), "utf8"))) - .sort(), - ["Pay-ChargedAtMostOnce.cfg", "Pay-EventuallyPaying.cfg"], + writeFileSync(join(dir, "Jump.cfg"), "SPECIFICATION Spec\nINVARIANT NotThree\n"); + writeFileSync(join(dir, "Jump.map.mjs"), counterMap({})); + const r = node("statechart.mjs", ["from-spec", "Jump.tla", "--expect-broken", "NotThree"], dir); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match( + r.stderr, + /^Jump\.tla: 3 steps have the name Next, the whole next-state action\./m, ); - const expected = node( - "statechart.mjs", - ["from-spec", "Pay.tla", "--expect-broken", "EventuallyPaying"], - dir, + assert.equal( + readFileSync(join(dir, "Jump-statechart", "stories.md"), "utf8"), + `## 1. The counter shows 3 + +Rule broken: The counter never shows 3. +In the model, steps labeled "the counter moves" lead into this bug. TLC names them Next, the whole next-state action, so the story cannot count them. The steps below are the shortest path that TLC found. + +0. Start: The counter shows 0 +1. The counter moves. Now: The counter shows 3 +`, ); - assert.equal(expected.status, 0, expected.stderr); - assert.match(expected.stdout, /\n {2}EventuallyPaying is broken \(expected\)\. /); }, ); test( - "from-spec checks a rule that the config lists as both PROPERTY and INVARIANT once, as the INVARIANT, and charts its bug", + "a story says its path is the shortest only when more than one step leads into the bug", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeFileSync( - join(dir, "Pay.tla"), - pay( - "Pay", - `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, - ), + join(dir, "Skip.tla"), + `---- MODULE Skip ---- +EXTENDS Naturals +VARIABLE x +Init == x = 0 +Up == x < 2 /\\ x' = x + 1 +Skip == x = 0 /\\ x' = 2 +Spec == Init /\\ [][Up \\/ Skip]_x +\\* The counter never shows 1. +NotOne == x # 1 +\\* The counter never shows 2. +NotTwo == x # 2 +==== +`, ); writeFileSync( - join(dir, "Pay.cfg"), - "SPECIFICATION Spec\nPROPERTY NoDoubleCharge\nINVARIANT NoDoubleCharge\n", + join(dir, "Skip.cfg"), + "SPECIFICATION Spec\nINVARIANT NotOne\nINVARIANT NotTwo\n", ); - writeFileSync(join(dir, "Pay.map.mjs"), payMap); - const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); - assert.equal(r.status, 1, r.stderr); - assert.equal( - r.stdout.split("\n\n")[0], - `TLC on Pay.tla with Pay.cfg: - NoDoubleCharge is broken. - TLC ran 3 times in N s.`, + writeFileSync(join(dir, "Skip.map.mjs"), counterMap({})); + const r = node( + "statechart.mjs", + ["from-spec", "Skip.tla", "--expect-broken", "NotOne,NotTwo"], + dir, ); - assert.match(r.stdout, /\nRule broken: `NoDoubleCharge`\n/); + assert.equal(r.status, 0, r.stdout + r.stderr); assert.equal( - r.stdout.split("\n").at(-2), + readFileSync(join(dir, "Skip-statechart", "stories.md"), "utf8"), + `## 1. The counter shows 1 + +Rule broken: The counter never shows 1. +In the model, one step leads into this bug: "the counter moves". + +0. Start: The counter shows 0 +1. The counter moves. Now: The counter shows 1 + +## 2. The counter shows 2 + +Rule broken: The counter never shows 2. +In the model, one step leads into this bug: "the counter moves". The steps below are the shortest path that TLC found. + +0. Start: The counter shows 0 +1. The counter moves. Now: The counter shows 2 +`, + ); + }, +); + +test( + "a story says which step into the bug keeps the state, so that no chart draws it", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Panel.tla"), + `---- MODULE Panel ---- +VARIABLES panel, saved +vars == <> +Init == panel = "closed" /\\ saved = TRUE +Open == panel = "closed" /\\ panel' = "open" /\\ UNCHANGED saved +Close == panel = "open" /\\ panel' = "closed" /\\ UNCHANGED saved +Lose == panel = "open" /\\ saved /\\ saved' = FALSE /\\ UNCHANGED panel +Pop == panel = "closed" /\\ saved /\\ panel' = "open" /\\ saved' = FALSE +Spec == Init /\\ [][Open \\/ Close \\/ Lose \\/ Pop]_vars +\\* The note is never lost. +NeverLost == saved +==== +`, + ); + writeFileSync(join(dir, "Panel.cfg"), "SPECIFICATION Spec\nINVARIANT NeverLost\n"); + writeFileSync( + join(dir, "Panel.map.mjs"), + 'export const state = (v) => `The panel is ${v.panel}`;\nexport const step = (a) => ({ Open: "you open the panel", Close: "you close the panel", Lose: "the note is lost", Pop: "the panel pops open" })[a];\n', + ); + const r = node( + "statechart.mjs", + ["from-spec", "Panel.tla", "--expect-broken", "NeverLost"], + dir, + ); + assert.equal(r.status, 0, r.stdout + r.stderr); + const out = join(dir, "Panel-statechart"); + assert.match( + readFileSync(join(out, "stories.md"), "utf8"), + /^In the model, 2 different steps lead into this bug: "the note is lost" \(the state stays the same\) and "the panel pops open"\. The steps below are the shortest path that TLC found\.\n\n0\. Start: The panel is closed\n1\. The panel pops open\. Now: The panel is open\n$/m, + ); + for (const chart of readdirSync(out).filter((f) => f.endsWith(".mmd"))) + assert.doesNotMatch(readFileSync(join(out, chart), "utf8"), /the note is lost/, chart); + }, +); + +test( + "--png exits 2 unless every chart renders, even with --expect-broken, and keeps the files it wrote", + { skip: needsTlc.skip || needsPng.skip }, + () => { + const args = [ + "from-chart", + "sync.mmd", + "--expect-broken", + "NeverSyncedWhileOffline", + "--title", + "Sync", + "--png", + ]; + const titles = { chart: "Sync", "bug-1": "Sync, bug 1" }; + const xdg = fakeRenderer(); + for (const [unrendered, status, written] of [ + [["chart", "bug-1"], 2, ["chart.mmd", "bug-1.mmd"]], + [["bug-1"], 2, ["chart.mmd", "bug-1.mmd", "chart.png"]], + [[], 0, ["chart.mmd", "bug-1.mmd", "chart.png", "bug-1.png"]], + ]) { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); + const rendered = join(dir, "rendered"); + const r = node("statechart.mjs", args, dir, { + XDG_CACHE_HOME: xdg, + TLA2TOOLS: tools.jar, + JAVA: tools.java, + RENDERED: rendered, + UNRENDERED: unrendered.map((c) => titles[c]).join("|"), + }); + assert.equal(r.status, status, `${unrendered}: ${r.stdout}${r.stderr}`); + assert.equal( + r.stderr, + unrendered + .map((c) => `could not render Sync-statechart/${c}.mmd: Pele cannot lay out this chart\n`) + .join(""), + ); + assert.deepEqual( + readFileSync(rendered, "utf8").split("\n").filter(Boolean), + ["Sync", "Sync, bug 1"], + "tries every chart", + ); + assert.match(r.stdout, /\nBroken as expected: NeverSyncedWhileOffline\.\n/); + assert.equal( + r.stdout.split("\n").at(-2), + `Wrote ${[...written, "stories.md", "summary.json"].join(", ")} in Sync-statechart.`, + ); + const out = join(dir, "Sync-statechart"); + assert.deepEqual( + readdirSync(out).sort(), + [...written, "stories.md", "summary.json", "tlc"].sort(), + ); + for (const png of written.filter((f) => f.endsWith(".png"))) + assert.ok(decodePng(readFileSync(join(out, png))).width > 0, png); + } + }, +); + +const SOURCE = fixture("regions.mmd"); +const SOURCE_LINES = [...SOURCE.matchAll(/state "([^"]+)" as|--> \w+ : (.+)$/gm)] + .flatMap((m) => (m[1] ?? m[2]).split("
")) + .sort(); +const drawnLines = (svg) => + labeledBoxes(svg) + .flatMap((b) => b.lines) + .sort(); + +function rectOf(svg, id) { + const [ox, oy] = // + .exec(svg) + .slice(1) + .map(Number); + const [tx, ty, x, y, w, h] = new RegExp( + `data-id="${id}"(?: transform="translate\\(([\\d.-]+),([\\d.-]+)\\)")?> +(v ?? 0)); + const [left, top] = [ox + tx + x, oy + ty + y]; + return { left, top, right: left + w, bottom: top + h }; +} + +test( + "render writes a PNG at twice the chart's size, next to the chart or at --out", + needsPng, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "regions.mmd"), SOURCE); + writeFileSync(join(dir, "regions.txt"), SOURCE); + for (const [args, file, theme] of [ + [["regions.mmd"], "regions.png", "light"], + [["regions.txt"], "regions.txt.png", "light"], + ...Object.keys(THEMES).map((t) => [ + ["regions.mmd", "--theme", t, "--out", join("new", t, "chart.png")], + join("new", t, "chart.png"), + t, + ]), + ]) { + const r = node("statechart.mjs", ["render", ...args], dir); + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, ""); + assert.equal(r.stdout, `Wrote ${file}.\n`); + const chart = renderChart(lib, SOURCE, { theme: THEMES[theme] }); + const png = decodePng(readFileSync(join(dir, file))); + assert.deepEqual([png.width, png.height], [2 * chart.width, 2 * chart.height], file); + assert.equal(png.pixel(0, 0), THEMES[theme].background, file); + } + }, +); + +test( + "a chart too big to draw at twice its size draws at about 100 million pixels", + needsPng, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const steps = Array.from({ length: 60 }, (_, i) => + [0, 1, 2].map((k) => ` s${i} --> s${(i * 7 + k * 13 + 1) % 60} : e${i}x${k}`), + ).flat(); + const src = `stateDiagram-v2\n [*] --> s0\n${steps.join("\n")}\n`; + writeFileSync(join(dir, "c.mmd"), src); + const chart = renderChart(lib, src); + assert.ok(4 * chart.width * chart.height > 1.2e8, `${chart.width}x${chart.height}`); + const r = node("statechart.mjs", ["render", "c.mmd"], dir); + assert.equal(r.status, 0, r.stderr); + const png = readFileSync(join(dir, "c.png")); + const pixels = png.readUInt32BE(16) * png.readUInt32BE(20); + assert.ok(pixels > 0.99e8 && pixels < 1.001e8, `${pixels} pixels`); + }, +); + +test( + "a PNG fills a state with its theme's group color, and a state that breaks a rule with the theme's wrong fill and border", + needsPng, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "regions.mmd"), SOURCE); + const chartFill = /classDef group0 fill:(#\w{6})/.exec(SOURCE)[1]; + for (const name of [undefined, ...Object.keys(THEMES)]) { + const theme = THEMES[name ?? "light"]; + const r = node( + "statechart.mjs", + ["render", "regions.mmd", ...(name ? ["--theme", name] : [])], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const png = decodePng(readFileSync(join(dir, "regions.png"))); + const { svg } = renderChart(lib, SOURCE, { theme }); + const inside = (box) => png.pixel(2 * (box.left + 8), box.top + box.bottom); + const borderWidth = (box) => { + const [x, y] = [2 * box.left, box.top + box.bottom]; + const ink = png.pixel(x, y); + let n = 0; + for (let i = x - 6; i <= x + 6; i++) if (png.pixel(i, y) === ink) n++; + return n; + }; + const wrong = rectOf(svg, "r1s3"); + const group = rectOf(svg, "r1s2"); + assert.equal(inside(wrong), theme.wrong.fill, `${name} wrong fill`); + assert.equal( + png.pixel(2 * wrong.left, wrong.top + wrong.bottom), + theme.wrong.border, + `${name} wrong border`, + ); + assert.ok( + borderWidth(wrong) >= 4 && borderWidth(group) <= 3, + `${name} wrong border is ${borderWidth(wrong)}px and a group border ${borderWidth(group)}px at twice the chart's size`, + ); + assert.equal(inside(group), theme.groups?.[0] ?? chartFill, `${name} group fill`); + } + }, +); + +test( + "a PNG draws a state that breaks a rule in some cases, one reached only after a bug, and one that breaks a rule only with the fix, in its theme's colors", + needsPng, + () => { + const source = [ + "stateDiagram-v2", + ...STATUS_CLASSES.map((c) => ` ${c}`), + ...["Mixed", "After", "Worse"].map((b) => ` ${groupClassDef("group0", "#d6eaf8", b)}`), + ' state "Works" as w', + ' state "Wrong in some cases" as m', + ' state "Reached after a bug" as a', + ' state "Wrong only with the fix" as o', + ' state "Wrong in some cases, in a group" as gm', + ' state "Reached after a bug, in a group" as ga', + ' state "Wrong only with the fix, in a group" as go', + " [*] --> w", + " w --> m : one", + " m --> a : two", + " a --> o : five", + " w --> gm : three", + " gm --> ga : four", + " ga --> go : six", + " class w works", + " class m mixed", + " class a after", + " class o worse", + " class gm group0Mixed", + " class ga group0After", + " class go group0Worse", + "", + ].join("\n"); + for (const [name, theme] of Object.entries(THEMES)) { + const { svg } = renderChart(lib, source, { theme }); + for (const [id, status] of [ + ["m", "mixed"], + ["a", "after"], + ["o", "worse"], + ["gm", "mixed"], + ["ga", "after"], + ["go", "worse"], + ]) { + const [box, label] = new RegExp( + `data-id="${id}"[^>]*>]*style="([^"]*)"/>]*style="([^"]*)"`, + ) + .exec(svg) + .slice(1) + .map((s) => + Object.fromEntries( + s + .split(";") + .filter(Boolean) + .map((d) => d.split(":")), + ), + ); + assert.deepEqual( + [box.fill, box.stroke, label.fill], + [theme[status]?.fill, theme[status]?.border, theme[status]?.text], + `${name} ${id}`, + ); + } + } + }, +); + +test("a PNG draws no edge over a group's title band", needsPng, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const source = fixture("band.mmd"); + writeFileSync(join(dir, "band.mmd"), source); + for (const [name, theme] of Object.entries(THEMES)) { + const r = node("statechart.mjs", ["render", "band.mmd", "--theme", name], dir); + assert.equal(r.status, 0, r.stderr); + const png = decodePng(readFileSync(join(dir, "band.png"))); + const { svg } = renderChart(lib, source, { theme }); + const group = rectOf(svg, "G"); + const [rectY, dividerY] = + /data-id="G">]*\/> fcMatch(f)?.name === f, +); +const type1 = ["Nimbus Sans Narrow", "Nimbus Sans", "URW Gothic"].find((f) => + /\.(pfb|pfa|t1)$/.test(fcMatch(f)?.file ?? ""), +); +const fcFile = (pattern) => + spawnSync("fc-list", [pattern, "file"], { encoding: "utf8" }).stdout?.match(/^(.+): $/m)?.[1]; +const type1Styles = [ + "Nimbus Sans:style=Regular:fontformat=CFF", + "Nimbus Sans:style=Bold:fontformat=Type 1", + "Nimbus Sans:style=Italic:fontformat=Type 1", +].map(fcFile); +test( + "each label fits its box in the font it is drawn in, and a wider font gets a wider wrap, so lines break where the chart breaks them", + { + skip: + needsPng.skip || + (fonts.length < 2 && "needs two of DejaVu Sans, Liberation Sans, Noto Sans, and Inter"), + }, + () => { + const runs = fonts.map((font) => { + const measure = fontMeasurer(lib.Resvg, font); + const chart = renderChart(lib, SOURCE, { systemFont: font }); + for (const { kind, id, box, size, lines } of labeledBoxes(chart.svg)) + for (const line of lines) + assert.ok( + measure.width(line, size) <= box, + `${font}: ${kind} ${id} "${line}" fits ${box}`, + ); + assert.deepEqual( + drawnLines(chart.svg), + SOURCE_LINES, + `${font} breaks lines as the chart does`, + ); + return { + font, + wrap: chart.wrap, + ink: SOURCE_LINES.reduce((s, l) => s + measure.width(l, 16), 0), + }; + }); + runs.sort((a, b) => a.ink - b.ink); + const [narrow, wide] = [runs[0], runs.at(-1)]; + assert.ok( + wide.wrap > narrow.wrap, + `${wide.font} wraps at ${wide.wrap}, ${narrow.font} at ${narrow.wrap}`, + ); + }, +); + +test( + "a font that resvg cannot read keeps Pele's widths, so the chart keeps every label", + { skip: needsPng.skip || (!type1 && "cannot find a Type 1 font") }, + () => { + assert.deepEqual(drawnLines(renderChart(lib, SOURCE, { systemFont: type1 }).svg), SOURCE_LINES); + }, +); + +test( + "a bold or italic face that resvg cannot read is measured in its regular face, so the chart keeps every label", + { + skip: + needsPng.skip || + (!type1Styles.every(Boolean) && + "needs Nimbus Sans with an OpenType regular face and Type 1 bold and italic faces"), + }, + () => { + const dir = mkdtempSync(join(tmpdir(), "fontconfig-")); + for (const file of type1Styles) symlinkSync(file, join(dir, basename(file))); + writeFileSync( + join(dir, "fonts.conf"), + `${dir}${dir}`, + ); + const styled = SOURCE.replace(/(--> \w+ : )(.+)$/m, "$1$2").replace( + /(--> \w+ : )([^<].*)$/m, + "$1$2", + ); + const run = spawnSync( + process.execPath, + [ + "--input-type=module", + "-e", + `import { loadRenderer, renderChart } from ${JSON.stringify(pathToFileURL(join(scripts, "render.mjs")).href)}; +process.stdout.write(renderChart(await loadRenderer(), ${JSON.stringify(styled)}, { systemFont: "Nimbus Sans" }).svg);`, + ], + { encoding: "utf8", env: { ...process.env, FONTCONFIG_FILE: join(dir, "fonts.conf") } }, + ); + assert.equal(run.status, 0, run.stderr); + assert.deepEqual(drawnLines(run.stdout), SOURCE_LINES); + }, +); + +test("a PNG puts parallel regions side by side and the legend above them", needsPng, () => { + const { svg } = renderChart(lib, SOURCE); + const [parcel, payment, legend, regions] = ["region0", "region1", "legend", "regions"].map((id) => + rectOf(svg, id), + ); + assert.ok(parcel.top < payment.bottom && payment.top < parcel.bottom, "the regions share rows"); + assert.ok(parcel.right <= payment.left, "the regions do not share columns"); + assert.ok(legend.bottom <= regions.top, "the legend is above the regions"); +}); + +test("a renderer that npm cannot install exits 2 and writes nothing", () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "c.mmd"), SOURCE); + writeFileSync(join(dir, "sync.mmd"), fixture("sync.mmd")); + const xdg = mkdtempSync(join(tmpdir(), "xdg-")); + const env = { + XDG_CACHE_HOME: xdg, + npm_config_cache: mkdtempSync(join(tmpdir(), "npm-")), + npm_config_offline: "true", + }; + for (const args of [ + ["render", "c.mmd"], + ["from-chart", "sync.mmd", "--png"], + ]) { + const r = node("statechart.mjs", args, dir, env); + assert.equal(r.status, 2, `${args}: ${r.stdout}${r.stderr}`); + assert.equal(r.stdout, ""); + assert.ok( + r.stderr.startsWith( + `Installing pele@1.8.0 and @resvg/resvg-js@2.6.2 with npm in ${join(xdg, "statechart", CACHE_NAME)}. This happens once.\n`, + ), + r.stderr, + ); + assert.match( + r.stderr, + /\ncannot install pele@1\.8\.0 and @resvg\/resvg-js@2\.6\.2 with npm:\n/, + ); + assert.deepEqual(readdirSync(dir).sort(), ["c.mmd", "sync.mmd"]); + assert.deepEqual(readdirSync(join(xdg, "statechart")), []); + } +}); + +test("a renderer that cannot load exits 2, names its folder, and says to delete it", () => { + const xdg = mkdtempSync(join(tmpdir(), "xdg-")); + const cache = join(xdg, "statechart", CACHE_NAME); + mkdirSync(cache, { recursive: true }); + writeFileSync(join(cache, "entry.mjs"), 'export { render } from "no-such-package";\n'); + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "c.mmd"), SOURCE); + const r = node("statechart.mjs", ["render", "c.mmd"], dir, { XDG_CACHE_HOME: xdg }); + assert.equal(r.status, 2); + assert.equal(r.stdout, ""); + assert.ok(r.stderr.startsWith(`cannot load Pele and resvg from ${cache}: `), r.stderr); + assert.ok(r.stderr.endsWith("\nDelete that folder to install them again.\n"), r.stderr); + assert.deepEqual(readdirSync(dir), ["c.mmd"]); +}); + +test("a chart too long for Pele exits 2, writes no PNG, and says how to get one", needsPng, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const lines = Array.from({ length: 3000 }, (_, i) => ` s${i} --> s${i + 1} : step ${i}`); + writeFileSync(join(dir, "c.mmd"), `stateDiagram-v2\n${lines.join("\n")}\n`); + const r = node("statechart.mjs", ["render", "c.mmd"], dir); + assert.equal(r.status, 2); + assert.equal(r.stdout, ""); + assert.ok(r.stderr.startsWith("could not render c.mmd: "), r.stderr); + assert.ok( + r.stderr.endsWith( + " The chart is too big for one PNG, and the .mmd file still has all of it. A chart with fewer states fits. Run node statechart.mjs from-spec SPEC.tla --cover --png to draw a smaller overview of the spec.\n", + ), + r.stderr, + ); + assert.deepEqual(readdirSync(dir), ["c.mmd"]); +}); + +const pay = (name, charge) => `---- MODULE ${name} ---- +EXTENDS Naturals +VARIABLES screen, charges +vars == <> +Init == screen = "Cart" /\\ charges = 0 +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED charges +Charge == ${charge} +Refund == charges = 2 /\\ charges' = 1 /\\ screen' = "Refunded" +Next == Pay \\/ Charge \\/ Refund +Spec == Init /\\ [][Next]_vars +NoDoubleCharge == charges <= 1 +==== +`; +const payMap = `export const readerActions = ["Pay"]; +export const state = (v) => v.screen + ["", ", charged", ", charged twice"][v.charges]; +export const step = (action) => ({ Pay: "you pay", Charge: "the bank charges you", Refund: "the bank refunds you" })[action] ?? ""; +export const cause = () => "each charge request charges again"; +`; + +test( + "from-spec checks each PROPERTY in its own TLC run, and names the counterexample of a broken one", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay("Pay", `screen = "Paying" /\\ charges = 0 /\\ charges' = 1 /\\ UNCHANGED screen`).replace( + "====", + 'ChargedAtMostOnce == [](charges <= 1)\nEventuallyPaying == <>(screen = "Paying")\n====', + ), + ); + writeFileSync( + join(dir, "Pay.cfg"), + "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\nPROPERTIES ChargedAtMostOnce EventuallyPaying\n", + ); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); + assert.equal(r.status, 1, r.stderr); + assert.equal( + r.stdout.split("\n\n")[0], + `TLC on Pay.tla with Pay.cfg: + NoDoubleCharge holds. + ChargedAtMostOnce holds. + EventuallyPaying is broken. The chart draws only INVARIANT rules, so read TLC's counterexample in ${join("Pay-statechart", "tlc", "Pay-EventuallyPaying.out")}. + TLC ran 4 times in N s.`, + ); + const stories = `## Broken rules that the chart does not draw + +Rule broken: \`EventuallyPaying\`. The run stops in "Cart" forever. TLC's counterexample is in ${join("tlc", "Pay-EventuallyPaying.out")}. +`; + assert.ok(r.stdout.includes(`\nNo state breaks an INVARIANT rule.\n\n${stories}`), r.stdout); + const tlcDir = join(dir, "Pay-statechart", "tlc"); + assert.match( + readFileSync(join(tlcDir, "Pay-EventuallyPaying.out"), "utf8"), + /^Error: Temporal properties were violated\./m, + ); + assert.deepEqual( + readdirSync(tlcDir) + .filter((f) => f.endsWith(".cfg") && /PROPERT/.test(readFileSync(join(tlcDir, f), "utf8"))) + .sort(), + ["Pay-ChargedAtMostOnce.cfg", "Pay-EventuallyPaying.cfg"], + ); + const expected = node( + "statechart.mjs", + ["from-spec", "Pay.tla", "--expect-broken", "EventuallyPaying"], + dir, + ); + assert.equal(expected.status, 0, expected.stderr); + assert.match(expected.stdout, /\n {2}EventuallyPaying is broken \(expected\)\. /); + }, +); + +test( + "from-spec names the reader states that a broken PROPERTY's counterexample loops through, without a state twice in a row and without the states before the loop", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Bell.tla"), + `---- MODULE Bell ---- +VARIABLES door, bell, knocked +vars == <> +Init == door = "closed" /\\ bell = "off" /\\ knocked = FALSE +SwitchOn == bell = "off" /\\ bell' = "quiet" /\\ UNCHANGED <> +Knock == bell = "quiet" /\\ ~knocked /\\ knocked' = TRUE /\\ UNCHANGED <> +Ring == knocked /\\ bell' = "ringing" /\\ knocked' = FALSE /\\ UNCHANGED door +Hush == bell = "ringing" /\\ bell' = "quiet" /\\ UNCHANGED <> +Open == bell = "ringing" /\\ door' = "open" /\\ UNCHANGED <> +Spec == Init /\\ [][SwitchOn \\/ Knock \\/ Ring \\/ Hush \\/ Open]_vars /\\ WF_vars(SwitchOn \\/ Knock \\/ Ring \\/ Hush) +EventuallyOpen == <>(door = "open") +==== +`, + ); + writeFileSync(join(dir, "Bell.cfg"), "SPECIFICATION Spec\nPROPERTY EventuallyOpen\n"); + writeFileSync( + join(dir, "Bell.map.mjs"), + `export const readerActions = ["Open"]; +export const state = (v, facts) => v.door === "open" ? "Door open" : v.bell === "off" ? "Bell off" : facts.enabled.has("Hush") ? "Bell ringing" : "Waiting"; +export const step = (action) => action; +`, + ); + const r = node( + "statechart.mjs", + ["from-spec", "Bell.tla", "--expect-broken", "EventuallyOpen"], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const out = join(dir, "Bell-statechart"); + assert.match( + readFileSync(join(out, "tlc", "Bell-EventuallyOpen.out"), "utf8"), + /^Back to state 2: /m, + ); + const stories = `## Broken rules that the chart does not draw + +Rule broken: \`EventuallyOpen\`. The run loops through "Waiting" and "Bell ringing" forever. TLC's counterexample is in ${join("tlc", "Bell-EventuallyOpen.out")}. +`; + assert.ok(r.stdout.includes(`\nNo state breaks an INVARIANT rule.\n\n${stories}`), r.stdout); + assert.deepEqual( + JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).brokenProperties[0].repeats, + { states: ["Waiting", "Bell ringing"], stutters: false }, + ); + }, +); + +test( + "from-spec checks a rule that the config lists as both PROPERTY and INVARIANT once, as the INVARIANT, and charts its bug", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync( + join(dir, "Pay.cfg"), + "SPECIFICATION Spec\nPROPERTY NoDoubleCharge\nINVARIANT NoDoubleCharge\n", + ); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); + assert.equal(r.status, 1, r.stderr); + assert.equal( + r.stdout.split("\n\n")[0], + `TLC on Pay.tla with Pay.cfg: + NoDoubleCharge is broken. + TLC ran 3 times in N s.`, + ); + assert.match(r.stdout, /\nRule broken: `NoDoubleCharge`\n/); + assert.equal( + r.stdout.split("\n").at(-2), "Wrote chart.mmd, bug-1.mmd, stories.md, summary.json in Pay-statechart.", ); const unknown = node( @@ -1406,31 +3137,165 @@ test("from-spec reports a broken PROPERTY in each form that TLC reports it in", `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, ).replace( "====", - `ChargedAtMostOnce == [](charges <= 1)\nNeverChargedAgain == [][charges' <= 1]_charges\nStartsPaying == screen = "Paying"\n====`, + `ChargedAtMostOnce == [](charges <= 1)\nNeverChargedAgain == [][charges' <= 1]_charges\nStartsPaying == screen = "Paying"\nNeverTwiceNorRefunded == [](charges <= 1) /\\ [](screen # "Refunded")\nStartsPayingUncharged == screen = "Paying" /\\ charges = 0\n====`, ), ); writeFileSync( join(dir, "Pay.cfg"), - "SPECIFICATION Spec\nPROPERTIES ChargedAtMostOnce NeverChargedAgain StartsPaying\n", + "SPECIFICATION Spec\nPROPERTIES ChargedAtMostOnce NeverChargedAgain StartsPaying NeverTwiceNorRefunded StartsPayingUncharged\n", ); writeFileSync(join(dir, "Pay.map.mjs"), payMap); const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir); assert.equal(r.status, 1, r.stderr); const broken = (p) => ` ${p} is broken. The chart draws only INVARIANT rules, so read TLC's counterexample in ${join("Pay-statechart", "tlc", `Pay-${p}.out`)}.`; + const initialOnly = (p) => + ` TLC checked ${p} only in the initial states, because it has no [], <>, or ~>. List it as an INVARIANT to check every state.`; assert.equal( r.stdout.split("\n\n")[0], `TLC on Pay.tla with Pay.cfg: Pay.cfg has no INVARIANT, so the chart marks no state broken. ${broken("ChargedAtMostOnce")} ${broken("NeverChargedAgain")} -${broken("StartsPaying")} - TLC ran 4 times in N s.`, +${broken("StartsPaying")}${initialOnly("StartsPaying")} +${broken("NeverTwiceNorRefunded")} +${broken("StartsPayingUncharged")}${initialOnly("StartsPayingUncharged")} + TLC ran 6 times in N s.`, + ); + const tlcDir = join(dir, "Pay-statechart", "tlc"); + assert.match( + readFileSync(join(tlcDir, "Pay-NeverTwiceNorRefunded.out"), "utf8"), + /^Error: Invariant line \d+, col \d+ to line \d+, col \d+ of module Pay is violated\.$/m, + ); + assert.match( + readFileSync(join(tlcDir, "Pay-StartsPayingUncharged.out"), "utf8"), + /^Error: Property line \d+, col \d+ to line \d+, col \d+ of module Pay is violated by the initial state:$/m, ); }); test( - "a state reached only after a bug takes the bug's color, and the fix edge leaves the state before the changed step", + "from-spec reports a broken refinement of an instanced module in its output, the stories, and the summary", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "TicketApi.tla"), + `---- MODULE TicketApi ---- +VARIABLE status +Init == status = "open" +Take == status = "open" /\\ status' = "taken" +Close == status = "taken" /\\ status' = "closed" +Spec == Init /\\ [][Take \\/ Close]_status +==== +`, + ); + writeFileSync( + join(dir, "Desk.tla"), + `---- MODULE Desk ---- +VARIABLES phase, agent +vars == <> +Init == phase = "open" /\\ agent = "none" +Assign == phase = "open" /\\ phase' = "taken" /\\ agent' = "ana" +Finish == phase = "taken" /\\ phase' = "closed" /\\ UNCHANGED agent +Drop == phase = "open" /\\ phase' = "closed" /\\ UNCHANGED agent +Spec == Init /\\ [][Assign \\/ Finish \\/ Drop]_vars +Known == phase \\in {"open", "taken", "closed"} +Api == INSTANCE TicketApi WITH status <- phase +\\* A ticket closes only after someone takes it. +ApiRefines == Api!Spec +StartsOpen == phase = "open" +==== +`, + ); + writeFileSync( + join(dir, "Desk.cfg"), + "SPECIFICATION Spec\nINVARIANT Known\nPROPERTIES ApiRefines StartsOpen\n", + ); + writeFileSync( + join(dir, "Desk.map.mjs"), + `export const readerActions = []; +export const state = (v) => "Ticket " + v.phase; +export const step = (action) => ({ Assign: "an agent takes it", Finish: "the agent closes it", Drop: "it closes unassigned" })[action]; +`, + ); + const r = node( + "statechart.mjs", + ["from-spec", "Desk.tla", "--expect-broken", "ApiRefines"], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const out = join(dir, "Desk-statechart"); + assert.equal( + r.stdout.split("\n\n")[0], + `TLC on Desk.tla with Desk.cfg: + Known holds. + ApiRefines is broken (expected). The chart draws only INVARIANT rules, so read TLC's counterexample in ${join("Desk-statechart", "tlc", "Desk-ApiRefines.out")}. + StartsOpen holds. TLC checked StartsOpen only in the initial states, because it has no [], <>, or ~>. List it as an INVARIANT to check every state. + TLC ran 4 times in N s.`, + ); + const stories = `## Broken rules that the chart does not draw + +Rule broken: A ticket closes only after someone takes it. TLC's counterexample is in ${join("tlc", "Desk-ApiRefines.out")}. +`; + assert.ok(r.stdout.includes(`\nNo state breaks an INVARIANT rule.\n\n${stories}`), r.stdout); + assert.equal(readFileSync(join(out, "stories.md"), "utf8"), stories); + assert.deepEqual(JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).brokenProperties, [ + { + spec: "Desk.tla", + property: "ApiRefines", + rule: "A ticket closes only after someone takes it.", + repeats: null, + counterexample: join("tlc", "Desk-ApiRefines.out"), + }, + ]); + }, +); + +test( + "from-spec explains an invariant and a property that name rules of a named INSTANCE with substitutions in the instanced module's words", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "LampRules.tla"), lampRules); + writeFileSync( + join(dir, "Lamp.tla"), + `---- MODULE Lamp ---- +VARIABLES lamp, cable +Rules == INSTANCE LampRules WITH light <- lamp, wire <- cable +Safe == Rules!SafeLight +Sound == + Rules!NeverShorts +${lampSteps}`, + ); + writeFileSync(join(dir, "Lamp.cfg"), "SPECIFICATION Spec\nINVARIANT Safe\nPROPERTIES Sound\n"); + writeFileSync(join(dir, "Lamp.map.mjs"), lampMap); + const r = node( + "statechart.mjs", + ["from-spec", "Lamp.tla", "--expect-broken", "Safe", "--expect-broken", "Sound"], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const out = join(dir, "Lamp-statechart"); + const stories = readFileSync(join(out, "stories.md"), "utf8"); + assert.match(stories, /^Rule broken: The light never comes on over a shorted wire\.$/m); + assert.match(stories, /^1\. The wire shorts\. Now: Off\n2\. You switch it on\. Now: On$/m); + assert.ok( + stories.includes( + `\nRule broken: The wire never shorts. TLC's counterexample is in ${join("tlc", "Lamp-Sound.out")}.\n`, + ), + stories, + ); + assert.deepEqual( + JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).brokenProperties.map( + (b) => b.rule, + ), + ["The wire never shorts."], + ); + }, +); + +test( + "a state reached only after a bug gets its own color, and the fix edge leaves the state before the changed step", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); @@ -1455,17 +3320,6 @@ test( assert.match(tlc(dir, "Pay").stdout, /Invariant NoDoubleCharge is violated/); assert.doesNotMatch(tlc(dir, "PayFixed").stdout, /is violated/); - const statusOf = (chart) => { - const names = new Map( - [...chart.matchAll(/^ {2}state "([^"]+)" as (s\d+)$/gm)].map((m) => [ - m[2], - m[1].replace(//g, " "), - ]), - ); - return Object.fromEntries( - [...chart.matchAll(/^ {2}class (s\d+) (\w+)$/gm)].map((m) => [names.get(m[1]), m[2]]), - ); - }; const args = [ "--dot", "Pay.dot", @@ -1484,7 +3338,7 @@ test( Paying: "works", "Paying, charged": "works", "Paying, charged twice": "wrong", - "Refunded, charged": "wrong", + "Refunded, charged": "after", }); const withFix = [ @@ -1550,28 +3404,12 @@ test( }; assert.match( story(), - /^In the spec you see the same screens and steps, but the spec does not check this rule\.$/m, + /^In the spec you see the same states and steps, but the spec does not check this rule\.$/m, ); assert.match( story("--against-traces", "Pay.out", "--against-rules", "NoDoubleCharge"), - /^In the spec you see the same screens and steps, so the spec does not change this path\.$/m, + /^In the spec you see the same states and steps, so the spec does not change this path\.$/m, ); - const alone = node( - "tlc-to-chart.mjs", - [ - "--dot", - "Pay.dot", - "--map", - "pay.map.mjs", - "--against-dot", - "Pay.dot", - "--against-traces", - "Pay.out", - ], - dir, - ); - assert.equal(alone.status, 2); - assert.match(alone.stderr, /^--against-traces and --against-rules go together\./); }, ); @@ -1621,13 +3459,13 @@ test( TLC ran 3 times in N s. Chart: 5 states and 4 transitions from 5 model states. -3 work, 2 break a rule. +3 work, 1 breaks a rule, 1 reached only after a bug. ## 1. Paying, charged twice Rule broken: \`NoDoubleCharge\` Why: each charge request charges again. -This is the only way to get here in the model. +In the model, one step leads into this bug: "the bank charges you". 0. Start: Cart 1. You pay. Now: Paying @@ -1652,7 +3490,7 @@ Wrote chart.mmd, bug-1.mmd, bug-1-developer.mmd, bug-1-layered.mmd, stories.md, ]); assert.equal( readFileSync(join(dir, "Pay-statechart", "tlc", "Pay.cfg"), "utf8"), - "SPECIFICATION Spec\n", + "SPECIFICATION Spec\nINVARIANT StatechartRules\n", ); assert.equal( readFileSync(join(dir, "Pay-statechart", "tlc", "Pay-NoDoubleCharge.cfg"), "utf8"), @@ -1701,9 +3539,205 @@ NeverSyncedWhileOffline == ~(conn = "Offline" /\\ badge = "Synced") summary.classes.map((c) => [c.end, c.readerSteps]), [["Offline | Synced", 0]], ); - assert.deepEqual(summary.chartStatus, { works: 2, wrong: 1, fixed: 0 }); + assert.deepEqual(summary.chartStatus, { works: 2, wrong: 1, fixed: 0, mixed: 0, after: 0 }); }); +test( + "a screen that a clean path reaches is marked wrong in some cases when a state under it breaks a rule after a bug", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Fork.tla"), + `---- MODULE Fork ---- +VARIABLE screen +Init == screen = "start" +Slip == screen = "start" /\\ screen' = "bad" +Go == screen = "start" /\\ screen' = "good" +Fall == screen = "bad" /\\ screen' = "worse" +Spec == Init /\\ [][Slip \\/ Go \\/ Fall]_screen +NeverBroken == screen \\notin {"bad", "worse"} +==== +`, + ); + writeFileSync(join(dir, "Fork.cfg"), "SPECIFICATION Spec\nINVARIANT NeverBroken\n"); + writeFileSync( + join(dir, "Fork.map.mjs"), + 'export const state = (v) => ({ start: "Home", bad: "Error", good: "Done", worse: "Done" })[v.screen];\nexport const step = (a) => ({ Slip: "you slip", Go: "you go", Fall: "you fall" })[a];\n', + ); + const r = node("statechart.mjs", ["from-spec", "Fork.tla"], dir); + assert.equal(r.status, 1, r.stderr); + const out = join(dir, "Fork-statechart"); + assert.deepEqual(statusOf(readFileSync(join(out, "chart.mmd"), "utf8")), { + Done: "mixed", + Error: "wrong", + Home: "works", + }); + const summary = JSON.parse(readFileSync(join(out, "summary.json"), "utf8")); + assert.deepEqual(summary.chartStatus, { works: 1, wrong: 1, fixed: 0, mixed: 1, after: 0 }); + assert.deepEqual(summary.mixedStatus, ["Done"]); + assert.match( + r.stdout, + /\n1 work, 1 breaks a rule, 1 breaks a rule in some cases\.\n\nThis state breaks a rule in some cases and not in others:\n {2}Done\n/, + ); + }, +); + +test( + "a bug that ends on the start screen's label marks that screen wrong in some cases, and a screen reached only after the bug gets its own color", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Echo.tla"), + `---- MODULE Echo ---- +VARIABLE screen +Init == screen = "idle" +Lose == screen = "idle" /\\ screen' = "lost" +Leave == screen = "lost" /\\ screen' = "gone" +Spec == Init /\\ [][Lose \\/ Leave]_screen +NeverLost == screen # "lost" +==== +`, + ); + writeFileSync(join(dir, "Echo.cfg"), "SPECIFICATION Spec\nINVARIANT NeverLost\n"); + writeFileSync( + join(dir, "Echo.map.mjs"), + 'export const state = (v) => (v.screen === "gone" ? "Away" : "Home");\nexport const step = (a) => ({ Lose: "the note is lost", Leave: "you leave" })[a];\n', + ); + const r = node("statechart.mjs", ["from-spec", "Echo.tla"], dir); + assert.equal(r.status, 1, r.stderr); + const out = join(dir, "Echo-statechart"); + const chart = readFileSync(join(out, "chart.mmd"), "utf8"); + assert.deepEqual(statusOf(chart), { Away: "after", Home: "mixed" }); + assert.deepEqual( + [...chart.matchAll(/^ {4}state "([^"]+)" as legend\w+$/gm)].map((m) => m[1]), + ["Breaks a rule in some cases", "Reached only after a bug"], + ); + const summary = JSON.parse(readFileSync(join(out, "summary.json"), "utf8")); + assert.deepEqual(summary.chartStatus, { works: 0, wrong: 0, fixed: 0, mixed: 1, after: 1 }); + assert.deepEqual(summary.mixedStatus, ["Home"]); + }, +); + +test( + "a bug story numbers the step into the bug and each step that changes what the rule reads, and names the steps it hides", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Reads.tla"), + `---- MODULE Reads ---- +EXTENDS Naturals +VARIABLES screen, reads, ticks +vars == <> +Init == screen = "Idle" /\\ reads = 0 /\\ ticks = 0 +Open == screen = "Idle" /\\ screen' = "Open" /\\ UNCHANGED <> +Read == screen = "Open" /\\ reads < 2 /\\ ticks = 2 * reads /\\ reads' = reads + 1 /\\ UNCHANGED <> +Tick == screen = "Open" /\\ ticks < 2 * reads /\\ ticks' = ticks + 1 /\\ UNCHANGED <> +Spec == Init /\\ [][Open \\/ Read \\/ Tick]_vars +ReadOnce == reads <= 1 +==== +`, + ); + writeFileSync(join(dir, "Reads.cfg"), "SPECIFICATION Spec\nINVARIANT ReadOnce\n"); + writeFileSync( + join(dir, "reads.map.mjs"), + 'export const state = (v) => v.screen;\nexport const step = (a) => ({ Open: "you open the page", Read: "the page is read" })[a] ?? "";\n', + ); + assert.match(tlc(dir, "Reads").stdout, /Invariant ReadOnce is violated/); + const run = (...flags) => { + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "Reads.dot", "--map", "reads.map.mjs", "--traces", "Reads.out", ...flags], + ...["--story", "story.md", "--summary", "summary.json", "--out", "chart.mmd"], + ], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const story = readFileSync(join(dir, "story.md"), "utf8"); + return { + steps: story.slice(story.indexOf("0. Start")), + readerSteps: JSON.parse(readFileSync(join(dir, "summary.json"), "utf8")).classes[0] + .readerSteps, + numbered: edges(parseMermaid(readFileSync(join(dir, "chart.mmd"), "utf8")).regions.Main), + }; + }; + assert.deepEqual(run("--spec", "Reads.tla"), { + steps: `0. Start: Idle +1. You open the page. Now: Open +2. The page is read. Now: Open + (2 hidden steps: Tick) +3. The page is read. Now: Open +`, + readerSteps: 3, + numbered: ["Idle -> Open : 1. you open the page", "Open -> Open : 2, 3. the page is read"], + }); + assert.equal( + run().steps, + `0. Start: Idle +1. You open the page. Now: Open + (3 hidden steps: the page is read, Tick) +2. The page is read. Now: Open +`, + ); + }, +); + +test( + "a story of a bug that only the spec has numbers a step that changes what the spec's rule reads, skips the variables that the spec lacks, and a step that the spec lacks cannot happen in the spec", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Code.tla"), + `---- MODULE Code ---- +VARIABLES s, cache +vars == <> +Init == s = "Idle" /\\ cache = "fresh" +Go == s = "Idle" /\\ s' = "Busy" /\\ UNCHANGED cache +Rot == s = "Busy" /\\ cache = "fresh" /\\ cache' = "stale" /\\ UNCHANGED s +Spec == Init /\\ [][Go \\/ Rot]_vars +NoStale == ~(s = "Busy" /\\ cache = "stale") +==== +`, + ); + writeFileSync( + join(dir, "Agreed.tla"), + `---- MODULE Agreed ---- +VARIABLES s, n +vars == <> +Init == s = "Idle" /\\ n = 0 +Go == s = "Idle" /\\ s' = "Busy" /\\ UNCHANGED n +Tick == s = "Busy" /\\ n = 0 /\\ n' = 1 /\\ UNCHANGED s +Fail == s = "Busy" /\\ n = 1 /\\ s' = "Broken" /\\ UNCHANGED n +Spec == Init /\\ [][Go \\/ Tick \\/ Fail]_vars +NoStale == ~(s = "Broken" /\\ n = 1) +==== +`, + ); + for (const name of ["Code", "Agreed"]) + writeFileSync(join(dir, `${name}.cfg`), "SPECIFICATION Spec\nINVARIANT NoStale\n"); + writeFileSync(join(dir, "Code.map.mjs"), "export const state = (v) => v.s;\n"); + const r = node( + "statechart.mjs", + ["from-spec", "Code.tla", "--against", "Agreed.tla", "--expect-broken", "NoStale"], + dir, + ); + assert.equal(r.status, 0, r.stderr); + const stories = readFileSync(join(dir, "Code-statechart", "stories.md"), "utf8"); + assert.match( + stories, + /^## 2\. Only in the spec: Broken\n[^#]*\n0\. Start: Idle\n1\. Go\. Now: Busy\n2\. Tick\. Now: Busy\n3\. Fail\. Now: Broken\n/m, + ); + assert.match( + stories, + /^1\. Go\. Now: Busy\n2\. Rot\. Now: Busy\n\nIn the spec, step 2 \(Rot\) cannot happen, so you stay at: Busy\.\n/m, + ); + }, +); + const edit = (name, fixed) => `---- MODULE ${name} ---- VARIABLES s, flashed vars == <> @@ -1719,7 +3753,7 @@ NoLostEdit == s # "Lost edit" /\\ ~(s = "Flash" /\\ flashed) `; test( - "a bug that only the fixed model has is numbered after the current bugs and drawn wrong", + "a bug that only the fixed model has is numbered after the current bugs and drawn as breaking a rule only with the fix", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); @@ -1776,15 +3810,165 @@ test( [2, "Flash", "new with the fix"], ], ); - assert.deepEqual(summary.chartStatus, { works: 2, wrong: 1, fixed: 1 }); + assert.deepEqual(summary.chartStatus, { + works: 2, + wrong: 0, + fixed: 1, + mixed: 0, + after: 0, + worse: 1, + }); + assert.equal(statusOf(readFileSync(join(dir, "chart.mmd"), "utf8")).Flash, "worse"); assert.match(readFileSync(join(dir, "story.md"), "utf8"), /^## 2\. New with the fix: Flash$/m); const two = node( "tlc-to-chart.mjs", [...args, "--highlight", "2", "--path-only", "--out", "bug2.mmd"], dir, ); - assert.equal(two.status, 0, two.stderr); - assert.match(readFileSync(join(dir, "bug2.mmd"), "utf8"), /: 2\. the screen flashes\n/); + assert.equal(two.status, 0, two.stderr); + assert.match(readFileSync(join(dir, "bug2.mmd"), "utf8"), /: 2\. the screen flashes\n/); + }, +); + +test( + "with --fixed, a state that breaks a rule only with the fix has its own status, legend line, and count", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const counter = (name, leap) => `---- MODULE ${name} ---- +EXTENDS Naturals +VARIABLE x +Init == x = 0 +CountUp == x < 2 /\\ x' = x + 1 +Leap == x \\in {1, 2} /\\ x' = 5 +Spec == Init /\\ [][CountUp${leap ? " \\/ Leap" : ""}]_x +\\* The counter never shows five. +NotFive == x # 5 +==== +`; + writeFileSync(join(dir, "Counter.tla"), counter("Counter", false)); + writeFileSync(join(dir, "CounterFixed.tla"), counter("CounterFixed", true)); + writeFileSync(join(dir, "Counter.cfg"), "SPECIFICATION Spec\nINVARIANT NotFive\n"); + writeFileSync( + join(dir, "Counter.map.mjs"), + `export const state = (v) => \`The counter shows \${v.x}\`; +export const step = (a) => ({ CountUp: "the counter goes up", Leap: "the counter leaps" })[a]; +`, + ); + const r = node( + "statechart.mjs", + ["from-spec", "Counter.tla", "--fixed", "CounterFixed.tla"], + dir, + ); + assert.equal(r.status, 1, r.stdout + r.stderr); + assert.match( + r.stdout, + /\n3 work today, 0 still wrong with the fix, 0 fixed or new with the fix, 1 breaks a rule only with the fix\.\n\n## 1\. New with the fix: The counter shows 5\n/, + ); + const out = join(dir, "Counter-statechart"); + const chart = readFileSync(join(out, "chart.mmd"), "utf8"); + assert.equal(statusOf(chart)["The counter shows 5"], "worse"); + assert.deepEqual( + [...chart.matchAll(/^ {4}state "([^"]+)" as legend\w+$/gm)].map((m) => m[1]), + ["Works today", "Breaks a rule only with the fix"], + ); + assert.deepEqual(JSON.parse(readFileSync(join(out, "summary.json"), "utf8")).chartStatus, { + works: 3, + wrong: 0, + fixed: 0, + mixed: 0, + after: 0, + worse: 1, + }); + }, +); + +test( + "only --cover trims the overview, and it lists what it leaves out and counts every state", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Route.tla"), + `---- MODULE Route ---- +VARIABLE x +Init == x = "start" +Side == x = "start" /\\ x' = "porch" +Front == x = "start" /\\ x' = "hall" +Arrive == x = "porch" /\\ x' = "attic" +Leave == x = "attic" /\\ x' = "start" +Spec == Init /\\ [][Side \\/ Front \\/ Arrive \\/ Leave]_x +NoPorch == x # "porch" +==== +`, + ); + writeFileSync(join(dir, "Route.cfg"), "SPECIFICATION Spec\nINVARIANT NoPorch\n"); + const mapping = (readerActions) => + `export const readerActions = ${JSON.stringify(readerActions)}; +export const state = (v) => v.x; +export const step = (a) => ({ Side: "you take the side door", Front: "you take the front door", Arrive: "you climb the stairs", Leave: "the lights go out" })[a]; +`; + writeFileSync(join(dir, "Route.map.mjs"), mapping(["Side", "Front", "Arrive"])); + writeFileSync(join(dir, "nobody.map.mjs"), mapping(["Knock"])); + const run = (out, ...flags) => { + const r = node( + "statechart.mjs", + ["from-spec", "Route.tla", "--expect-broken", "NoPorch", "--out", out, ...flags], + dir, + ); + assert.equal(r.status, 0, r.stdout + r.stderr); + return { r, summary: JSON.parse(readFileSync(join(dir, out, "summary.json"), "utf8")) }; + }; + + const whole = run("whole"); + assert.equal(whole.summary.chartStates, 4); + assert.equal(whole.summary.chartTransitions, 4); + assert.deepEqual(whole.summary.chartStatus, { + works: 2, + wrong: 1, + fixed: 0, + mixed: 0, + after: 1, + }); + assert.equal(whole.summary.cover, undefined); + assert.doesNotMatch(whole.r.stdout, /--cover/); + + const covered = run("covered", "--cover"); + assert.equal(covered.summary.chartStates, 3); + assert.equal(covered.summary.chartTransitions, 2); + assert.deepEqual(covered.summary.chartStatus, { + works: 2, + wrong: 1, + fixed: 0, + mixed: 0, + after: 1, + }); + assert.deepEqual(covered.summary.cover, { + drawn: true, + states: 4, + transitions: 4, + leftOutStates: ["attic"], + leftOutTransitions: [ + "attic --> start: the lights go out", + "porch --> attic: you climb the stairs", + ], + }); + assert.match( + covered.r.stdout, + /^--cover leaves out 1 of 4 states and 2 of 4 transitions\. The counts of states that work or break a rule include the states it leaves out\.\nStates left out:\n {2}attic\nTransitions left out:\n {2}attic --> start: the lights go out\n {2}porch --> attic: you climb the stairs\n2 work, 1 breaks a rule, 1 reached only after a bug\.$/m, + ); + + const none = run("none", "--cover", "--map", "nobody.map.mjs"); + assert.equal(none.summary.chartStates, 4); + assert.equal(none.summary.chartTransitions, 4); + assert.deepEqual(none.summary.cover, { + drawn: false, + why: "no step in TLC's graph is a reader action", + }); + assert.match( + none.r.stdout, + /^--cover draws every state, because no step in TLC's graph is a reader action\.$/m, + ); }, ); @@ -1946,17 +4130,18 @@ TLC on SaveAgreed.tla with SaveCode.cfg: NoFalseSaved holds. TLC ran 3 times in N s. -Chart: 4 states and 4 transitions from 10 model states. +Chart: 4 states and 6 transitions from 10 model states. 3 work, 1 breaks a rule, 0 only in the agreed chart. ## 1. Saved, draft Rule broken: You never see "Saved" while the note is still a draft. Why: the app stops after one retry and shows Saved anyway. -There are 2 ways to get here in the model. This is the shortest. +In the model, one step leads into this bug: "the save times out". The steps below are the shortest path that TLC found. 0. Start: Editing, draft 1. You press Save. Now: Saving, draft + (1 hidden step: the app tries again) 2. The save times out. Now: Saved, draft In the agreed chart, step 2 (the save times out) cannot happen, so you stay at: Saving, draft. @@ -1983,7 +4168,7 @@ ${uncitedWarning}`; ); assert.match( chart, - /^ {2}s3 --> s3 : in the agreed chart, the savetimes out cannot happen$/m, + /^ {2}s3 --> s3 : in the agreed chart, the savetimes out cannot happen atstep 2 of bug 1$/m, ); const bug = readFileSync(join(out, "bug-1.mmd"), "utf8"); assert.deepEqual([...bug.matchAll(/^ {2}state "([^"]+)" as s\d+$/gm)].map((m) => m[1]).sort(), [ @@ -2038,11 +4223,33 @@ ${uncitedWarning}`; "${v.note.text} r${v.note.rev}`", ), ); - const lacks = run("--map", "rev.map.mjs", "--expect-broken", "NoFalseSaved"); + const tmp = join(dir, "tmp"); + mkdirSync(tmp); + const lacks = node( + "statechart.mjs", + [ + ...base, + "--against-config", + "SaveCode.cfg", + "--map", + "rev.map.mjs", + "--expect-broken", + "NoFalseSaved", + ], + dir, + { TMPDIR: tmp }, + ); assert.equal(lacks.status, 2); + const [, kept] = lacks.stderr.match( + / The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/, + ); assert.equal( lacks.stderr, - `${warnings}rev.map.mjs: the agreed chart has no note.rev, and the mapping reads v.note.rev on its states. Label states and steps only from what both models have, or test "rev" in v.note first.\n`, + `${warnings}rev.map.mjs: the agreed chart has no note.rev, and the mapping reads v.note.rev on its states. Label states and steps only from what both models have, or test "rev" in v.note first. The files that this run wrote are in ${kept}.\n`, + ); + assert.match( + readFileSync(join(dir, kept, "tlc", "SaveCode-NoFalseSaved.out"), "utf8"), + /^Error: Invariant NoFalseSaved is violated/m, ); writeFileSync( join(dir, "rev.map.mjs"), @@ -2113,8 +4320,603 @@ ${uncitedWarning}`; const fresh = run("--expect-broken", "NoFalseSaved"); assert.equal(fresh.status, 0, fresh.stderr); assert.match( - fresh.stdout, - /\nNot listed: no config of SaveAgreed\.tla expects TLC to break this rule\. Its configs expect TLC to break `TypeOK`\.\n/, + fresh.stdout, + /\nNot listed: no config of SaveAgreed\.tla expects TLC to break this rule\. Its configs expect TLC to break `TypeOK`\.\n/, + ); + }, +); + +const cartCode = `---- MODULE CartCode ---- +VARIABLES phase, note, tag, mark, extra +vars == <> +Init == phase = "cart" /\\ note = [text |-> "hi"] /\\ tag = [id |-> "a"] /\\ mark = [on |-> TRUE] /\\ extra = "kept" +Pay == phase = "cart" /\\ phase' = "paid" /\\ note' = [text |-> note.text, rev |-> 1] /\\ UNCHANGED <> +Back == phase = "paid" /\\ phase' = "cart" /\\ UNCHANGED <> +Spec == Init /\\ [][Pay \\/ Back]_vars +Ok == phase \\in {"cart", "paid"} +==== +`; +const cartAgreed = `---- MODULE CartAgreed ---- +VARIABLES phase, note, tag, mark +vars == <> +Init == phase = "cart" /\\ note = [text |-> "hi"] /\\ tag = "a" /\\ mark = [on |-> TRUE] +Pay == phase = "cart" /\\ phase' = "paid" /\\ mark' = [off |-> TRUE] /\\ UNCHANGED <> +Back == phase = "paid" /\\ phase' = "cart" /\\ UNCHANGED <> +Spec == Init /\\ [][Pay \\/ Back]_vars +Ok == phase \\in {"cart", "paid"} +==== +`; + +test( + "from-spec --against stops when the mapping reads what the spec lacks through a spread, a later state, a field of another type, or a field some states lack", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "CartCode.tla"), cartCode); + writeFileSync(join(dir, "CartAgreed.tla"), cartAgreed); + writeFileSync(join(dir, "CartCode.cfg"), "SPECIFICATION Spec\nINVARIANT Ok\n"); + const map = (name, state, more = "") => + writeFileSync( + join(dir, name), + `export const readerActions = ["Pay", "Back"]; +export const state = ${state}; +export const step = (a) => ({ Pay: "you pay", Back: "you go back" })[a]; +${more}`, + ); + const tmp = join(dir, "tmp"); + mkdirSync(tmp); + const stops = (name, message) => { + const r = node( + "statechart.mjs", + [ + "from-spec", + "CartCode.tla", + "--against", + "CartAgreed.tla", + "--label", + "the agreed chart", + "--against-config", + "CartCode.cfg", + "--map", + name, + ], + dir, + { TMPDIR: tmp }, + ); + assert.equal(r.status, 2, r.stdout); + const [, kept] = + r.stderr.match(/ The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/) ?? []; + assert.equal(r.stderr, `${message} The files that this run wrote are in ${kept}.\n`); + }; + + map("spread.map.mjs", "(v) => { const c = { ...v }; return `${c.phase} ${c.extra}`; }"); + stops( + "spread.map.mjs", + "spread.map.mjs: the agreed chart has no extra, and the mapping lists the fields of v on its states. Label states and steps only from what both models have, and read the fields it needs by name.", + ); + + map("later.map.mjs", "(v) => `${v.phase} ${v.note.rev}`"); + stops( + "later.map.mjs", + 'later.map.mjs: the agreed chart has no note.rev, and the mapping reads v.note.rev on its states. Label states and steps only from what both models have, or test "rev" in v.note first.', + ); + + map("typed.map.mjs", "(v) => `${v.phase} ${v.tag.id}`"); + stops( + "typed.map.mjs", + "typed.map.mjs: the agreed chart has tag as a value of another type, and the mapping reads v.tag on its states. Label states and steps only from what both models have.", + ); + + map("partial.map.mjs", "(v) => `${v.phase} ${v.mark.on}`"); + stops( + "partial.map.mjs", + 'partial.map.mjs: the mapping labels a state of the agreed chart "paid undefined", so it reads a field the agreed chart does not have there. Label states and steps only from what both models have.', + ); + + map( + "guarded.map.mjs", + "(v) => v.phase", + "export const guard = (a, b) => `tag ${b.mark.on}`;\n", + ); + stops( + "guarded.map.mjs", + 'guarded.map.mjs: the mapping labels a step of the agreed chart "tag undefined", so it reads a field the agreed chart does not have there. Label states and steps only from what both models have.', + ); + }, +); + +const seatsSpec = `---- MODULE Seats ---- +VARIABLES seat, grid +vars == <> +Init == seat = <<"empty", "empty">> /\\ grid = << <<"a", "b">>, <<"c", "d">> >> +Fill == seat[1] = "empty" /\\ seat' = [seat EXCEPT ![1] = "full"] /\\ UNCHANGED grid +Next == Fill +Spec == Init /\\ [][Next]_vars +==== +`; + +function seats() { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "Seats.tla"), seatsSpec); + writeFileSync(join(dir, "Seats.cfg"), "SPECIFICATION Spec\n"); + const draw = (name, map) => { + writeFileSync(join(dir, `${name}.map.mjs`), map); + return node( + "statechart.mjs", + ["from-spec", "Seats.tla", "--map", `${name}.map.mjs`, "--out", name], + dir, + ); + }; + const chartOf = (name) => readFileSync(join(dir, name, "chart.mmd"), "utf8"); + const states = (name, map) => { + const r = draw(name, map); + assert.equal(r.status, 0, r.stderr); + return Object.keys(statusOf(chartOf(name))).sort(); + }; + return { draw, chartOf, states }; +} + +test( + "a map reads a tuple from 1, as in TLA+, stops when it reads index 0, and keeps length, array methods, spread, and JSON", + needsTlc, + () => { + const { draw, chartOf } = seats(); + + const first = draw( + "first", + 'export const state = (v) => `first seat ${v.seat[1]}`;\nexport const step = () => "you take the first seat";\n', + ); + assert.equal(first.status, 0, first.stderr); + assert.deepEqual(edges(parseMermaid(chartOf("first")).regions.Main), [ + "first seat empty -> first seat full : you take the first seat", + ]); + + const zero = draw("zero", "export const state = (v) => `first seat ${v.seat[0]}`;\n"); + assert.equal(zero.status, 2, zero.stdout); + assert.match( + zero.stderr, + /^zero\.map\.mjs: the mapping reads v\.seat\[0\], .*reads from 1.*\bv\.seat\[1\]/m, + ); + + const shapes = draw( + "shapes", + 'export const state = (v) =>\n `${v.seat.filter((s) => s === "full").length} of ${v.seat.length} full, ${[...v.seat].join(" and ")}, corner ${v.grid[1][2]}`;\nexport const step = (action, before, after) => JSON.stringify(after);\n', + ); + assert.equal(shapes.status, 0, shapes.stderr); + const chart = chartOf("shapes"); + assert.deepEqual(Object.keys(statusOf(chart)).sort(), [ + "0 of 2 full, empty and empty, corner b", + "1 of 2 full, full and empty, corner b", + ]); + assert.deepEqual( + [...chart.matchAll(/^ {2}s\d+ --> s\d+ : (.*)$/gm)].map((m) => m[1]), + [ + JSON.stringify({ + grid: [ + ["a", "b"], + ["c", "d"], + ], + seat: ["full", "empty"], + }), + ], + ); + }, +); + +const zeroIndex = (at) => + new RegExp( + String.raw`the mapping looks for index 0 in ${at}, .*reads from 1, as in TLA\+\. Use \[\.\.\.${at}\] for an array that reads from 0\.`, + ); + +test( + "a map's in and hasOwnProperty see a tuple's indexes from 1, and asking for index 0 stops the run", + needsTlc, + () => { + const { draw, states } = seats(); + assert.deepEqual( + states( + "has", + "export const state = (v) =>\n `${v.seat[1]}: ${[1, 2, 3].map((i) => i in v.seat)} ${v.seat.hasOwnProperty(2)}`;\n", + ), + ["empty: true,true,false true", "full: true,true,false true"], + ); + + const zero = draw("zero", "export const state = (v) => `${0 in v.seat}`;\n"); + assert.equal(zero.status, 2, zero.stdout); + assert.match( + zero.stderr, + new RegExp(`^zero\\.map\\.mjs: ${zeroIndex("v\\.seat").source}`, "m"), + ); + }, +); + +test("a map's Object.keys and Object.entries list a tuple's indexes from 1", needsTlc, () => { + const { states } = seats(); + assert.deepEqual( + states( + "keys", + 'export const state = (v) =>\n `${v.seat[1]}: ${Object.keys(v.seat)} ${Object.entries(v.seat).map(([i, s]) => `${i}=${s}`).join(" ")}`;\n', + ), + ["empty: 1,2 1=empty 2=empty", "full: 1,2 1=full 2=empty"], + ); +}); + +test("a map that spreads a tuple into an object gets its items at keys from 1", needsTlc, () => { + const { states } = seats(); + assert.deepEqual( + states( + "spread", + "export const state = (v) => {\n const s = { ...v.seat };\n return `${s[1]} ${s[2]} ${0 in s}`;\n};\n", + ), + ["empty empty false", "full empty false"], + ); +}); + +test( + "a map that hands a tuple to concat stops the run and names the tuple it wrote", + needsTlc, + () => { + const { draw } = seats(); + const r = draw("concat", 'export const state = (v) => [].concat(v.seat).join(" ");\n'); + assert.equal(r.status, 2, r.stdout); + assert.match(r.stderr, new RegExp(`^concat\\.map\\.mjs: ${zeroIndex("v\\.seat").source}`, "m")); + }, +); + +test( + "a map flattens nested tuples with flat and flatMap, and flattening an array that holds a tuple stops the run", + needsTlc, + () => { + const { draw, states } = seats(); + assert.deepEqual( + states( + "flat", + 'export const state = (v) =>\n `${v.seat[1]} ${v.grid.flat().join("")} ${v.grid.flatMap((row) => row).join("")} ${v.grid.flat(Infinity).join("")}`;\n', + ), + ["empty abcd abcd abcd", "full abcd abcd abcd"], + ); + + const inArray = draw("inarray", 'export const state = (v) => [v.seat].flat().join(" ");\n'); + assert.equal(inArray.status, 2, inArray.stdout); + assert.match( + inArray.stderr, + new RegExp(`^inarray\\.map\\.mjs: ${zeroIndex("v\\.seat").source}`, "m"), + ); + }, +); + +test("a map flattens a set of tuples with flat and flatMap", needsTlc, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pairs.tla"), + `---- MODULE Pairs ---- +VARIABLES sent +Init == sent = {<<1, "a">>} +Send == sent = {<<1, "a">>} /\\ sent' = sent \\cup {<<2, "b">>} +Spec == Init /\\ [][Send]_sent +==== +`, + ); + writeFileSync(join(dir, "Pairs.cfg"), "SPECIFICATION Spec\n"); + writeFileSync( + join(dir, "Pairs.map.mjs"), + 'export const state = (v) =>\n `${v.sent.flat().join("")} ${v.sent.flatMap((m) => m).join("")} ${structuredClone(v.sent).flat().join("")}`;\n', + ); + const r = node("statechart.mjs", ["from-spec", "Pairs.tla", "--out", "out"], dir); + assert.equal(r.status, 0, r.stderr); + assert.deepEqual( + Object.keys(statusOf(readFileSync(join(dir, "out", "chart.mmd"), "utf8"))).sort(), + ["1a 1a 1a", "1a2b 1a2b 1a2b"], + ); +}); + +test("a map's structuredClone copies tuples that still read and write from 1", needsTlc, () => { + const { draw, states } = seats(); + assert.deepEqual( + states( + "clone", + 'export const state = (v) => {\n const c = structuredClone(v);\n c.seat[2] = "held";\n return `${c.seat[1]} ${c.seat[2]} ${v.seat[2]} ${c.grid[2][1]} ${Object.keys(c.seat)} ${structuredClone(v.grid)[2][2]}`;\n};\n', + ), + ["empty held empty c 1,2 d", "full held empty c 1,2 d"], + ); + + const zero = draw( + "clonezero", + 'export const state = (v) => {\n const c = structuredClone(v);\n c.seat[0] = "held";\n return c.seat[1];\n};\n', + ); + assert.equal(zero.status, 2, zero.stdout); + assert.match( + zero.stderr, + /^clonezero\.map\.mjs: the mapping writes v\.seat\[0\], but a tuple reads from 1, as in TLA\+\. Write v\.seat\[1\] for its first item\./m, + ); +}); + +test( + "from-spec --against draws a step that step() does not name by its action in the developer and layered views", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const packing = (name, more = "") => `---- MODULE ${name} ---- +VARIABLES phase +Init == phase = "cart" +Pay == phase = "cart" /\\ phase' = "paid" +Pack == phase = "paid" /\\ phase' = "packed" +Ship == phase = "packed" /\\ phase' = "shipped" +${more}Spec == Init /\\ [][Pay \\/ Pack \\/ Ship${more && " \\/ Refund"}]_phase +NotShipped == phase # "shipped" +==== +`; + writeFileSync(join(dir, "PackCode.tla"), packing("PackCode")); + writeFileSync( + join(dir, "PackAgreed.tla"), + packing("PackAgreed", `Refund == phase = "paid" /\\ phase' = "cart"\n`), + ); + writeFileSync(join(dir, "PackCode.cfg"), "SPECIFICATION Spec\nINVARIANT NotShipped\n"); + writeFileSync( + join(dir, "pack.map.mjs"), + `export const readerActions = ["Pay"]; +export const state = (v) => v.phase; +export const detail = (v) => v.phase; +export const step = (a) => ({ Pay: "you pay", Ship: "we ship" })[a]; +`, + ); + const r = node( + "statechart.mjs", + [ + ...["from-spec", "PackCode.tla", "--against", "PackAgreed.tla"], + ...["--label", "the agreed chart", "--against-config", "PackCode.cfg"], + ...["--map", "pack.map.mjs", "--out", "out"], + ], + dir, + ); + assert.equal(r.status, 1, r.stderr); + assert.doesNotMatch(r.stderr, /the mapping labels/); + for (const view of ["developer", "layered"]) { + const chart = readFileSync(join(dir, "out", `bug-1-${view}.mmd`), "utf8"); + assert.doesNotMatch(chart, /undefined/, view); + assert.match(chart, /--> s\d+ : 2\. Pack$/m, view); + } + }, +); + +test( + "from-spec --against compares a rule's variable of another type without blaming the mapping", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "ShopCode.tla"), + `---- MODULE ShopCode ---- +VARIABLES phase, tag +vars == <> +Init == phase = "cart" /\\ tag = [id |-> "a"] +Pay == phase = "cart" /\\ phase' = "paid" /\\ UNCHANGED tag +Spec == Init /\\ [][Pay]_vars +Ok == ~(phase = "paid" /\\ tag = tag) +==== +`, + ); + writeFileSync( + join(dir, "ShopAgreed.tla"), + `---- MODULE ShopAgreed ---- +VARIABLES phase, tag, seen +vars == <> +Init == phase = "cart" /\\ tag = "a" /\\ seen = FALSE +Peek == ~seen /\\ seen' = TRUE /\\ tag' = "b" /\\ UNCHANGED phase +Pay == phase = "cart" /\\ phase' = "paid" /\\ UNCHANGED <> +Spec == Init /\\ [][Peek \\/ Pay]_vars +Ok == ~(phase = "paid" /\\ tag = tag) +==== +`, + ); + writeFileSync(join(dir, "ShopCode.cfg"), "SPECIFICATION Spec\nINVARIANT Ok\n"); + writeFileSync( + join(dir, "shop.map.mjs"), + `export const readerActions = ["Pay"]; +export const state = (v) => v.phase; +export const step = (a) => ({ Pay: "you pay" })[a]; +`, + ); + const r = node( + "statechart.mjs", + [ + "from-spec", + "ShopCode.tla", + "--against", + "ShopAgreed.tla", + "--label", + "the agreed chart", + "--against-config", + "ShopCode.cfg", + "--map", + "shop.map.mjs", + ], + dir, + ); + assert.equal(r.status, 1, r.stderr); + assert.doesNotMatch(r.stderr, /the mapping reads/); + }, +); + +const shipCode = `---- MODULE ShipCode ---- +EXTENDS Naturals +VARIABLES phase, n, mark +vars == <> +Init == phase = "cart" /\\ n = 0 /\\ mark = [on |-> TRUE] +Pay == phase = "cart" /\\ phase' = "paid" /\\ n' = 1 /\\ UNCHANGED mark +Ship == phase = "paid" /\\ phase' = "shipped" /\\ UNCHANGED <> +Spec == Init /\\ [][Pay \\/ Ship]_vars +Ok == phase \\in {"cart", "paid", "shipped"} +NotShipped == phase # "shipped" +==== +`; +const shipAgreed = `---- MODULE ShipAgreed ---- +VARIABLES phase, n, mark +vars == <> +Init == phase = "cart" /\\ n = "0" /\\ mark = [on |-> TRUE] +Pay == phase = "cart" /\\ phase' = "paid" /\\ n' = "1" /\\ UNCHANGED mark +Ship == phase = "paid" /\\ phase' = "shipped" /\\ mark' = "off" /\\ UNCHANGED n +Spec == Init /\\ [][Pay \\/ Ship]_vars +Ok == phase \\in {"cart", "paid", "shipped"} +NotShipped == phase # "shipped" +==== +`; +const shipMap = (state) => `export const readerActions = ["Pay", "Ship"]; +export const state = ${state}; +export const step = (a) => ({ Pay: "you pay", Ship: "we ship" })[a]; +`; +const stopsWith = (r, message) => { + assert.equal(r.status, 2, r.stdout); + const [, kept] = + r.stderr.match(/ The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/) ?? []; + assert.equal(r.stderr, `${message} The files that this run wrote are in ${kept}.\n`); +}; +const codeAgainstAgreed = [ + ...["ShipCode.tla", "--against", "ShipAgreed.tla", "--label", "the agreed chart"], + ...["--against-config", "ShipCode.cfg"], +]; +function shipAgainst(map, models = codeAgainstAgreed, agreed = shipAgreed) { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "ShipCode.tla"), shipCode); + writeFileSync(join(dir, "ShipAgreed.tla"), agreed); + writeFileSync(join(dir, "ShipCode.cfg"), "SPECIFICATION Spec\nINVARIANT Ok\n"); + writeFileSync(join(dir, "ship.map.mjs"), map); + mkdirSync(join(dir, "tmp")); + return node("statechart.mjs", ["from-spec", ...models, "--map", "ship.map.mjs"], dir, { + TMPDIR: join(dir, "tmp"), + }); +} + +test( + "from-spec --against stops when the mapping reads a field that one model has as a number and the other as a string", + needsTlc, + () => { + stopsWith( + shipAgainst(shipMap('(v) => `${v.phase}, ${v.n === 0 ? "free" : "charged"}`')), + "ship.map.mjs: the agreed chart has n as a value of another type, and the mapping reads v.n on its states. Label states and steps only from what both models have.", + ); + stopsWith( + shipAgainst(shipMap('(v) => `${v.phase}, ${v.n === "0" ? "free" : "charged"}`'), [ + "ShipAgreed.tla", + "--config", + "ShipCode.cfg", + "--against", + "ShipCode.tla", + ]), + "ship.map.mjs: the spec has n as a value of another type, and the mapping reads v.n on its states. Label states and steps only from what both models have.", + ); + }, +); + +test( + "from-spec --against stops when the mapping reads a field that is a record in some of the spec's states and not in others", + needsTlc, + () => { + stopsWith( + shipAgainst(shipMap('(v) => `${v.phase}, ${v.mark.on ? "marked" : "plain"}`')), + "ship.map.mjs: the agreed chart has mark as a value of another type, and the mapping reads v.mark on its states. Label states and steps only from what both models have.", + ); + + const where = shipAgainst( + shipMap( + '(v) => (v.phase === "shipped" ? v.phase : `${v.phase}, ${v.mark.on ? "marked" : "plain"}`)', + ), + ); + assert.equal(where.status, 0, where.stderr); + }, +); + +test( + "from-spec --against checks the fields of a record that is a record in only some of the spec's states", + needsTlc, + () => { + const recordStates = shipMap( + '(v) => (v.phase === "shipped" ? v.phase : `${v.phase}, ${v.mark.on === true ? "marked" : "plain"}`)', + ); + stopsWith( + shipAgainst(recordStates, codeAgainstAgreed, shipAgreed.replaceAll("[on |->", "[off |->")), + 'ship.map.mjs: the agreed chart has no mark.on, and the mapping reads v.mark.on on its states. Label states and steps only from what both models have, or test "on" in v.mark first.', + ); + stopsWith( + shipAgainst(recordStates, codeAgainstAgreed, shipAgreed.replaceAll("TRUE", '"yes"')), + "ship.map.mjs: the agreed chart has mark.on as a value of another type, and the mapping reads v.mark.on on its states. Label states and steps only from what both models have.", + ); + }, +); + +test( + "tlc-to-chart stops when the mapping reads a field of another type that only a trace of the second model has", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "ShipCode.tla"), shipCode); + writeFileSync(join(dir, "ShipAgreed.tla"), shipAgreed); + writeFileSync(join(dir, "ShipCode.cfg"), "SPECIFICATION Spec\nINVARIANT NotShipped\n"); + writeFileSync(join(dir, "ShipAgreed.cfg"), "SPECIFICATION Spec\nINVARIANT NotShipped\n"); + tlc(dir, "ShipCode"); + tlc(dir, "ShipAgreed"); + renameSync(join(dir, "ShipAgreed.out"), join(dir, "ShipAgreedTraces.out")); + writeFileSync(join(dir, "ShipAgreed.cfg"), "SPECIFICATION Spec\nCONSTRAINT NotShipped\n"); + tlc(dir, "ShipAgreed"); + assert.doesNotMatch(readFileSync(join(dir, "ShipAgreed.dot"), "utf8"), /shipped/); + writeFileSync( + join(dir, "ship.map.mjs"), + shipMap('(v) => `${v.phase}, ${v.mark.on ? "marked" : "plain"}`'), + ); + const r = node( + "tlc-to-chart.mjs", + [ + ...["--dot", "ShipCode.dot", "--map", "ship.map.mjs", "--traces", "ShipCode.out"], + ...["--against-dot", "ShipAgreed.dot", "--against-traces", "ShipAgreedTraces.out"], + ...["--against-rules", "NotShipped", "--label", "the agreed chart", "--out", "chart.mmd"], + ], + dir, + ); + assert.equal(r.status, 1, r.stdout); + assert.match( + r.stderr, + /^Error: the agreed chart has mark as a value of another type, and the mapping reads v\.mark on its states\./m, + ); + }, +); + +test( + "from-spec --against names the group when the mapping groups a state of the spec by a field that state lacks", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync(join(dir, "CartCode.tla"), cartCode); + writeFileSync(join(dir, "CartAgreed.tla"), cartAgreed); + writeFileSync(join(dir, "CartCode.cfg"), "SPECIFICATION Spec\nINVARIANT Ok\n"); + writeFileSync( + join(dir, "group.map.mjs"), + `export const readerActions = ["Pay", "Back"]; +export const state = (v) => v.phase; +export const step = (a) => ({ Pay: "you pay", Back: "you go back" })[a]; +export const group = (v) => \`mark \${v.mark.on}\`; +`, + ); + mkdirSync(join(dir, "tmp")); + const r = node( + "statechart.mjs", + [ + "from-spec", + "CartCode.tla", + "--against", + "CartAgreed.tla", + "--label", + "the agreed chart", + "--against-config", + "CartCode.cfg", + "--map", + "group.map.mjs", + ], + dir, + { TMPDIR: join(dir, "tmp") }, + ); + stopsWith( + r, + 'group.map.mjs: the mapping puts a state of the agreed chart in group "mark undefined", so it reads a field the agreed chart does not have there. Label states and steps only from what both models have.', ); }, ); @@ -2220,6 +5022,44 @@ test( }, ); +test( + "from-spec draws parallel regions whose arrows together make a step back to the same state, which the overview leaves out", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Knobs.tla"), + `---- MODULE Knobs ---- +VARIABLES dial, light +vars == <> +Init == dial = "low" /\\ light = "off" +Turn == dial = "low" /\\ light = "off" /\\ dial' \\in {"low", "mid"} /\\ light' \\in {"off", "on"} +Push == dial = "mid" /\\ dial' = "high" /\\ UNCHANGED light +Next == Turn \\/ Push +Spec == Init /\\ [][Next]_vars +==== +`, + ); + writeFileSync(join(dir, "Knobs.cfg"), "SPECIFICATION Spec\nCHECK_DEADLOCK FALSE\n"); + writeFileSync( + join(dir, "Knobs.map.mjs"), + `export const regionNames = ["Dial", "Light"]; +export const state = (v) => \`\${v.dial}: \${v.light}\`; +export const step = (a) => ({ Turn: "you turn", Push: "you push" })[a]; +`, + ); + const r = node("statechart.mjs", ["from-spec", "Knobs.tla"], dir); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match(r.stdout, /\nIt has 2 parallel regions, of 3 and 2 states\. /); + const again = node( + "compare-charts.mjs", + ["tlc/regions-all.mmd", "tlc/regions-proof.mmd"], + join(dir, "Knobs-statechart"), + ); + assert.equal(again.stdout, "same\n", again.stderr); + }, +); + test( "the mapping's guard words show on an arrow only when every model step behind it has them", needsTlc, @@ -2379,6 +5219,8 @@ test( "probe.map.mjs", "--traces", "Door-NoLostReply.out", + "--broken", + "Door.out", "--highlight", "0", ...extra, @@ -2422,11 +5264,10 @@ export const group = (v) => (v.reply === "shown" ? "Reply shown" : "Waiting"); assert.deepEqual( coarse.stdout.split("\n").filter((l) => /^ {2}(class|classDef) /.test(l)), [ - " classDef plain fill:#f4f6f7,stroke:#566573,color:#1b2631", - " classDef plainWrong fill:#f4f6f7,stroke:#c0392b,stroke-width:4px,color:#7b241c", - " class legendWrong plainWrong", - " class s0 plainWrong", - " class s1 plain", + " classDef plainMixed fill:#f4f6f7,stroke:#d35400,stroke-width:4px,color:#7e5109", + " class legendMixed plainMixed", + " class s0 plainMixed", + " class s1 plainMixed", ], ); const summary = JSON.parse(readFileSync(join(tlcDir, "coarse.json"), "utf8")); @@ -2493,19 +5334,25 @@ test( dir, ); assert.equal(r.status, 1, r.stderr); - const chart = readFileSync(join(dir, "Order-statechart", "chart.mmd"), "utf8"); + const transitions = (f) => + parseMermaid(readFileSync(join(dir, "Order-statechart", f), "utf8")).regions.Main + .transitions; return { story: readFileSync(join(dir, "Order-statechart", "stories.md"), "utf8"), - fixEdges: parseMermaid(chart) - .regions.Main.transitions.filter((t) => t.label.includes("with the fix")) + fixEdges: transitions("chart.mmd") + .filter((t) => t.label.includes("with the fix")) .map((t) => `${t.from} --> ${t.to} : ${t.label}`), + bugLoops: transitions("bug-1.mmd") + .filter((t) => t.from === t.to) + .map((t) => `${t.from} --> ${t.to} : ${t.label}`) + .sort(), }; }; const hidden = withFix(chargeAgain); assert.deepEqual(hidden.fixEdges, []); assert.match( hidden.story, - /^With the fix you see the same screens and steps, so the fix does not change this path\.$/m, + /^With the fix you see the same states and steps, so the fix does not change this path\.$/m, ); const once = withFix(chargeOnce); assert.deepEqual(once.fixEdges, [ @@ -2523,13 +5370,242 @@ test( assert.deepEqual(unseen.fixEdges, ["Paying --> Paid : with the fix, the bank charges you"]); assert.match( unseen.story, - /^With the fix, after step 1, "the bank charges you" happens and takes you to: Paid\.$/m, + /^With the fix, step 3 \(the bank charges you\) takes you to: Paid\.$/m, ); const blocked = withFix(chargeFirstOnly, unlabeledMap); - assert.deepEqual(blocked.fixEdges, []); + assert.deepEqual(blocked.fixEdges, [ + "Paying --> Paying : with the fix, the bank charges you cannot happen at step 3 of bug 1", + ]); + assert.deepEqual(blocked.bugLoops, [ + "Paying --> Paying : 2, 3. the bank charges you", + "Paying --> Paying : with the fix, the bank charges you cannot happen at step 3", + ]); assert.match( blocked.story, - /yet the rule holds, so the state labels do not show what the fix changes/, + /^With the fix, step 3 \(the bank charges you\) cannot happen, so you stay at: Paying\.$/m, + ); + }, +); + +test( + "a step that the fix cannot take keeps its own loop when the fix keeps another bug on the same state", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const shop = (name, steps) => `---- MODULE ${name} ---- +VARIABLES screen, charged, refunded, declined +vars == <> +Init == screen = "Cart" /\\ charged = FALSE /\\ refunded = FALSE /\\ declined = FALSE +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED <> +${steps} +Spec == Init /\\ [][Next]_vars +NoCharge == ~charged +NoRefund == ~refunded +==== +`; + writeFileSync( + join(dir, "Shop.tla"), + shop( + "Shop", + `Charge == screen = "Paying" /\\ ~charged /\\ charged' = TRUE /\\ UNCHANGED <> +Refund == screen = "Paying" /\\ ~refunded /\\ refunded' = TRUE /\\ UNCHANGED <> +Next == Pay \\/ Charge \\/ Refund`, + ), + ); + writeFileSync( + join(dir, "ShopFixed.tla"), + shop( + "ShopFixed", + `Charge == screen = "Paying" /\\ ~declined /\\ declined' = TRUE /\\ UNCHANGED <> +Next == Pay \\/ Charge`, + ), + ); + writeFileSync( + join(dir, "Shop.cfg"), + "SPECIFICATION Spec\nINVARIANT NoCharge\nINVARIANT NoRefund\n", + ); + writeFileSync( + join(dir, "Shop.map.mjs"), + `export const readerActions = ["Pay", "Charge", "Refund"]; +export const state = (v) => v.screen; +export const step = (a, v, w) => ({ Pay: "you pay", Charge: w.charged ? "the bank charges you" : "the bank declines", Refund: "the bank refunds you" })[a]; +export const cause = () => null; +`, + ); + const r = node("statechart.mjs", ["from-spec", "Shop.tla", "--fixed", "ShopFixed.tla"], dir); + assert.equal(r.status, 1, r.stderr); + const loops = parseMermaid(readFileSync(join(dir, "Shop-statechart", "chart.mmd"), "utf8")) + .regions.Main.transitions.filter((t) => t.from === t.to) + .map((t) => t.label) + .sort(); + assert.deepEqual(loops, [ + "the bank charges you or refunds you with the fix, the bank declines", + "with the fix, the bank refunds you cannot happen at step 2 of bug 2", + ]); + }, +); + +test( + "the replay on the fix skips only the steps that the story hides, so a step that the fix stops cannot happen", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Order.tla"), + `---- MODULE Order ---- +EXTENDS Naturals +VARIABLES screen, charges +vars == <> +Init == screen = "Cart" /\\ charges = 0 +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED charges +Charge == screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen +Spec == Init /\\ [][Pay \\/ Charge]_vars +NoDoubleCharge == charges <= 1 +==== +`, + ); + writeFileSync( + join(dir, "OrderFixed.tla"), + `---- MODULE OrderFixed ---- +EXTENDS Naturals +VARIABLES screen, charges +vars == <> +Init == screen = "Cart" /\\ charges = 0 +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ UNCHANGED charges +Charge == screen = "Paying" /\\ charges < 1 /\\ charges' = charges + 1 /\\ UNCHANGED screen +Refund == charges = 1 /\\ charges' = 0 /\\ UNCHANGED screen +Spec == Init /\\ [][Pay \\/ Charge \\/ Refund]_vars +NoDoubleCharge == charges <= 1 +==== +`, + ); + writeFileSync(join(dir, "Order.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Order.map.mjs"), "export const state = (v) => v.screen;\n"); + const r = node("statechart.mjs", ["from-spec", "Order.tla", "--fixed", "OrderFixed.tla"], dir); + assert.equal(r.status, 1, r.stderr); + const story = readFileSync(join(dir, "Order-statechart", "stories.md"), "utf8"); + assert.match( + story, + /\n0\. Start: Cart\n1\. Pay\. Now: Paying\n2\. Charge\. Now: Paying\n3\. Charge\. Now: Paying\n\nWith the fix, step 3 \(Charge\) cannot happen, so you stay at: Paying\.\n/, + ); + }, +); + +const retry = (name, act, again) => `---- MODULE ${name} ---- +EXTENDS Naturals +VARIABLES screen, charges, key +vars == <> +Init == screen = "Cart" /\\ charges = 0 /\\ key = FALSE +Pay == screen = "Cart" /\\ screen' = "Paying" /\\ charges' = 1 /\\ UNCHANGED key +FetchKey == screen = "Paying" /\\ ~key /\\ key' = TRUE /\\ UNCHANGED <> +${act} == screen = "Paying" /\\ ${again} +Spec == Init /\\ [][Pay \\/ FetchKey \\/ ${act}]_vars +NoDoubleCharge == charges <= 1 +==== +`; + +test( + "the replay takes only the trace's own steps, matches a step by its words between the same two states, and quotes the second model's words", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Retry.tla"), + retry( + "Retry", + "Retry", + "charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED <>", + ), + ); + writeFileSync(join(dir, "Retry.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync( + join(dir, "Retry.map.mjs"), + `export const state = (v) => v.screen; +export const step = (a, before, after) => + a === "Pay" ? "you pay" + : a === "FetchKey" ? "the key arrives" + : after.charges > before.charges ? "you retry and the bank charges you again" + : "you retry and see the first charge"; +`, + ); + const withFix = (act, again) => { + writeFileSync(join(dir, "RetryFixed.tla"), retry("RetryFixed", act, again)); + const r = node( + "statechart.mjs", + ["from-spec", "Retry.tla", "--fixed", "RetryFixed.tla"], + dir, + ); + assert.equal(r.status, 1, r.stderr); + const story = readFileSync(join(dir, "Retry-statechart", "stories.md"), "utf8"); + assert.match( + story, + /\n0\. Start: Cart\n1\. You pay\. Now: Paying\n2\. You retry and the bank charges you again\. Now: Paying\n\n/, + ); + return story.split("\n").find((l) => l.startsWith("With the fix")); + }; + assert.equal( + withFix("Retry", `key /\\ screen' = "Paid" /\\ UNCHANGED <>`), + "With the fix, step 2 (you retry and the bank charges you again) cannot happen, so you stay at: Paying.", + ); + assert.equal( + withFix("Retry", `screen' = "Paid" /\\ UNCHANGED <>`), + 'With the fix, step 2 (you retry and the bank charges you again) happens as "you retry and see the first charge" and takes you to: Paid.', + ); + assert.equal( + withFix( + "Recharge", + "charges < 2 /\\ charges' = charges + 1 /\\ key' = TRUE /\\ UNCHANGED screen", + ), + "With the fix you see the same states and steps, so the fix does not change this path.", + ); + }, +); + +const upload = (name, cache, copy) => `---- MODULE ${name} ---- +EXTENDS Naturals +VARIABLES screen, cached, copies, merged +vars == <> +Init == screen = "Picking" /\\ cached = FALSE /\\ copies = 0 /\\ merged = FALSE +Pick == screen = "Picking" /\\ screen' = "Ready" /\\ UNCHANGED <> +Cache == screen = "Ready" /\\ ~cached /\\ cached' = TRUE /\\ ${cache} +Upload == screen = "Ready" /\\ cached /\\ copies < 2 /\\ copies' = copies + 1 /\\ ${copy} +Spec == Init /\\ [][Pick \\/ Cache \\/ Upload]_vars +OneCopy == copies <= 1 \\/ merged +==== +`; + +test( + "the replay says which hidden step goes elsewhere in the fix, and when the rule holds on the same states and steps", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const keepCache = "UNCHANGED <>"; + const keepCopy = "UNCHANGED <>"; + writeFileSync(join(dir, "Upload.tla"), upload("Upload", keepCache, keepCopy)); + writeFileSync(join(dir, "Upload.cfg"), "SPECIFICATION Spec\nINVARIANT OneCopy\n"); + writeFileSync(join(dir, "Upload.map.mjs"), "export const state = (v) => v.screen;\n"); + const withFix = (cache, copy) => { + writeFileSync(join(dir, "UploadFixed.tla"), upload("UploadFixed", cache, copy)); + const r = node( + "statechart.mjs", + ["from-spec", "Upload.tla", "--fixed", "UploadFixed.tla"], + dir, + ); + assert.equal(r.status, 1, r.stderr); + const story = readFileSync(join(dir, "Upload-statechart", "stories.md"), "utf8"); + assert.match( + story, + /\n0\. Start: Picking\n1\. Pick\. Now: Ready\n {3}\(1 hidden step: Cache\)\n2\. Upload\. Now: Ready\n3\. Upload\. Now: Ready\n\n/, + ); + return story.split("\n").find((l) => l.startsWith("With the fix")); + }; + assert.equal( + withFix(`screen' = "Cached" /\\ UNCHANGED <>`, keepCopy), + 'With the fix, after step 1, "Cache" happens and takes you to: Cached.', + ); + assert.equal( + withFix(keepCache, "merged' = (copies = 1) /\\ UNCHANGED <>"), + "With the fix you see the same states and steps, yet the rule holds, so the state labels do not show what the fix changes.", ); }, ); @@ -2612,7 +5688,7 @@ export const cause = () => null; assert.doesNotMatch(story, /do not show what/); assert.match( story, - /^With the badge fix, after step 3, "a message arrives" happens and takes you to: Settings, muted, badge\.$/m, + /^With the badge fix, step 4 \(a message arrives\) takes you to: Settings, muted, badge\.$/m, ); const settings = classes.find((c) => c.end === "Settings, muted"); assert.deepEqual(fixEdges(`bug-${settings.n}.mmd`), [ @@ -2739,6 +5815,16 @@ Spec == \\* not [][Inc]_x unnamed.stderr, /^Count\.tla: 1 step has the name Next, the whole next-state action/, ); + writeFileSync( + join(dir, "Count.tla"), + count("Reset == x = 3 /\\ x' = 0\nNext == (Inc \\/ Reset) /\\ x \\in 0..3"), + ); + const conjunct = node("statechart.mjs", ["from-spec", "Count.tla"], dir); + assert.equal(conjunct.status, 0, conjunct.stderr); + assert.equal( + conjunct.stderr, + "Count.tla: 4 steps have the name Next, the whole next-state action. TLC goes down from Next through each \\/, \\E, LET, and definition, and names each step after the last definition on that path. It stops at any other expression, such as /\\, IF, or CASE. If the reader must tell these steps apart, join a definition for each kind of step with \\/ in Next, as in Next == Pay \\/ Charge, and move each conjunct of Next into those definitions.\n", + ); writeFileSync(join(dir, "Count.tla"), count("Next == Inc")); const named = node("statechart.mjs", ["from-spec", "Count.tla"], dir); assert.equal(named.status, 0, named.stderr); @@ -2821,6 +5907,19 @@ const ownTmp = (dir, name) => { return tmp; }; const tlcStarted = (tmp) => readdirSync(tmp).some((f) => f.startsWith("tlc-")); +const tlcChecking = (tmp) => { + try { + return readdirSync(tmp, { recursive: true }).some((f) => + /^tlc-[^/]+\/[^/]+\/[^/]+\.fp$/.test(f), + ); + } catch { + return false; + } +}; +const dumpChecking = (tmp) => + readdirSync(tmp).some( + (f) => f.startsWith("statechart-") && existsSync(join(tmp, f, "tlc", "Pay.cfg")), + ) && tlcChecking(tmp); const staged = (dir) => readdirSync(dir).filter((f) => f.startsWith(".")); test( @@ -2860,8 +5959,7 @@ const changedMidRun = async ({ wrap = [], env = {} }) => { const out = writeSlowPay(dir, 20000); const tmp = ownTmp(dir, "tmp-a"); const first = startRun(dir, { wrap, env: { ...env, TMPDIR: tmp } }); - await until(() => tlcStarted(tmp)); - await sleep(500); + await until(() => dumpChecking(tmp)); writeSlowPay(dir, 20000, "Receipt"); const second = startRun(dir, { wrap, env: { ...env, TMPDIR: ownTmp(dir, "tmp-b") } }); const [paid, receipt] = await Promise.all([first.done, second.done]); @@ -2883,6 +5981,35 @@ test( () => changedMidRun({}), ); +test( + "a run whose inputs get new times and a new inode but the same content still charts them", + needsTlc, + async () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const out = writeSlowPay(dir, 20000); + const tmp = ownTmp(dir, "tmp"); + const run = startRun(dir, { env: { TMPDIR: tmp } }); + await until(() => tlcStarted(tmp)); + await sleep(500); + const later = new Date(Date.now() + 60_000); + for (const name of ["Pay.tla", "Pay.cfg", "Pay.map.mjs"]) { + const file = join(dir, name); + const before = statSync(file); + writeFileSync(`${file}.new`, readFileSync(file)); + renameSync(`${file}.new`, file); + utimesSync(file, later, later); + const after = statSync(file); + assert.notEqual(after.ino, before.ino); + assert.notEqual(after.ctimeMs, before.ctimeMs); + } + const r = await run.done; + assert.equal(r.status, 0, r.stderr); + assert.equal(r.stderr, ""); + assert.match(readFileSync(join(out, "chart.mmd"), "utf8"), /Paid/); + assert.deepEqual(staged(dir), []); + }, +); + const unshare = ["unshare", "--user", "--map-root-user", "--pid", "--fork", "--mount-proc"]; const hidePidNamespace = `#define _GNU_SOURCE #include @@ -2937,19 +6064,89 @@ test( ); const failRenames = `#define _GNU_SOURCE +#include #include #include +#include +#include #include #include +#include #include static size_t hits; +static int raced; +static int removeOne(const char *path, const struct stat *s, int type, struct FTW *at) { + return remove(path); +} +static int hasFiles(const char *dir) { + DIR *d = opendir(dir); + if (!d) return 0; + struct dirent *e; + int found = 0; + while (!found && (e = readdir(d))) found = strcmp(e->d_name, ".") && strcmp(e->d_name, ".."); + closedir(d); + return found; +} +static int named(const char *path, const char *tail) { + size_t n = strlen(path), t = strlen(tail); + return n >= t && strcmp(path + n - t, tail) == 0 && (n == t || path[n - t - 1] == '/'); +} +static void makeAnotherRunsFolder(const char *shared) { + static int (*make)(const char *, mode_t); + if (!make) make = dlsym(RTLD_NEXT, "mkdir"); + char chart[4096]; + make(shared, 0777); + snprintf(chart, sizeof chart, "%s/chart.mmd", shared); + FILE *f = fopen(chart, "w"); + fputs("another run\\n", f); + fclose(f); +} +static int chosen(const char *race) { + const char *run = getenv("ANOTHER_RUN"); + return run && !raced && strcmp(run, race) == 0; +} +static int racing(const char *race) { + return chosen(race) && (raced = 1); +} +static void takeOver(const char *shared) { + rmdir(shared); + makeAnotherRunsFolder(shared); +} +int mkdir(const char *path, mode_t mode) { + static int (*real)(const char *, mode_t); + static int (*move)(const char *, const char *); + if (!real) real = dlsym(RTLD_NEXT, "mkdir"); + if (!move) move = dlsym(RTLD_NEXT, "rename"); + if (named(path, "shared") && racing("before-copy")) makeAnotherRunsFolder(path); + if (named(path, "shared") && chosen("when-made")) { + int made = real(path, mode); + if (made == 0 && racing("when-made")) takeOver(path); + return made; + } + if (named(path, "shared/tlc") && racing("during-copy")) { + char shared[4096], aside[4200]; + snprintf(shared, sizeof shared, "%.*s", (int)(strlen(path) - strlen("/tlc")), path); + snprintf(aside, sizeof aside, "%s-aside", shared); + move(shared, aside); + makeAnotherRunsFolder(shared); + real(path, mode); + } + if (named(path, "shared/tlc") && racing("removed-during-copy")) { + char shared[4096]; + snprintf(shared, sizeof shared, "%.*s", (int)(strlen(path) - strlen("/tlc")), path); + nftw(shared, removeOne, 16, FTW_DEPTH | FTW_PHYS); + real(shared, 0777); + } + return real(path, mode); +} int rename(const char *from, const char *to) { static int (*real)(const char *, const char *); if (!real) real = dlsym(RTLD_NEXT, "rename"); const char *base = strrchr(to, '/'), *fails = getenv("RENAME_FAILS"); const char *cannot = getenv("RENAME_CANNOT"); base = base ? base + 1 : to; - if (cannot && strstr(base, "shared")) { + if (named(to, "shared") && !hasFiles(from) && racing("before-copy")) makeAnotherRunsFolder(to); + if (cannot && strstr(base, "shared") && hasFiles(from)) { errno = strcmp(cannot, "EXDEV") == 0 ? EXDEV : access(to, F_OK) == 0 ? EISDIR : ENOSYS; return -1; } @@ -2957,7 +6154,9 @@ int rename(const char *from, const char *to) { errno = fails[hits++] == 'N' ? ENOTEMPTY : EIO; return -1; } - return real(from, to); + int renamed = real(from, to); + if (renamed == 0 && named(to, "shared") && racing("when-made")) takeOver(to); + return renamed; } `; const failingRenames = (() => { @@ -2986,63 +6185,141 @@ test( const [, kept] = back.stderr.match( /^cannot replace shared: EIO: .*\. The run left shared as it was\. The files that this run wrote are in (\/\S+)\.\n$/, ); - assert.equal(readFileSync(join(kept, "chart.mmd"), "utf8"), chart); - assert.equal(readFileSync(join(shared, "chart.mmd"), "utf8"), chart); + assert.equal(readFileSync(join(kept, "chart.mmd"), "utf8"), chart); + assert.equal(readFileSync(join(shared, "chart.mmd"), "utf8"), chart); + assert.deepEqual(staged(dir), []); + + const stuck = run("NII"); + assert.equal(stuck.status, 2, stuck.stderr); + const named = stuck.stderr.match( + /^cannot replace shared: EIO: .*\. This run moved the previous shared to (\.shared-\w+-old) and could not put it back\. The files that this run wrote are in \/\S+\.\n$/, + ); + assert.ok(named, stuck.stderr); + const aside = named[1]; + assert.equal(existsSync(shared), false); + assert.equal(readFileSync(join(dir, aside, "chart.mmd"), "utf8"), chart); + assert.deepEqual(staged(dir), [aside]); + }, +); + +test( + "a run on a filesystem that cannot rename copies its files into place", + { skip: needsTlc.skip || (!failingRenames && "cannot compile a rename preload") }, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeLamp(dir); + const args = ["from-spec", "Lamp.tla", "--out", "shared"]; + for (const [cannot, label] of [ + ["ENOSYS", "you flip the switch"], + ["EXDEV", "you press the switch"], + ["ENOSYS", "you tap the switch"], + ]) { + writeFileSync( + join(dir, "Lamp.map.mjs"), + `export const state = (v) => (v.on ? "On" : "Off");\nexport const step = () => "${label}";\n`, + ); + const r = node("statechart.mjs", args, dir, { + LD_PRELOAD: failingRenames, + RENAME_CANNOT: cannot, + }); + assert.equal(r.status, 0, r.stderr); + assert.match(readFileSync(join(dir, "shared", "chart.mmd"), "utf8"), new RegExp(label)); + assert.deepEqual(staged(dir), []); + } + }, +); + +test( + "a run that cannot rename replaces a folder that another run makes before its copy, and keeps one that another run makes during its copy", + { skip: needsTlc.skip || (!failingRenames && "cannot compile a rename preload") }, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeLamp(dir); + const shared = join(dir, "shared", "chart.mmd"); + const run = (anotherRun) => + node("statechart.mjs", ["from-spec", "Lamp.tla", "--out", "shared"], dir, { + LD_PRELOAD: failingRenames, + RENAME_CANNOT: "ENOSYS", + ANOTHER_RUN: anotherRun, + }); + + const before = run("before-copy"); + assert.equal(before.status, 0, before.stderr); + assert.match(readFileSync(shared, "utf8"), /you flip the switch/); + assert.deepEqual(staged(dir), []); + + const during = run("during-copy"); + assert.equal(during.status, 2, during.stderr); + const named = during.stderr.match( + /^cannot replace shared: \S*EEXIST\S*: .*\. This run moved the previous shared to (\.shared-\w+-old) and could not put it back\. The files that this run wrote are in \/\S+\.\n$/, + ); + assert.ok(named, during.stderr); + assert.equal(readFileSync(shared, "utf8"), "another run\n"); + assert.deepEqual(readdirSync(join(dir, "shared", "tlc")), []); + assert.match(readFileSync(join(dir, named[1], "chart.mmd"), "utf8"), /you flip the switch/); + assert.deepEqual(staged(dir), [named[1]]); + }, +); + +test( + "a run that cannot rename stops when another run removes its folder during its copy and makes an empty one in its place, and keeps that folder", + { skip: needsTlc.skip || (!failingRenames && "cannot compile a rename preload") }, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeLamp(dir); + const r = node("statechart.mjs", ["from-spec", "Lamp.tla", "--out", "shared"], dir, { + LD_PRELOAD: failingRenames, + RENAME_CANNOT: "ENOSYS", + ANOTHER_RUN: "removed-during-copy", + }); + assert.equal(r.status, 2, r.stderr); + assert.match( + r.stderr, + /^cannot replace shared: another run replaced the folder during the copy\. The run left shared as it was\. The files that this run wrote are in \/\S+\.\n$/, + ); + assert.ok(existsSync(join(dir, "shared"))); assert.deepEqual(staged(dir), []); - - const stuck = run("NII"); - assert.equal(stuck.status, 2, stuck.stderr); - const named = stuck.stderr.match( - /^cannot replace shared: EIO: .*\. This run moved the previous shared to (\.shared-\w+-old) and could not put it back\. The files that this run wrote are in \/\S+\.\n$/, - ); - assert.ok(named, stuck.stderr); - const aside = named[1]; - assert.equal(existsSync(shared), false); - assert.equal(readFileSync(join(dir, aside, "chart.mmd"), "utf8"), chart); - assert.deepEqual(staged(dir), [aside]); }, ); test( - "a run on a filesystem that cannot rename copies its files into place", + "a run that cannot rename keeps the folder that another run puts in place of its folder as soon as the run makes it", { skip: needsTlc.skip || (!failingRenames && "cannot compile a rename preload") }, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeLamp(dir); - const args = ["from-spec", "Lamp.tla", "--out", "shared"]; - for (const [cannot, label] of [ - ["ENOSYS", "you flip the switch"], - ["EXDEV", "you press the switch"], - ["ENOSYS", "you tap the switch"], - ]) { - writeFileSync( - join(dir, "Lamp.map.mjs"), - `export const state = (v) => (v.on ? "On" : "Off");\nexport const step = () => "${label}";\n`, - ); - const r = node("statechart.mjs", args, dir, { - LD_PRELOAD: failingRenames, - RENAME_CANNOT: cannot, - }); - assert.equal(r.status, 0, r.stderr); - assert.match(readFileSync(join(dir, "shared", "chart.mmd"), "utf8"), new RegExp(label)); - assert.deepEqual(staged(dir), []); - } + const r = node("statechart.mjs", ["from-spec", "Lamp.tla", "--out", "shared"], dir, { + LD_PRELOAD: failingRenames, + RENAME_CANNOT: "ENOSYS", + ANOTHER_RUN: "when-made", + }); + assert.equal(r.status, 2, r.stderr); + assert.match( + r.stderr, + /^cannot replace shared: \S*EEXIST\S*: .*\. The run left shared as it was\. The files that this run wrote are in \/\S+\.\n$/, + ); + assert.equal(readFileSync(join(dir, "shared", "chart.mmd"), "utf8"), "another run\n"); + assert.deepEqual(staged(dir), []); }, ); test( "a run whose input changes before --png renders stops before it renders, and keeps its files", - needsTlc, + { skip: needsTlc.skip || needsPng.skip }, async () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); writeSlowPay(dir, 20000); - const bin = ownTmp(dir, "bin"); const rendered = join(dir, "rendered"); - writeFileSync(join(bin, "npx"), `#!/bin/sh\ntouch "${rendered}"\nexit 1\n`, { mode: 0o755 }); const tmp = ownTmp(dir, "tmp"); const run = startRun(dir, { args: ["from-spec", "Pay.tla", "--png"], - env: { TMPDIR: tmp, PATH: `${bin}${delimiter}${process.env.PATH}` }, + env: { + TMPDIR: tmp, + XDG_CACHE_HOME: fakeRenderer(), + RENDERED: rendered, + TLA2TOOLS: tools.jar, + JAVA: tools.java, + }, }); await until(() => tlcStarted(tmp)); const map = join(dir, "Pay.map.mjs"); @@ -3057,8 +6334,98 @@ test( }, ); +const writeCount = (dir) => { + writeFileSync( + join(dir, "Count.tla"), + "---- MODULE Count ----\nEXTENDS Naturals, \\* the bound\n Limits\nVARIABLE n\nInit == n = 0\nUp == n < Max /\\ n' = n + 1\nNext == Up\nSpec == Init /\\ [][Next]_n\nSmall == n <= 2\nLast == n <= 2\n====\n", + ); + writeFileSync(join(dir, "Limits.tla"), "---- MODULE Limits ----\nMax == 2\n====\n"); + writeFileSync(join(dir, "Count.cfg"), "SPECIFICATION Spec\nINVARIANT Small\nINVARIANT Last\n"); + writeFileSync(join(dir, "labels.mjs"), 'export const up = "you add one";\n'); + writeFileSync( + join(dir, "Count.map.mjs"), + 'import { up } from "./labels.mjs";\nexport const state = (v) => `count ${v.n}`;\nexport const step = () => up;\n', + ); + return join(dir, "Count-statechart"); +}; +const EDIT_DURING_TLC = `#!/bin/sh +case "$*" in +*Count-Small.cfg*) cp "$EDITED" "$TARGET";; +*Count-Last.cfg*) [ -z "$RESTORE" ] || cp "$RESTORE" "$TARGET";; +esac +exec "$REAL_JAVA" "$@" +`; +const editDuringTlc = (file, edit, putBack) => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + const out = writeCount(dir); + const side = ownTmp(dir, "side"); + const target = join(dir, file); + writeFileSync(join(side, "edited"), edit(readFileSync(target, "utf8"))); + writeFileSync(join(side, "original"), readFileSync(target)); + writeFileSync(join(side, "java"), EDIT_DURING_TLC, { mode: 0o755 }); + const r = node("statechart.mjs", ["from-spec", "Count.tla"], dir, { + JAVA: join(side, "java"), + TLA2TOOLS: tools.jar, + REAL_JAVA: tools.java, + TARGET: target, + EDITED: join(side, "edited"), + RESTORE: putBack ? join(side, "original") : "", + }); + return { r, out }; +}; + +test( + "a spec or a module it EXTENDS that changes during a run and changes back gets the results of the files that the run started with", + needsTlc, + () => { + for (const [file, edit] of [ + ["Count.tla", (s) => s.replace("n < Max", "n < 5")], + ["Limits.tla", (s) => s.replace("Max == 2", "Max == 5")], + ]) { + const { r, out } = editDuringTlc(file, edit, true); + assert.equal(r.status, 0, `${file}: ${r.stdout}${r.stderr}`); + assert.match(r.stdout, /^ {2}Small holds\.\n {2}Last holds\.$/m, file); + assert.match(r.stdout, /^No state breaks an INVARIANT rule\.$/m, file); + assert.ok(existsSync(join(out, "chart.mmd")), file); + } + }, +); + +test( + "a module that the spec EXTENDS or a file that the map imports, changed during a run, stops the run with exit 2", + needsTlc, + () => { + for (const [file, edit] of [ + ["Limits.tla", (s) => s.replace("Max == 2", "Max == 5")], + ["labels.mjs", (s) => s.replace("one", "two")], + ]) { + const { r, out } = editDuringTlc(file, edit, false); + assert.equal(r.status, 2, `${file}: ${r.stdout}${r.stderr}`); + assert.match( + r.stderr, + new RegExp( + `^${file.replace(".", "\\.")} changed while this run read it, so the run left Count-statechart as it was\\. Run the command again\\. The files that this run wrote are in \\S+\\.\\n$`, + ), + ); + assert.equal(existsSync(out), false, file); + } + }, +); + +test("a copy keeps the folders of its input under the snapshot folder, on a Windows drive or share too", () => { + assert.equal(copyPath("/tmp/r", "/home/u/Count.tla", posix), "/tmp/r/home/u/Count.tla"); + assert.equal( + copyPath("C:\\Temp\\r", "C:\\Users\\u\\Count.tla", win32), + "C:\\Temp\\r\\C\\Users\\u\\Count.tla", + ); + assert.equal( + copyPath("C:\\Temp\\r", "\\\\host\\share\\Count.tla", win32), + "C:\\Temp\\r\\hostshare\\Count.tla", + ); +}); + test( - "a rule that TLC finds broken before TLC throws a Java exception is broken, and the run draws the paths TLC printed", + "a rule that TLC finds broken before TLC stops on an error is broken, and the run draws the paths TLC printed", needsTlc, () => { const dir = mkdtempSync(join(tmpdir(), "statechart-")); @@ -3103,7 +6470,7 @@ exit $s assert.match(r.stdout, /^ {2}NoDoubleCharge is broken \(expected\)\.$/m); assert.match( r.stderr, - /^Pay-statechart\/tlc\/Pay-NoDoubleCharge\.out: TLC found that NoDoubleCharge is broken, and then failed with java\.lang\.ArrayIndexOutOfBoundsException: Index 123 out of bounds for length 4\. .* Do not weaken NoDoubleCharge because of it\./m, + /^Pay-statechart\/tlc\/Pay-NoDoubleCharge\.out: TLC found that NoDoubleCharge is broken, and then stopped on an error: java\.lang\.ArrayIndexOutOfBoundsException: Index 123 out of bounds for length 4\. The paths that TLC printed before it stopped are complete, but TLC did not check every state, so the charts can miss states that break NoDoubleCharge\. A smaller constant can let TLC finish\.$/m, ); assert.match(r.stdout, /^## 1\. Paying, charged twice$/m); }, @@ -3174,6 +6541,253 @@ test( }, ); +test( + "a state graph too large for one string prints the rule results, keeps TLC's output, and draws no chart", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const java = join(dir, "java"); + writeFileSync( + java, + `#!/bin/sh +"$REAL_JAVA" "$@"; s=$? +while [ $# -gt 0 ]; do + [ "$1" = -dump ] && "$NODE" -e 'require("fs").truncateSync(process.argv[1], +process.argv[2])' "$3" "$SIZE" + shift +done +exit $s +`, + { mode: 0o755 }, + ); + const tmp = ownTmp(dir, "tmp"); + const r = node( + "statechart.mjs", + ["from-spec", "Pay.tla", "--expect-broken", "NoDoubleCharge"], + dir, + { + JAVA: java, + TLA2TOOLS: tools.jar, + REAL_JAVA: tools.java, + NODE: process.execPath, + SIZE: String(constants.MAX_STRING_LENGTH + 1), + TMPDIR: tmp, + }, + ); + assert.equal(r.status, 2, r.stderr); + assert.match(r.stdout, /^ {2}NoDoubleCharge is broken \(expected\)\.$/m); + const [, kept] = r.stderr.match( + /^Pay\.tla: TLC's state graph has [\d,]+ bytes, more than the [\d,]+ that the command can read, so it draws no chart\. The rule results above stand\. Smaller constants make a smaller graph\. The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/, + ); + assert.match( + readFileSync(join(dir, kept, "tlc", "Pay-NoDoubleCharge.out"), "utf8"), + /^Error: Invariant NoDoubleCharge is violated/m, + ); + assert.equal(existsSync(join(dir, "Pay-statechart")), false); + rmSync(dir, { recursive: true, force: true }); + }, +); + +test( + "a graph dump that grows past one string is stopped, and the rule results stay", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const pid = join(dir, "dump.pid"); + const java = join(dir, "java"); + writeFileSync( + java, + `#!/bin/sh +for a; do [ "$a" = -dump ] && dumping=1; done +[ -z "$dumping" ] && exec "$REAL_JAVA" "$@" +while [ "$1" != -dump ]; do shift; done +printf 'TLC2 Version 2.19\\n' +"$NODE" -e 'require("fs").writeFileSync(process.argv[1], ""); require("fs").truncateSync(process.argv[1], +process.argv[2])' "$3" "$SIZE" +echo $$ > "$PID" +exec sleep 20 +`, + { mode: 0o755 }, + ); + const tmp = ownTmp(dir, "tmp"); + const r = node( + "statechart.mjs", + ["from-spec", "Pay.tla", "--expect-broken", "NoDoubleCharge"], + dir, + { + JAVA: java, + TLA2TOOLS: tools.jar, + REAL_JAVA: tools.java, + NODE: process.execPath, + SIZE: String(constants.MAX_STRING_LENGTH + 1), + PID: pid, + TMPDIR: tmp, + }, + ); + assert.equal(r.status, 2, r.stderr); + assert.match(r.stdout, /^ {2}NoDoubleCharge is broken \(expected\)\.$/m); + const [, kept] = r.stderr.match( + /^Pay\.tla: TLC's state graph passed [\d,]+ bytes, the most that the command can read, so the command stopped TLC and draws no chart\. The rule results above stand\. Smaller constants make a smaller graph\. The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/, + ); + assert.throws(() => process.kill(Number(readFileSync(pid, "utf8")), 0), { code: "ESRCH" }); + assert.match( + readFileSync(join(dir, kept, "tlc", "Pay-NoDoubleCharge.out"), "utf8"), + /^Error: Invariant NoDoubleCharge is violated/m, + ); + assert.equal(existsSync(join(dir, "Pay-statechart")), false); + rmSync(dir, { recursive: true, force: true }); + }, +); + +test("TLC output that grows past one string stops TLC, and the command exits 2", needsTlc, () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay("Pay", `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const pid = join(dir, "tlc.pid"); + const java = join(dir, "java"); + writeFileSync( + java, + `#!/bin/sh +echo $$ > "$PID" +exec "$NODE" -e ' +const size = +process.argv[1]; +const chunk = "x".repeat(1 << 16); +let sent = 0; +process.stdout.write("TLC2 Version 2.19\\n"); +(function more() { + while (sent <= size) { + sent += chunk.length; + if (!process.stdout.write(chunk)) return process.stdout.once("drain", more); + } + setTimeout(() => {}, 20000); +})(); +' "$SIZE" +`, + { mode: 0o755 }, + ); + const tmp = ownTmp(dir, "tmp"); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, { + JAVA: java, + TLA2TOOLS: tools.jar, + NODE: process.execPath, + SIZE: String(constants.MAX_STRING_LENGTH), + PID: pid, + TMPDIR: tmp, + }); + assert.equal(r.status, 2, r.stderr); + const [, out] = r.stderr.match( + /^(\/\S+\/tlc\/Pay-NoDoubleCharge\.out): TLC printed more than [\d,]+ characters, the most that the command can read, so the command stopped TLC\. Smaller constants make TLC print less\.\n$/, + ); + assert.throws(() => process.kill(Number(readFileSync(pid, "utf8")), 0), { code: "ESRCH" }); + assert.equal(existsSync(out), true); + assert.equal(existsSync(join(dir, "Pay-statechart")), false); + rmSync(dir, { recursive: true, force: true }); +}); + +test( + "the rule results print before the graph dump, and stay when TLC stops on an error in the dump", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync(join(dir, "Pay.map.mjs"), payMap); + const java = join(dir, "java"); + writeFileSync( + java, + `#!/bin/sh +for a; do + if [ "$a" = -dump ]; then + printf 'TLC2 Version 2.19\\nError: TLC threw an unexpected exception.\\n' + exit 1 + fi +done +exec "$REAL_JAVA" "$@" +`, + { mode: 0o755 }, + ); + const tmp = ownTmp(dir, "tmp"); + const r = node( + "statechart.mjs", + ["from-spec", "Pay.tla", "--expect-broken", "NoDoubleCharge"], + dir, + { JAVA: java, TLA2TOOLS: tools.jar, REAL_JAVA: tools.java, TMPDIR: tmp }, + ); + assert.equal(r.status, 2, r.stderr); + assert.match(r.stdout, /^ {2}NoDoubleCharge is broken \(expected\)\.$/m); + const [, kept] = r.stderr.match( + /^(\/\S+)\/tlc\/Pay\.out: TLC stopped on an error that is not a broken rule: TLC threw an unexpected exception\.$/m, + ); + assert.match( + readFileSync(join(kept, "tlc", "Pay-NoDoubleCharge.out"), "utf8"), + /^Error: Invariant NoDoubleCharge is violated/m, + ); + rmSync(dir, { recursive: true, force: true }); + }, +); + +test( + "an error after TLC runs keeps TLC's output, and the message names its folder", + needsTlc, + () => { + const dir = mkdtempSync(join(tmpdir(), "statechart-")); + writeFileSync( + join(dir, "Pay.tla"), + pay( + "Pay", + `screen = "Paying" /\\ charges < 2 /\\ charges' = charges + 1 /\\ UNCHANGED screen`, + ), + ); + writeFileSync(join(dir, "Pay.cfg"), "SPECIFICATION Spec\nINVARIANT NoDoubleCharge\n"); + writeFileSync( + join(dir, "Pay.map.mjs"), + payMap.replace( + /^export const state = .*$/m, + `export const state = () => { throw new Error("the map cannot read this state"); };`, + ), + ); + const r = node("statechart.mjs", ["from-spec", "Pay.tla"], dir, { + TMPDIR: ownTmp(dir, "tmp"), + }); + assert.equal(r.status, 2, r.stderr); + const [, kept] = r.stderr.match( + / The files that this run wrote are in (tmp\/statechart-\w+)\.\n$/, + ); + assert.match( + readFileSync(join(dir, kept, "tlc", "Pay-NoDoubleCharge.out"), "utf8"), + /^Error: Invariant NoDoubleCharge is violated/m, + ); + assert.equal(existsSync(join(dir, "Pay-statechart")), false); + rmSync(dir, { recursive: true, force: true }); + }, +); + test( "runs of two specs that overlap on one output folder all finish, and the folder always holds one run's files", needsTlc, @@ -3228,64 +6842,13 @@ test( }, ); -test( - "the output folder model keeps every rule, and breaks one with a shared TLC folder, without each guard, or when a failed rename deletes the folder moved aside", - needsTlc, - () => { - const dir = mkdtempSync(join(tmpdir(), "statechart-")); - const model = (ext) => readFileSync(join(scripts, "model", `OutputFolder${ext}`), "utf8"); - const r = node( - "statechart.mjs", - ["from-spec", join(scripts, "model", "OutputFolder.tla"), "--out", join(dir, "out")], - dir, - ); - assert.equal(r.status, 0, r.stdout + r.stderr); - assert.match(r.stdout, /\nNo state breaks a rule\.\n/); - for (const [i, [rule, guard, without, story]] of [ - [ - "ChartsOneSpec", - "/\\ read' = Set(read, r, read[r] \\cup {spec})", - "/\\ read' = [q \\in Runs |-> read[q] \\cup {spec}]", - /You start run 2 in a second shell\..*\n.*Run 2 moves its folder into place\. Now: Two runs going: chart of both specs/, - ], - [ - "ChartsOneSpec", - '/\\ pc[r] = "publish" /\\ spec = snap[r] /\\ ~out.notes', - '/\\ pc[r] = "publish" /\\ ~out.notes', - /You edit the spec\..*\n.*Run 1 moves its folder into place\. Now: One run going: chart of both specs/, - ], - [ - "KeepsYourFiles", - '/\\ pc[r] = "aside" /\\ ~aside[r].notes\n', - '/\\ pc[r] = "aside"\n', - /Run 1 deletes the old folder and finishes\. Now: Done: chart of the spec\n/, - ], - [ - "NeverLosesChart", - " /\\ \\/ /\\ aside[r].exists /\\ ~out.exists\n /\\ out' = aside[r]\n /\\ aside' = Set(aside, r, NoFolder)\n \\/ UNCHANGED <>\n", - " /\\ out' = out\n /\\ aside' = Set(aside, r, IF aside[r].notes THEN aside[r] ELSE NoFolder)\n", - /Run 2 moves the old folder aside\. Now: .*, the old chart moved aside\n.*Run 2 cannot rename a folder and stops\. Now: One run going: no folder\n/, - ], - [ - "NeverLosesChart", - '/\\ pc[r] = "swap" /\\ out.exists /\\ aside[r].exists /\\ ~out.notes', - '/\\ pc[r] = "swap" /\\ out.exists /\\ aside[r].exists', - /You save notes in the folder\..*\n.*Run 2 finds another run's folder in place and deletes the old folder\. Now: Two runs going: your notes\n/, - ], - ].entries()) { - const mutant = join(dir, `${i}-${rule}`); - mkdirSync(mutant); - assert.ok(model(".tla").includes(guard), guard); - writeFileSync(join(mutant, "OutputFolder.tla"), model(".tla").replace(guard, without)); - for (const ext of [".cfg", ".map.mjs"]) - writeFileSync(join(mutant, `OutputFolder${ext}`), model(ext)); - const m = node( - "statechart.mjs", - ["from-spec", "OutputFolder.tla", "--expect-broken", rule], - mutant, - ); - assert.equal(m.status, 0, m.stdout + m.stderr); - assert.match(m.stdout, story); - } - }, -); +test("no file in scripts/ passes 1000 lines", () => { + const long = readdirSync(scripts, { withFileTypes: true }) + .filter((e) => e.isFile()) + .map((e) => [ + e.name, + readFileSync(join(scripts, e.name), "utf8").replace(/\n$/, "").split("\n").length, + ]) + .filter(([, lines]) => lines > 1000); + assert.deepEqual(long, [], "move cohesive parts of each listed file into their own modules"); +}); diff --git a/pstack/skills/statechart/scripts/tlc-to-chart-args.mjs b/pstack/skills/statechart/scripts/tlc-to-chart-args.mjs new file mode 100644 index 000000000..a2e0a3124 --- /dev/null +++ b/pstack/skills/statechart/scripts/tlc-to-chart-args.mjs @@ -0,0 +1,52 @@ +import process from "node:process"; + +export const args = { + traces: [], + fixedTraces: [], + againstTraces: [], + fixedRules: [], + againstRules: [], + specs: [], + next: [], + view: "customer", + highlight: 1, +}; +for (let i = 2; i < process.argv.length; i++) { + const a = process.argv[i]; + const next = () => process.argv[++i]; + if (a === "--dot") args.dot = next(); + else if (a === "--map") args.map = next(); + else if (a === "--traces") args.traces.push(next()); + else if (a === "--broken") args.broken = next(); + else if (a === "--fixed-dot") args.fixedDot = next(); + else if (a === "--fixed-traces") args.fixedTraces.push(next()); + else if (a === "--fixed-rules") args.fixedRules.push(...next().split(",")); + else if (a === "--fixed-broken") args.fixedBroken = next(); + else if (a === "--against-dot") args.againstDot = next(); + else if (a === "--against-traces") args.againstTraces.push(next()); + else if (a === "--against-broken") args.againstBroken = next(); + else if (a === "--against-rules") args.againstRules.push(...next().split(",")); + else if (a === "--listed") args.listed = next(); + else if (a === "--label") args.label = next(); + else if (a === "--spec") args.specs.push(next()); + else if (a === "--next") args.next.push(next()); + else if (a === "--view") args.view = next(); + else if (a === "--highlight") args.highlight = Number(next()); + else if (a === "--path-only") args.pathOnly = true; + else if (a === "--full-labels") args.fullLabels = true; + else if (a === "--flat") args.flat = true; + else if (a === "--loops") args.loops = true; + else if (a === "--regions") args.regions = true; + else if (a === "--proof") args.proof = next(); + else if (a === "--regions-all") args.regionsAll = next(); + else if (a === "--cover") args.cover = true; + else if (a === "--title") args.title = next(); + else if (a === "--out") args.out = next(); + else if (a === "--story") args.story = next(); + else if (a === "--summary") args.summary = next(); + else { + console.error(`unknown argument ${a}.`); + process.exit(2); + } +} +if (args.listed !== undefined) args.listed = JSON.parse(args.listed); diff --git a/pstack/skills/statechart/scripts/tlc-to-chart.mjs b/pstack/skills/statechart/scripts/tlc-to-chart.mjs index 37ae46e33..7aefe44be 100644 --- a/pstack/skills/statechart/scripts/tlc-to-chart.mjs +++ b/pstack/skills/statechart/scripts/tlc-to-chart.mjs @@ -8,7 +8,6 @@ import { GROUP_PALETTE, LEGEND_ID, PLAIN_FILL, - compareCharts, escapeMermaidQuotes, groupClassDef, groupColorProblem, @@ -16,251 +15,30 @@ import { joinAnd, joinLabels, parseDot, - parseMermaid, parseTraces, - productOf, - ruleSentences, stateKey, tlcOutputWarning, wrap, STATUS_CLASSES, } from "./lib.mjs"; - -const USAGE = "Usage: node tlc-to-chart.mjs --dot FILE --map FILE [options]"; -const HELP = `${USAGE} - -Collapses a TLC state graph onto the reader's states and writes a Mermaid stateDiagram-v2. -statechart.mjs runs it once for the overview and once for each bug chart. - - --dot FILE The state graph from TLC -dump dot,actionlabels FILE, for the current code. - Required. - --map FILE The mapping module. Required. It exports state(vars, facts), and - optionally detail(vars, facts), step(action, before, after), - guard(action, before, after), timer(action, before, after), - cause(vars, facts, invariant), group(vars, facts), groupColors, - regionNames, readerActions, and legend. guard is when the step - can happen, in plain words. An edge reads "step [words]" when every - model step behind it has words. timer is how many seconds the code waits - before a timer step, and the step reads "after N s, step". - cause is the hidden reason a bad state is bad: counterexamples with the - same invariant and cause are one bug. When cause returns nothing, the - end state stands in for it. readerActions are the actions the - reader takes. group is the group of a state, below. legend, such as - { works: "Works on main" }, gives the legend's words for the works, - wrong, and fixed statuses. - facts.enabled is the set of actions the state enables. facts.settled is - true when only reader actions are enabled, so nothing else happens until - the reader acts. - --traces FILE TLC output that holds counterexamples. Repeat it. TLC -continue gives one - per bad state. It refuses a file from a TLC run that stopped early or hit - an error other than a broken rule. When TLC found a rule broken and then - failed with a Java exception, it keeps the paths that TLC printed and - warns that the chart can miss states that break the rule. - --fixed-dot FILE The state graph of the model with the open fix. Each bug's steps replay - on it. - --fixed-traces FILE TLC output for the same invariants on the fixed model. Repeat it. It marks - states fixed or still wrong, and adds the bugs that only the fixed model - has after the current model's bugs. - --fixed-rules LIST The invariants that the fixed model's TLC runs check, as in A,B. It goes - with --fixed-traces. A rule that the fixed model does not check stays - as wrong as it is today, and its stories say "not checked". - --against-dot FILE The state graph of the spec that the current code must follow. - Each bug's steps replay on it. Do not use it with --fixed-dot. - --against-traces FILE TLC output for the spec. Repeat it. Its bugs are drawn wrong, and the - bugs that only the spec has number after the code's bugs. - --against-rules LIST The invariants that the spec's TLC runs check. It goes with - --against-traces. Stories of a rule that the spec does not check say - "not checked". - --listed JSON What the spec's configs expect, as in - {"spec":"Spec.tla","breaks":{"A.cfg":"Rule"}}: each config whose - "\\* expect:" line names a rule, and that rule. It goes with - --against-dot. Each story then says whether a config of the spec already - expects TLC to break its rule, and summary.json has second.listed and - each bug's listed configs. - --label TEXT The name of the second model in the chart and the stories. Default: - "the fix" with --fixed-dot, "the spec" with --against-dot. - --spec FILE A TLA+ file whose \\* comment lines above a rule are the rule's sentence in - the stories. Repeat it. The first file that has a sentence wins. - --view NAME customer (state labels, the default), developer (detail labels), or - layered (detail inside state). - --cover Keep only the shortest path to each settled reader state, plus the - counterexample paths. - --highlight N Draw bug N as a numbered path. Default 1. 0 draws none. - --path-only Draw only the highlighted path. - --full-labels Write each label on an edge in full, joined by " or ", and mark the chart - "${FULL_LABELS}". Default: write the leading words that the labels - share once, as in "you tap Pay or Back". - --flat Draw every state on its own, with no boxes. - --regions When every state reads "context: progress", draw two parallel regions, - context and progress, when they are smaller and proved. They draw the - steps of the shortest way to each region state, of each bug path, and - the nearest step with each label. - Otherwise draw the chart the other flags ask for, and print why. - Customer view, --highlight 0, no --flat or --path-only. - --proof FILE With --regions, write the flat chart that the proof compares with. - --regions-all FILE With --regions, also write the regions with every step. - --title TEXT The chart title. - --out FILE The Mermaid chart. Default: stdout. - --story FILE The reader stories, one per bug. - --summary FILE The counts for the chart and the bugs, as JSON. - -h, --help Print this help, and exit 0. - -Each bug replays on the second model by its screens and step words, which are what the reader -sees. The replay takes the second model's steps that keep the screen and that the reader does not -take. A bug step that the reader does not see must keep the screen on the second model too, unless -the second model cannot take it. The replay stops at the first step that the second model shows -the reader differently. There the chart draws "with LABEL, STEP" (or "in LABEL, STEP" for ---against-dot) to where the second model goes, or a loop "... STEP cannot happen" when it cannot -take a step that the reader sees. - -The second model may have other variables and actions than the current model. Variables and -record fields that one start state has and the other does not are left out when the two models -are compared. One mapping draws both models, so it may read only what both have. The command -stops when the mapping reads a variable or field of the current model on a state of the second -model that lacks it. Test "name" in v first to label such a state another way. A step of an -action that the second model does not have is skipped when it changes no shared variable, and -matches any step to the same shared state when it does. - -In the customer view, states whose labels read "context: progress" with the same context sit in -one box named context, and each shows only its progress. A context with one state stays a plain -state. An arrow from a box's border stands for the same step from every state in the box: it is -drawn only when every state in the box takes that step, with the same labels, to the same state. -The layered view puts each detail state in a box named by its state label. - -With --regions, each "context: progress" label is one state in the context region and one in the -progress region. A step that happens beside only some states of the other region gets a guard: -"[when A or B]" or "[unless A or B]", whichever is shorter, and a "%% when [not] ids" line under it. -The mapping's guard words, when the step has them, replace "when A or B" in the label. -The proof combines the regions with every step as compare-charts.mjs does, and needs exactly the -flat chart's states, start states, and labeled steps. Each state's color must be the worse color -of its two region states. Then the drawn arrows, combined, must make only steps of the flat chart: -a step that they make wrongly gets back every arrow that the full regions take there. -compare-charts.mjs on the --regions-all and --proof files repeats the first proof. -mapping.regionNames = [context, progress] names the two regions. - -Outside regions, two states have at most one arrow. Its lines are the numbered steps, then the -current model's steps, then the second model's steps, with a blank line between each group. - -Without group(), the legend shows only the statuses the chart uses. A chart with a second model -has the line "%% second model (fix|against): LABEL" right after stateDiagram-v2. Each chart -starts with front matter that keeps Mermaid from shrinking it to the page or rewrapping its labels. - -Groups: group(vars, facts) returns the name of the group that a model state is in, such as -"Toast shown", or nothing. A chart state is in a group when all its model states are in it. A -region state is in a group when every state that it is part of is in it. Each group fills its -states with one color, and the legend lists only the groups that the chart draws: first the -groups in groupColors, in that order, then the others from A to Z. groupColors, such as -{ "Toast shown": "#d6eaf8" }, gives a group a light "#rgb" or "#rrggbb" color. A group with no -color, or null, takes the next free palette color: light blue, orange, purple, yellow, teal, tan, -then pink. A state in no group, or in more than one, is gray. summary.json counts the drawn -states of each group in chartGroups, and lists the chart states with more than one in mixedGroups. -TLC decides the status, and a group cannot change it. On top of the group color, a state that -breaks a rule has a thick red border, and a fixed state or a state only in the second model a -thick green border. The legend then shows that border. A thick blue border marks the bug path on -states that work. On other states of the path, the numbered arrows show it. A group cannot have -the name of a status, such as "Breaks a rule". - -Writes only the files that --out, --story, and --summary name. - -Exit codes - 0 It wrote the chart. - 1 It cannot use an input: a mapping without state(), a mapping that reads what the second - model lacks, TLC output that it refuses, or a --highlight with no such bug. The error says - why. - 2 A bad argument.`; -const more = "Run node tlc-to-chart.mjs --help for the flags."; - -const args = { - traces: [], - fixedTraces: [], - againstTraces: [], - fixedRules: [], - againstRules: [], - specs: [], - view: "customer", - highlight: 1, -}; -for (let i = 2; i < process.argv.length; i++) { - const a = process.argv[i]; - const next = () => process.argv[++i]; - if (a === "-h" || a === "--help") { - console.log(HELP); - process.exit(0); - } else if (a === "--dot") args.dot = next(); - else if (a === "--map") args.map = next(); - else if (a === "--traces") args.traces.push(next()); - else if (a === "--fixed-dot") args.fixedDot = next(); - else if (a === "--fixed-traces") args.fixedTraces.push(next()); - else if (a === "--fixed-rules") args.fixedRules.push(...next().split(",")); - else if (a === "--against-dot") args.againstDot = next(); - else if (a === "--against-traces") args.againstTraces.push(next()); - else if (a === "--against-rules") args.againstRules.push(...next().split(",")); - else if (a === "--listed") args.listed = next(); - else if (a === "--label") args.label = next(); - else if (a === "--spec") args.specs.push(next()); - else if (a === "--view") args.view = next(); - else if (a === "--highlight") args.highlight = Number(next()); - else if (a === "--path-only") args.pathOnly = true; - else if (a === "--full-labels") args.fullLabels = true; - else if (a === "--flat") args.flat = true; - else if (a === "--regions") args.regions = true; - else if (a === "--proof") args.proof = next(); - else if (a === "--regions-all") args.regionsAll = next(); - else if (a === "--cover") args.cover = true; - else if (a === "--title") args.title = next(); - else if (a === "--out") args.out = next(); - else if (a === "--story") args.story = next(); - else if (a === "--summary") args.summary = next(); - else { - console.error(`unknown argument ${a}. ${more}`); - process.exit(2); - } -} -const fail = (message) => { - console.error(`${message} ${more}`); - process.exit(2); -}; -if (!args.dot || !args.map) fail(USAGE); -if (!["customer", "developer", "layered"].includes(args.view)) - fail(`--view must be customer, developer, or layered, not ${args.view}.`); -if (args.fixedDot && args.againstDot) fail("use --fixed-dot or --against-dot, not both."); -if (args.fixedTraces.length && !args.fixedDot) fail("--fixed-traces needs --fixed-dot."); -if (args.againstTraces.length && !args.againstDot) fail("--against-traces needs --against-dot."); -if (!args.fixedTraces.length !== !args.fixedRules.length) - fail("--fixed-traces and --fixed-rules go together. The rules are what those TLC runs check."); -if (!args.againstTraces.length !== !args.againstRules.length) - fail( - "--against-traces and --against-rules go together. The rules are what those TLC runs check.", - ); -if (args.listed !== undefined) { - if (!args.againstDot) - fail("--listed says what the spec's configs expect; it needs --against-dot."); - const shape = `--listed must be JSON such as {"spec":"Spec.tla","breaks":{"A.cfg":"Rule"}}`; - try { - args.listed = JSON.parse(args.listed); - } catch (e) { - fail(`${shape}: ${e.message}.`); - } - const { spec, breaks } = isRecord(args.listed) ? args.listed : {}; - if ( - typeof spec !== "string" || - !isRecord(breaks) || - Object.values(breaks).some((r) => typeof r !== "string") - ) - fail(`${shape}.`); -} -if (args.label !== undefined && !args.fixedDot && !args.againstDot) - fail("--label names the second model; it needs --fixed-dot or --against-dot."); -if (args.proof && !args.regions) fail("--proof writes the regions proof; it needs --regions."); -if (args.regionsAll && !args.regions) - fail("--regions-all writes every regions arrow; it needs --regions."); -if ( - args.regions && - (args.view !== "customer" || args.flat || args.pathOnly || args.highlight !== 0) -) - fail( - "--regions draws the whole customer chart: use --highlight 0, without --flat or --path-only.", - ); +import { args } from "./tlc-to-chart-args.mjs"; +import { ruleBook } from "./rule-book.mjs"; +import { SEP, addGuard, edgeId, frontMatter, guarded } from "./chart-parts.mjs"; +import { + badStates, + byOrder, + diverges, + firstBad, + labelStatus, + readBroken, + worst, +} from "./bug-states.mjs"; +import { chartEdges } from "./chart-edges.mjs"; +import { regionsChart } from "./regions-chart.mjs"; +import { mapView } from "./map-view.mjs"; +import { stepsIntoBugs, waysLine } from "./ways-in.mjs"; +import { divergence, replaySecond } from "./second-replay.mjs"; +import { readCheck } from "./read-check.mjs"; const role = args.fixedDot ? "fix" : args.againstDot ? "against" : null; const L = args.label ?? (role === "fix" ? "the fix" : "the spec"); @@ -272,13 +50,14 @@ if (typeof mapping.state !== "function") throw new Error("the mapping module must export state(vars)"); if (args.view !== "customer" && typeof mapping.detail !== "function") throw new Error(`--view ${args.view} needs detail(vars) in the mapping module`); -const SEP = "\u0000"; -const edgeId = (from, to) => `${from}${SEP}${SEP}${to}`; const rules = new Map(); -for (const f of args.specs) - for (const [name, sentence] of ruleSentences(fs.readFileSync(f, "utf8"))) - if (!rules.has(name)) rules.set(name, sentence); +const ruleReads = new Map(); +for (const spec of args.specs) + for (const [name, { sentence, reads }] of ruleBook(spec)) { + if (sentence && !rules.has(name)) rules.set(name, sentence); + ruleReads.set(name, new Set([...(ruleReads.get(name) ?? []), ...reads])); + } const readerActions = new Set(mapping.readerActions ?? []); const factsFrom = (enabled) => ({ @@ -290,11 +69,16 @@ function keyOf(vars, facts) { const outer = mapping.state(vars, facts); if (typeof outer !== "string") throw new Error(`state() must return a string; got ${JSON.stringify(outer)}`); - return args.view === "layered" ? `${outer}${SEP}${mapping.detail(vars, facts)}` : outer; + if (args.view !== "layered") return outer; + return [outer, String(mapping.detail(vars, facts))] + .map((part) => part.trim()) + .filter(Boolean) + .join(SEP); } function stepLabel(action, before, after, view = args.view) { const said = typeof mapping.step === "function" ? mapping.step(action, before, after) : action; - const named = view === "customer" || said === action ? said : `${said} [${action}]`; + const named = + view === "customer" || said === action ? said : said ? `${said} [${action}]` : action; const seconds = typeof mapping.timer === "function" ? mapping.timer(action, before, after) : null; if (seconds === undefined || seconds === null || seconds === "" || !named) return named; if (typeof seconds !== "number" || !Number.isFinite(seconds) || seconds <= 0) @@ -309,64 +93,40 @@ function stepGuard(action, before, after) { .replace(/[[\]]/g, "") .trim(); } -function guarded(guards, label) { - const words = guards?.get(label); - return words && !words.has("") ? `${label} [${[...words].sort().join(" or ")}]` : label; -} -const addGuard = (guards, label, words) => { - if (!guards.has(label)) guards.set(label, new Set()); - guards.get(label).add(words); -}; -function lacking(a, b, prefix = []) { - return Object.keys(a).flatMap((k) => - !(k in b) - ? [[...prefix, k]] - : isRecord(a[k]) && isRecord(b[k]) - ? lacking(a[k], b[k], [...prefix, k]) - : [], +const definedLabel = (label, what) => { + if (typeof label !== "string" || !/\bundefined\b/.test(label)) return label; + const does = what === "group" ? `puts a state of ${L} in group` : `labels a ${what} of ${L}`; + throw new Error( + `the mapping ${does} "${label.replaceAll(SEP, " ")}", so it reads a field ${L} does not have there. Label states and steps only from what both models have.`, ); -} -const holder = (o, p) => p.slice(0, -1).reduce((x, k) => (isRecord(x?.[k]) ? x[k] : null), o); -function omit(vars, paths) { - if (!paths.length) return vars; - const out = structuredClone(vars); - for (const p of paths) { - const o = holder(out, p); - if (o) delete o[p.at(-1)]; - } - return out; -} -function guard(vars, paths, at = "v") { - if (!paths.length || !isRecord(vars)) return vars; - const missing = new Set(); - const deeper = new Map(); - for (const [k, ...rest] of paths) - if (!rest.length) missing.add(k); - else deeper.set(k, [...(deeper.get(k) ?? []), rest]); - return new Proxy(vars, { - get(target, k) { - if (missing.has(k) && !(k in target)) { - const name = `${at}.${k}`.slice(2); - throw new Error( - `${L} has no ${name}, and the mapping reads ${at}.${k} on its states. Label states and steps only from what both models have, or test "${k}" in ${at} first.`, - ); - } - return deeper.has(k) ? guard(target[k], deeper.get(k), `${at}.${k}`) : target[k]; - }, - }); -} +}; -function modelOf(graph) { +function modelOf(graph, guarded = (vars) => vars, checked = (label) => label) { const byId = new Map([...graph.nodes.keys()].map((id) => [id, new Set()])); for (const e of [...graph.edges, ...graph.loops]) byId.get(e.from).add(e.action); - const index = new Map([...graph.nodes.values()].map((n) => [stateKey(n.vars), byId.get(n.id)])); + const stateKeys = new Map([...graph.nodes.values()].map((n) => [n.id, stateKey(n.vars)])); + const index = new Map([...stateKeys].map(([id, k]) => [k, byId.get(id)])); const facts = (vars) => factsFrom(index.get(stateKey(vars)) ?? new Set()); + const forMap = (vars) => guarded(mapView(vars)); return { graph, + stateKeys, facts, - key: (vars) => keyOf(vars, facts(vars)), - customer: (vars) => String(mapping.state(vars, facts(vars))), + key: (vars) => checked(keyOf(forMap(vars), facts(vars)), "state"), + customer: (vars) => checked(String(mapping.state(forMap(vars), facts(vars))), "state"), + step: (action, before, after, view) => + checked(stepLabel(action, forMap(before), forMap(after), view), "step"), + guard: (action, before, after) => + checked(stepGuard(action, forMap(before), forMap(after)), "step"), + cause: (vars, invariant) => { + const why = + typeof mapping.cause === "function" + ? mapping.cause(forMap(vars), facts(vars), invariant) + : null; + return why === undefined || why === null ? null : checked(String(why), "bug"); + }, + group: (vars) => checked(mapping.group(forMap(vars), facts(vars)), "group"), }; } @@ -379,17 +139,17 @@ function quotient(m) { [...graph.nodes.values()].filter((n) => n.init).map((n) => keys.get(n.id)), ); const edges = new Map(); - for (const e of graph.edges) { + for (const e of [...graph.edges, ...graph.loops]) { const from = keys.get(e.from); const to = keys.get(e.to); - if (from === to) continue; + if (from === to && !args.loops) continue; const id = edgeId(from, to); if (!edges.has(id)) edges.set(id, { from, to, counts: new Map(), guards: new Map() }); const [before, after] = [graph.nodes.get(e.from).vars, graph.nodes.get(e.to).vars]; - const label = stepLabel(e.action, before, after); + const label = m.step(e.action, before, after); if (!label) continue; edges.get(id).counts.set(label, (edges.get(id).counts.get(label) ?? 0) + 1); - addGuard(edges.get(id).guards, label, stepGuard(e.action, before, after)); + addGuard(edges.get(id).guards, label, m.guard(e.action, before, after)); } for (const e of edges.values()) e.labels = new Set([...e.counts].sort((a, b) => b[1] - a[1]).map(([l]) => l)); @@ -402,192 +162,73 @@ function quotient(m) { const graph = parseDot(fs.readFileSync(args.dot, "utf8")); const main = modelOf(graph); const now = quotient(main); -const mainInit = [...graph.nodes.values()].find((n) => n.init).vars; const secondDot = args.fixedDot ?? args.againstDot; const secondGraph = secondDot ? parseDot(fs.readFileSync(secondDot, "utf8")) : null; -const secondInit = secondGraph && [...secondGraph.nodes.values()].find((n) => n.init).vars; -const lacks = secondGraph ? lacking(mainInit, secondInit) : []; -const extra = secondGraph ? lacking(secondInit, mainInit) : []; -const second = - secondGraph && - modelOf({ - ...secondGraph, - nodes: new Map( - [...secondGraph.nodes].map(([id, n]) => [ - id, - { ...n, raw: n.vars, vars: guard(n.vars, lacks) }, - ]), - ), - }); +const rawTraces = readTraces(args.traces); +const secondTraceFiles = role === "fix" ? args.fixedTraces : args.againstTraces; +const secondRules = new Set(role === "fix" ? args.fixedRules : args.againstRules); +const rawSecondTraces = readTraces(secondTraceFiles); +const traces = firstBad(rawTraces); +const secondTraces = firstBad(rawSecondTraces); +const { lacks, extra, guard } = secondGraph + ? readCheck({ graph, rawTraces, secondGraph, rawSecondTraces, L }) + : {}; +const second = secondGraph && modelOf(secondGraph, guard, definedLabel); const secondQ = second && quotient(second); -function readerPath(trace, key, labelOf = stepLabel) { +const hiddenStep = (rule, sameLabel, action, before, after) => + sameLabel && + !readerActions.has(action) && + ![...(ruleReads.get(rule) ?? [])].some( + (v) => v in before && stateKey(before[v]) !== stateKey(after[v]), + ); + +function readerPath(trace, m, key, view) { + const last = trace.steps.length - 1; const out = [{ key: key(trace.steps[0].vars), label: null, modelStep: 0 }]; for (let i = 1; i < trace.steps.length; i++) { const { action } = trace.steps[i]; const k = key(trace.steps[i].vars); - if (k === out[out.length - 1].key && !readerActions.has(action)) continue; const [before, after] = [trace.steps[i - 1].vars, trace.steps[i].vars]; - const label = labelOf(action, before, after) || action; - const guard = stepGuard(action, before, after); + const sameLabel = k === out[out.length - 1].key; + if (i < last && hiddenStep(trace.invariant, sameLabel, action, before, after)) continue; + const label = m.step(action, before, after, view) || action; + const guard = m.guard(action, before, after); out.push({ key: k, label, guard, action, modelStep: i }); } return out; } -const readerLabel = (action, before, after) => stepLabel(action, before, after, "customer"); -function replaySecond(trace) { - if (!second) return null; - const shared = (vars) => stateKey(omit(vars, lacks)); - const sharedOf = (n) => stateKey(omit(n.raw, extra)); - const out = new Map(); - for (const e of [...second.graph.edges, ...second.graph.loops]) { - if (!out.has(e.from)) out.set(e.from, []); - out.get(e.from).push(e); - } - const actions = new Set([...second.graph.edges, ...second.graph.loops].map((e) => e.action)); - const nodeOf = (id) => second.graph.nodes.get(id); - const screen = new Map( - [...second.graph.nodes.values()].map((n) => [n.id, second.customer(n.vars)]), - ); - const words = (e) => readerLabel(e.action, nodeOf(e.from).vars, nodeOf(e.to).vars) || e.action; - const settle = (ids) => { - const all = new Set(ids); - for (const id of all) - for (const e of out.get(id) ?? []) - if (!readerActions.has(e.action) && screen.get(e.to) === screen.get(id)) all.add(e.to); - return all; - }; - const seen = readerPath(trace, main.customer, readerLabel); - const shownAt = new Map(seen.map((p, j) => [p.modelStep, j])); - let here = settle( - [...second.graph.nodes.values()] - .filter((n) => n.init && screen.get(n.id) === seen[0].key) - .map((n) => n.id), - ); - if (here.size === 0) return { reason: "starts on another screen" }; - const passed = new Set(here); - let j = 0; - for (let i = 1; i < trace.steps.length; i++) { - const { action } = trace.steps[i]; - const hidden = !shownAt.has(i); - if (!hidden) j = shownAt.get(i); - const want = seen[j]; - const [before, after] = [trace.steps[i - 1].vars, trace.steps[i].vars]; - const known = actions.has(action); - if (!known && (hidden || shared(before) === shared(after))) continue; - const tries = [...here].flatMap((id) => - (out.get(id) ?? []).filter((e) => - known ? e.action === action : sharedOf(nodeOf(e.to)) === shared(after), - ), - ); - if (hidden && tries.length === 0) continue; - const same = known - ? tries.filter((e) => screen.get(e.to) === want.key && (hidden || words(e) === want.label)) - : tries; - if (same.length) { - here = settle(same.map((e) => e.to)); - for (const id of here) passed.add(id); - continue; - } - const outcomes = new Map(); - for (const e of tries) { - const id = `${screen.get(e.to)}${SEP}${words(e)}`; - outcomes.set(id, [...(outcomes.get(id) ?? []), e]); - } - const taken = [...outcomes.values()].sort((a, b) => b.length - a.length)[0]?.[0]; - const from = main.key(before); - const said = hidden ? readerLabel(action, before, after) || action : want.label; - const at = { step: j, said, from, hidden }; - if (!taken) - return { - ...at, - key: from, - label: stepLabel(action, before, after) || action, - disabled: true, - }; - const [b, a] = [nodeOf(taken.from).vars, nodeOf(taken.to).vars]; - return { ...at, key: second.key(a), label: stepLabel(taken.action, b, a) || taken.action }; - } - const bad = new Set( - rawSecondTraces - .filter((t) => t.invariant === trace.invariant) - .map((t) => stateKey(t.steps[t.steps.length - 1].vars)), - ); - const checked = secondRules.has(trace.invariant); - const ruleHolds = checked && ![...passed].some((id) => bad.has(stateKey(nodeOf(id).vars))); - return { sameEnd: true, checked, ruleHolds }; -} +const mainActions = new Set([...graph.edges, ...graph.loops].map((e) => e.action)); -function readTraces(files, paths = []) { +function readTraces(files) { return files.flatMap((f) => { const text = fs.readFileSync(f, "utf8"); const warning = tlcOutputWarning(text, f); if (warning) console.error(`${f}: ${warning}`); - return parseTraces(text).map((t) => ({ - ...t, - steps: t.steps.map((s) => ({ ...s, vars: guard(s.vars, paths) })), - })); + return parseTraces(text); }); } -function firstBad(raw) { - const badStates = new Map(); - for (const t of raw) { - if (!badStates.has(t.invariant)) badStates.set(t.invariant, new Set()); - badStates.get(t.invariant).add(stateKey(t.steps[t.steps.length - 1].vars)); - } - const cuts = []; - const seen = new Set(); - for (const t of raw) { - const first = t.steps.findIndex((s) => badStates.get(t.invariant).has(stateKey(s.vars))); - const cut = { invariant: t.invariant, steps: t.steps.slice(0, first + 1) }; - const id = `${t.invariant}${SEP}${stateKey(cut.steps[cut.steps.length - 1].vars)}`; - if (seen.has(id)) continue; - seen.add(id); - cuts.push(cut); - } - return cuts; -} -const rawTraces = readTraces(args.traces); -const secondTraceFiles = role === "fix" ? args.fixedTraces : args.againstTraces; -const secondRules = new Set(role === "fix" ? args.fixedRules : args.againstRules); -const rawSecondTraces = readTraces(secondTraceFiles, lacks); -const traces = firstBad(rawTraces); -const secondTraces = firstBad(rawSecondTraces); function group(cuts, m) { const byId = new Map(); for (const trace of cuts) { - const rp = readerPath(trace, m.customer); + const rp = readerPath(trace, m, m.customer); const last = trace.steps[trace.steps.length - 1].vars; - const said = - typeof mapping.cause === "function" - ? mapping.cause(last, m.facts(last), trace.invariant) - : null; - const cause = said === undefined || said === null ? null : String(said); + const cause = m.cause(last, trace.invariant); const id = `${trace.invariant}${SEP}${cause ?? rp[rp.length - 1].key}`; const prev = byId.get(id); + const ends = new Set([...(prev?.ends ?? []), stateKey(last)]); if ( !prev || rp.length < prev.steps || (rp.length === prev.steps && trace.steps.length < prev.trace.steps.length) ) { - byId.set(id, { - id, - cause, - steps: rp.length, - trace, - invariant: trace.invariant, - count: (prev?.count ?? 0) + 1, - }); - } else prev.count++; + byId.set(id, { id, cause, steps: rp.length, trace, invariant: trace.invariant, ends }); + } else prev.ends = ends; } return byId; } -const byOrder = (a, b) => - a.steps - b.steps || - a.trace.steps.length - b.trace.steps.length || - (a.id < b.id ? -1 : a.id > b.id ? 1 : 0); const inSecond = (l) => `${within} ${L}, ${l}`; const modelEdges = new Map(now.edges); if (secondQ) @@ -599,70 +240,74 @@ if (secondQ) guards: new Map([...e.guards].map(([l, words]) => [inSecond(l), words])), secondOnly: true, }); -function divergence(s) { - if (s?.key === undefined) return s; - const plain = edgeId(s.from, s.key); - const text = inSecond(`${s.label}${s.disabled ? " cannot happen" : ""}`); - const drawn = !s.disabled && [s.label, text].some((l) => modelEdges.get(plain)?.labels.has(l)); - return { ...s, text, edge: drawn ? plain : `${plain}${SEP}second` }; -} const withPath = (c, m) => { - const path = readerPath(c.trace, m.key); + const path = readerPath(c.trace, m, m.key); return { ...c, path, end: path[path.length - 1].key }; }; const classesById = group(traces, main); -const mainClasses = [...classesById.values()] - .sort(byOrder) - .map((c) => ({ ...withPath(c, main), second: divergence(replaySecond(c.trace)) })); +const mainClasses = [...classesById.values()].sort(byOrder).map((c, i) => { + const bug = withPath(c, main); + return { + ...bug, + second: divergence( + replaySecond(c.trace, { + main, + lacks, + extra, + second, + hiddenStep, + readerPath, + mainActions, + secondRules, + rawSecondTraces, + }), + bug.path, + i + 1, + { inSecond, modelEdges }, + ), + }; +}); const introduced = second ? [...group(secondTraces, second).values()] .filter((c) => !classesById.has(c.id)) .sort(byOrder) .map((c) => ({ ...withPath(c, second), second: null, introduced: true })) : []; -const classes = [...mainClasses, ...introduced]; +const classes = [ + ...stepsIntoBugs(mainClasses, main, rawTraces, args), + ...stepsIntoBugs(introduced, second, rawSecondTraces, args), +]; -const diverges = (c) => c.second?.key !== undefined; -function badKeys(m, keys, raw) { - const g = m.graph; - const bad = new Set(raw.map((t) => stateKey(t.steps[t.steps.length - 1].vars))); - const out = new Set(raw.map((t) => m.key(t.steps[t.steps.length - 1].vars))); - const ok = (id) => !bad.has(stateKey(g.nodes.get(id).vars)); - const next = new Map(); - for (const e of g.edges) { - if (!next.has(e.from)) next.set(e.from, []); - next.get(e.from).push(e.to); - } - const clean = new Set([...g.nodes.values()].filter((n) => n.init && ok(n.id)).map((n) => n.id)); - const queue = [...clean]; - while (queue.length) { - for (const to of next.get(queue.shift()) ?? []) { - if (clean.has(to) || !ok(to)) continue; - clean.add(to); - queue.push(to); - } - } - const reached = new Set([...clean].map((id) => keys.get(id))); - for (const k of keys.values()) if (!reached.has(k)) out.add(k); - return out; -} -const badNow = badKeys(main, now.keys, rawTraces); -const uncheckedTraces = rawTraces.filter((t) => !secondRules.has(t.invariant)); -const badSecond = second - ? new Set([ - ...badKeys(second, secondQ.keys, rawSecondTraces), - ...(role === "fix" ? badKeys(main, now.keys, uncheckedTraces) : []), - ]) - : new Set(); +const brokenNow = readBroken(args.broken); +const statusNow = labelStatus(main, now.keys, badStates(rawTraces, brokenNow)); +const statusSecond = second + ? labelStatus( + second, + secondQ.keys, + badStates(rawSecondTraces, readBroken(args.fixedBroken ?? args.againstBroken)), + ) + : () => undefined; +const statusUnchecked = + role === "fix" + ? labelStatus( + main, + now.keys, + badStates(rawTraces, brokenNow, (r) => !secondRules.has(r)), + ) + : () => undefined; const allNodes = new Set([...now.nodes, ...(secondQ?.nodes ?? [])]); const initial = new Set([...now.initial, ...(secondQ?.initial ?? [])]); -const onlySecond = (k) => second && !now.nodes.has(k); const status = (k) => { - if (role === "against") - return badNow.has(k) || badSecond.has(k) ? "wrong" : onlySecond(k) ? "fixed" : "works"; - if (role === "fix") - return badSecond.has(k) ? "wrong" : badNow.has(k) || onlySecond(k) ? "fixed" : "works"; - return badNow.has(k) ? "wrong" : "works"; + if (role === "against") { + const s = worst(statusNow(k), statusSecond(k)); + return s === "works" && !now.nodes.has(k) ? "fixed" : s; + } + if (role === "fix") { + const s = worst(statusSecond(k), statusUnchecked(k)); + if (s === "wrong" && !["wrong", "mixed"].includes(statusNow(k))) return "worse"; + return s === "works" && statusNow(k) !== "works" ? "fixed" : s; + } + return statusNow(k); }; const chosen = args.highlight > 0 ? classes[args.highlight - 1] : null; @@ -671,75 +316,23 @@ if (args.highlight > 0 && classes.length > 0 && !chosen) const onPath = new Set(chosen ? chosen.path.map((p) => p.key) : []); if (chosen && diverges(chosen)) onPath.add(chosen.second.key); -const edgesAlong = (c) => - c.path.slice(1).map((p, i) => ({ - id: edgeId(c.path[i].key, p.key), - from: c.path[i].key, - to: p.key, - label: p.label, - guard: p.guard, - })); -function chartEdges() { - const edges = new Map(modelEdges); - for (const { second: s } of classes.filter(diverges)) { - if (modelEdges.has(s.edge)) continue; - if (!edges.has(s.edge)) - edges.set(s.edge, { from: s.from, to: s.key, labels: new Set(), secondOnly: true }); - edges.get(s.edge).labels.add(s.text); - } - const pathNumbers = new Map(); - (chosen ? edgesAlong(chosen) : []).forEach((step, i) => { - if (!pathNumbers.has(step.id)) pathNumbers.set(step.id, []); - const shown = step.guard ? `${step.label} [${step.guard}]` : step.label; - pathNumbers.get(step.id).push({ n: i + 1, label: step.label, shown }); - if (!edges.has(step.id)) - edges.set(step.id, { from: step.from, to: step.to, labels: new Set([step.label]) }); - }); - if (!args.cover) return { edges, pathNumbers, coverEdges: null }; - - const coverEdges = new Set(); - const bfs = (q, targets) => { - const parent = new Map([...q.initial].map((k) => [k, null])); - const queue = [...q.initial]; - const outEdges = new Map(); - for (const [id, e] of q.edges) { - if (!outEdges.has(e.from)) outEdges.set(e.from, []); - outEdges.get(e.from).push([id, e]); - } - while (queue.length) { - const k = queue.shift(); - for (const [id, e] of outEdges.get(k) ?? []) { - if (parent.has(e.to)) continue; - parent.set(e.to, id); - queue.push(e.to); - } - } - for (const t of targets) { - for (let id = parent.get(t); id; id = parent.get(q.edges.get(id).from)) coverEdges.add(id); - } - }; - bfs(now, now.settled); - if (secondQ) - bfs( - secondQ, - [...secondQ.settled].filter((k) => !now.nodes.has(k)), - ); - const pathLabels = new Map(); - for (const c of classes) { - for (const step of edgesAlong(c)) { - coverEdges.add(step.id); - pathLabels.set(step.id, (pathLabels.get(step.id) ?? new Set()).add(step.label)); - } - if (diverges(c)) coverEdges.add(c.second.edge); - } - for (const id of coverEdges) { - if (edges.has(id)) continue; - const [from, to] = id.split(`${SEP}${SEP}`); - edges.set(id, { from, to, labels: pathLabels.get(id) }); - } - return { edges, pathNumbers, coverEdges }; -} -const { edges, pathNumbers, coverEdges } = chartEdges(); +const coverOff = + args.cover && + ![graph, secondGraph] + .filter(Boolean) + .some((g) => [...g.edges, ...g.loops].some((e) => readerActions.has(e.action))) + ? "no step in TLC's graph is a reader action" + : null; +if (coverOff && !args.pathOnly) console.error(`--cover: drew every state, because ${coverOff}`); +const { edges, pathNumbers, coverEdges } = chartEdges({ + args, + now, + secondQ, + modelEdges, + classes, + chosen, + coverOff, +}); const LEGEND = { works: ["legendWorks", mapping.legend?.works ?? (role === "fix" ? "Works today" : "Works")], @@ -751,6 +344,17 @@ const LEGEND = { "legendFixed", mapping.legend?.fixed ?? (role === "fix" ? `Fixed or new with ${L}` : `Only in ${L}`), ], + mixed: [ + "legendMixed", + mapping.legend?.mixed ?? `Breaks a rule in some cases${role === "fix" ? ` with ${L}` : ""}`, + ], + after: [ + "legendAfter", + mapping.legend?.after ?? `Reached only after a bug${role === "fix" ? ` with ${L}` : ""}`, + ], + ...(role === "fix" + ? { worse: ["legendWorse", mapping.legend?.worse ?? `Breaks a rule only with ${L}`] } + : {}), }; const grouping = typeof mapping.group === "function"; @@ -760,7 +364,7 @@ const groupNames = []; const fills = new Map(); if (grouping) { const add = (k, vars, m) => { - const g = mapping.group(vars, m.facts(vars)); + const g = m.group(vars); if (g !== undefined && g !== null && typeof g !== "string") throw new Error(`group() must return a group name or nothing; got ${JSON.stringify(g)}`); if (!groupsOf.has(k)) groupsOf.set(k, new Set()); @@ -811,7 +415,14 @@ if (grouping) { } const mixedGroups = [...groupsOf].filter(([, gs]) => gs.size > 1).map(([k]) => k); -const BORDER = { works: "", wrong: "Wrong", fixed: "Fixed" }; +const BORDER = { + works: "", + wrong: "Wrong", + fixed: "Fixed", + mixed: "Mixed", + after: "After", + worse: "Worse", +}; const classFor = (g, s, path) => grouping ? `${g ? `group${groupNames.indexOf(g)}` : "plain"}${BORDER[s] || (path ? "Path" : "")}` @@ -826,15 +437,6 @@ const groupCounts = (drawn) => })) .filter((g) => g.states > 0); -const frontMatter = (title) => [ - "---", - ...(title ? [`title: ${JSON.stringify(title)}`] : []), - "config:", - " markdownAutoWrap: false", - " state:", - " useMaxWidth: false", - "---", -]; function head(drawn) { const out = [...frontMatter(args.title), "stateDiagram-v2"]; if (role) out.push(` %% second model (${role}): ${L}`); @@ -853,7 +455,7 @@ function head(drawn) { const styleClasses = new Set([...drawn.map((d) => d.cls), ...legend.map(([, , c]) => c)]); const bases = [["plain", PLAIN_FILL], ...groupNames.map((g, i) => [`group${i}`, fills.get(g)])]; for (const [base, fill] of bases) - for (const border of ["", "Wrong", "Fixed", "Path"]) + for (const border of [...Object.values(BORDER), "Path"]) if (styleClasses.has(base + border)) out.push(` ${groupClassDef(base, fill, border)}`); } else for (const c of STATUS_CLASSES) out.push(` ${c}`); if (legend.length) { @@ -865,293 +467,22 @@ function head(drawn) { return out; } -function regionsChart({ states, starts, edges, classes, proofPath, allPath }) { - const norm = (s) => s.replaceAll(";", ",").replace(/\s+/g, " ").trim(); - const split = (k) => { - const at = k.indexOf(": "); - return at > 0 && at + 2 < k.length ? [norm(k.slice(0, at)), norm(k.slice(at + 2))] : null; - }; - const nodes = [...states].sort(); - if (!nodes.every(split)) return null; - const steps = []; - for (const e of edges.values()) { - if (e.labels.size === 0) return { why: "a step has no label" }; - for (const l of e.labels) - steps.push({ - from: split(e.from), - to: split(e.to), - label: norm(l), - words: e.guards?.get(l) ?? new Set([""]), - }); - } - const bar = (p) => p.join(" | "); - const flat = { - Main: { - states: new Set(nodes.map((k) => bar(split(k)))), - initial: new Set(starts.map((k) => bar(split(k)))), - transitions: steps.map((s) => ({ from: bar(s.from), to: bar(s.to), label: s.label })), - }, - }; - const ids = new Map([...flat.Main.states].map((s, i) => [s, `p${i}`])); - const proof = [ - ...frontMatter(), - "stateDiagram-v2", - ...[...ids].map(([s, id]) => ` state "${escapeMermaidQuotes(s)}" as ${id}`), - ...[...flat.Main.initial].map((s) => ` [*] --> ${ids.get(s)}`), - ...steps.map((s) => ` ${ids.get(bar(s.from))} --> ${ids.get(bar(s.to))} : ${s.label}`), - ]; - if (proofPath) fs.writeFileSync(proofPath, proof.join("\n") + "\n"); - - const beside = [new Map(), new Map()]; - for (const k of nodes) { - const p = split(k); - for (const i of [0, 1]) { - if (!beside[i].has(p[i])) beside[i].set(p[i], new Set()); - beside[i].get(p[i]).add(p[1 - i]); - } - } - const size = beside[0].size + beside[1].size; - if (size >= nodes.length) - return { why: `${size} region states are not fewer than the ${nodes.length} flat states` }; - - const rank = ["works", "fixed", "wrong"]; - const regionStatus = [new Map(), new Map()]; - for (const k of nodes) { - const p = split(k); - for (const i of [0, 1]) { - const prev = regionStatus[i].get(p[i]); - if (!prev || rank.indexOf(status(k)) < rank.indexOf(prev)) - regionStatus[i].set(p[i], status(k)); - } - } - for (const k of nodes) { - const p = split(k); - const shown = rank[Math.max(...[0, 1].map((i) => rank.indexOf(regionStatus[i].get(p[i]))))]; - if (shown !== status(k)) - return { - why: `"${k}" is "${LEGEND[status(k)][1]}", and its two region states would show "${LEGEND[shown][1]}"`, - }; - } - - const moves = [new Map(), new Map()]; - for (const s of steps) - for (const i of [0, 1]) { - const id = [s.from[i], s.label, s.to[i]].join(SEP); - if (!moves[i].has(id)) - moves[i].set(id, { - from: s.from[i], - label: s.label, - to: s.to[i], - beside: new Set(), - guards: new Map(), - }); - moves[i].get(id).beside.add(s.from[1 - i]); - for (const w of s.words) addGuard(moves[i].get(id).guards, s.label, norm(w)); - } - for (const m of moves) - for (const [id, x] of m) - if ( - x.from === x.to && - ![...m.values()].some((y) => y.from === x.from && y.label === x.label && y.to !== x.from) - ) - m.delete(id); - const rid = beside.map((m, i) => new Map([...m.keys()].sort().map((s, j) => [s, `r${i}s${j}`]))); - const regionGroup = [new Map(), new Map()]; - for (const k of nodes) { - const p = split(k); - for (const i of [0, 1]) { - const prev = regionGroup[i].get(p[i]); - regionGroup[i].set(p[i], prev === undefined || prev === groupOf(k) ? groupOf(k) : ""); - } - } - const drawn = rid.map((m, i) => - [...m.keys()].map((s) => drawnState(regionGroup[i].get(s), regionStatus[i].get(s))), - ); - const draw = (only) => { - const out = head(drawn.flat()); - out.push(` state "At the same time" as regions {`); - let count = 0; - for (const i of [0, 1]) { - if (i) out.push(" --"); - const name = mapping.regionNames?.[i]; - const pad = name ? " " : " "; - if (name) out.push(` state "${escapeMermaidQuotes(wrap(name))}" as region${i} {`); - for (const s of new Set(starts.map((k) => split(k)[i]))) - out.push(`${pad}[*] --> ${rid[i].get(s)}`); - for (const [s, id] of rid[i]) - out.push(`${pad}state "${escapeMermaidQuotes(wrap(s))}" as ${id}`); - const shownMoves = [...moves[i]] - .filter(([id]) => !only || only[i].has(id)) - .map(([, m]) => m) - .sort((a, b) => a.from.localeCompare(b.from) || a.label.localeCompare(b.label)); - for (const m of shownMoves) { - const all = beside[i].get(m.from); - const not = [...all].filter((s) => !m.beside.has(s)).sort(); - const yes = [...m.beside].sort(); - const unless = not.length > 0 && not.length < yes.length; - const guard = not.length === 0 ? null : unless ? not : yes; - const worded = guarded(m.guards, m.label); - const text = - worded !== m.label || !guard - ? worded - : `${m.label} [${unless ? "unless" : "when"} ${guard.join(" or ")}]`; - out.push(`${pad}${rid[i].get(m.from)} --> ${rid[i].get(m.to)} : ${wrap(text, 30)}`); - if (guard) - out.push( - `${pad}%% when ${unless ? "not " : ""}${guard.map((s) => rid[1 - i].get(s)).join(", ")}`, - ); - count++; - } - if (name) out.push(" }"); - } - out.push(" }"); - for (const i of [0, 1]) - [...rid[i].values()].forEach((id, j) => out.push(` class ${id} ${drawn[i][j].cls}`)); - return { mermaid: out.join("\n") + "\n", count }; - }; - - const all = draw(null); - const differences = compareCharts(parseMermaid(all.mermaid).regions, flat); - if (differences.length) { - const n = differences.reduce((sum, [, xs]) => sum + xs.length, 0); - return { - why: `the regions do not combine into the flat chart, with ${n} difference(s), such as ${differences[0][0]}: ${differences[0][1][0]}`, - }; - } - if (allPath) fs.writeFileSync(allPath, all.mermaid); - - const parts = new Map(nodes.map((k) => [bar(split(k)), split(k)])); - const key = (from, label, to) => `${from} --> ${to} : ${label}`; - const real = new Set(steps.map((s) => key(bar(s.from), s.label, bar(s.to)))); - const next = new Map(); - for (const s of steps) { - if (!next.has(bar(s.from))) next.set(bar(s.from), []); - next.get(bar(s.from)).push(s); - } - const keep = [new Set(), new Set()]; - const movesOf = (s) => - [0, 1] - .filter((i) => s.from[i] !== s.to[i]) - .map((i) => [i, [s.from[i], s.label, s.to[i]].join(SEP)]); - const price = (s) => movesOf(s).filter(([i, id]) => !keep[i].has(id)).length * nodes.length + 1; - const wanted = []; - const want = (s) => { - wanted.push(s); - for (const [i, id] of movesOf(s)) keep[i].add(id); - }; - const stepsOf = (e, said) => { - const labels = [...e.labels].map(norm); - return (labels.includes(said) ? [said] : labels).map((label) => ({ - from: split(e.from), - to: split(e.to), - label, - })); - }; - for (const c of classes) { - for (const step of edgesAlong(c)) { - const e = edges.get(step.id); - if (e) stepsOf(e, norm(step.label)).forEach(want); - } - if (diverges(c)) stepsOf(edges.get(c.second.edge), "").forEach(want); - } - const cheapest = () => { - const cost = new Map(starts.map((k) => [bar(split(k)), 0])); - const parent = new Map(); - const open = new Set(cost.keys()); - while (open.size) { - const k = [...open].reduce((a, b) => (cost.get(b) < cost.get(a) ? b : a)); - open.delete(k); - for (const s of next.get(k) ?? []) { - const c = cost.get(k) + price(s); - if (c < (cost.get(bar(s.to)) ?? Infinity)) { - cost.set(bar(s.to), c); - parent.set(bar(s.to), s); - open.add(bar(s.to)); - } - } - } - return { cost, parent }; - }; - const targets = [ - ...[0, 1].flatMap((i) => - [...beside[i].keys()].sort().map((r) => ({ at: (k) => parts.get(k)[i] === r })), - ), - ...[...new Set(steps.map((s) => s.label))] - .filter((l) => !wanted.some((s) => s.label === l)) - .sort() - .map((l) => ({ ends: (s) => s.label === l })), - ]; - while (targets.length) { - const { cost, parent } = cheapest(); - let best = null; - targets.forEach((t, n) => { - const ways = t.at - ? [...cost.keys()].filter(t.at).map((k) => ({ k, c: cost.get(k), last: null })) - : steps - .filter(t.ends) - .filter((s) => cost.has(bar(s.from))) - .map((s) => ({ k: bar(s.from), c: cost.get(bar(s.from)) + price(s), last: s })); - for (const w of ways) if (!best || w.c < best.c) best = { ...w, n }; - }); - if (!best) break; - targets.splice(best.n, 1); - const path = best.last ? [best.last] : []; - for (let s = parent.get(best.k); s; s = parent.get(bar(s.from))) path.push(s); - path.forEach(want); - } - const takes = (x, label) => - [0, 1].flatMap((i) => - [...moves[i]] - .filter(([, m]) => m.from === x[i] && m.label === label && m.beside.has(x[1 - i])) - .map(([id]) => [i, id]), - ); - let scoped; - let made; - for (;;) { - scoped = draw(keep); - made = productOf(parseMermaid(scoped.mermaid).regions); - const madeKeys = new Set(made.transitions.map((t) => key(t.from, t.label, t.to))); - const redo = [ - ...made.transitions - .filter((t) => parts.has(t.from) && !real.has(key(t.from, t.label, t.to))) - .map((t) => [parts.get(t.from), t.label]), - ...wanted - .filter((s) => made.states.has(bar(s.from))) - .filter((s) => !madeKeys.has(key(bar(s.from), s.label, bar(s.to)))) - .map((s) => [s.from, s.label]), - ]; - const before = keep[0].size + keep[1].size; - for (const [x, label] of redo) for (const [i, id] of takes(x, label)) keep[i].add(id); - if (keep[0].size + keep[1].size === before) break; - } - const sound = - [...made.states].every((s) => parts.has(s)) && - made.transitions.every((t) => real.has(key(t.from, t.label, t.to))); - if (!sound) scoped = all; - - return { - mermaid: scoped.mermaid, - states: [beside[0].size, beside[1].size], - transitions: scoped.count, - all: all.count, - flat: { states: nodes.length, transitions: steps.length }, - status: Object.fromEntries( - rank.map((s) => [ - s, - regionStatus.reduce((n, m) => n + [...m.values()].filter((x) => x === s).length, 0), - ]), - ), - groups: groupCounts(drawn.flat()), - }; -} const regions = args.regions ? regionsChart({ states: allNodes, starts: [...initial], edges, + loops: Boolean(args.loops), classes, proofPath: args.proof, allPath: args.regionsAll, + mapping, + status, + LEGEND, + groupOf, + drawnState, + groupCounts, + head, }) : null; if (regions?.why) console.error(`regions: not drawn, because ${regions.why}`); @@ -1174,8 +505,9 @@ const inBox = (k, box) => { if (!boxes.has(box)) boxes.set(box, []); boxes.get(box).push(k); }; -if (args.view === "layered") for (const k of sorted) inBox(k, k.split(SEP)[0]); -else if (args.view === "customer" && !args.flat) { +if (args.view === "layered" && !args.flat) { + for (const k of sorted) if (k.includes(SEP)) inBox(k, k.slice(0, k.indexOf(SEP))); +} else if (args.view === "customer" && !args.flat) { for (const k of sorted) { const at = k.indexOf(": "); if (at > 0 && at + 2 < k.length) inBox(k, k.slice(0, at)); @@ -1187,8 +519,8 @@ for (const [box, members] of boxes) for (const k of members) boxOf.set(k, box); const declared = [...[...boxes.values()].flat(), ...sorted.filter((k) => !boxOf.has(k))]; const stateIds = new Map(declared.map((k, i) => [k, `s${i}`])); const idOf = (k) => stateIds.get(k); -const caption = (k) => - args.view === "layered" ? k.split(SEP)[1] : boxOf.has(k) ? k.slice(boxOf.get(k).length + 2) : k; +const show = (k) => (args.view === "layered" ? k.replace(SEP, ": ") : k); +const caption = (k) => (boxOf.has(k) ? show(k).slice(boxOf.get(k).length + 2) : show(k)); function numbered(steps) { const runs = []; @@ -1205,7 +537,7 @@ for (const [id, e] of edges) { if (!flatStates.has(e.from) || !flatStates.has(e.to)) continue; if (args.pathOnly && chosen && !pathNumbers.has(id) && id !== chosen.second?.edge) continue; if (coverEdges && !coverEdges.has(id)) continue; - const pair = edgeId(e.from, e.to); + const pair = e.apart ? id : edgeId(e.from, e.to); if (!arrows.has(pair)) arrows.set(pair, { from: e.from, to: e.to, groups: [[], [], []], plain: true, labels: [] }); const a = arrows.get(pair); @@ -1241,7 +573,9 @@ const transitions = [...arrows.values()].map(({ from, to, groups, plain, labels })); const fromBox = []; const folded = new Set(); +const plainNames = new Set(sorted.filter((k) => !boxOf.has(k)).map(caption)); for (const [box, members] of boxes) { + if (plainNames.has(box)) continue; const mine = new Set(members); const byStep = new Map(); for (const l of transitions) { @@ -1277,7 +611,6 @@ const mermaid = regions?.mermaid ?? out.join("\n") + "\n"; if (args.out) fs.writeFileSync(args.out, mermaid); else process.stdout.write(mermaid); -const show = (k) => (args.view === "layered" ? k.replace(SEP, ": ") : k); const ruleOf = (c) => rules.get(c.invariant) ?? `\`${c.invariant}\``; function secondLine(c) { const W = cap(within); @@ -1286,21 +619,23 @@ function secondLine(c) { if (!s) return null; if (s.sameEnd) return !s.checked - ? `${W} ${L} you see the same screens and steps, but ${L} does not check this rule.` + ? `${W} ${L} you see the same states and steps, but ${L} does not check this rule.` : s.ruleHolds - ? `${W} ${L} you see the same screens and steps, yet the rule holds, so the state labels do not show what ${L} changes.` - : `${W} ${L} you see the same screens and steps, so ${L} does not change this path.`; + ? `${W} ${L} you see the same states and steps, yet the rule holds, so the state labels do not show what ${L} changes.` + : `${W} ${L} you see the same states and steps, so ${L} does not change this path.`; if (s.reason) return `${cap(L)} cannot take these steps: it ${s.reason}.`; const outcome = s.disabled ? "cannot happen, so you stay at" : s.key === s.from ? "keeps you at" : "takes you to"; + const as = s.own && s.own !== s.said ? ` as "${s.own}"` : ""; if (s.hidden) { const when = s.step ? `after step ${s.step}` : "before your first step"; - return `${W} ${L}, ${when}, "${s.said}" happens and ${outcome}: ${show(s.key)}.`; + return `${W} ${L}, ${when}, "${s.said}" happens${as} and ${outcome}: ${show(s.key)}.`; } - return `${W} ${L}, step ${s.step} (${s.said}) ${outcome}: ${show(s.key)}.`; + const happens = as ? `happens${as} and ` : ""; + return `${W} ${L}, step ${s.step} (${s.said}) ${happens}${outcome}: ${show(s.key)}.`; } const listedIn = (c) => Object.keys(args.listed.breaks).filter((cfg) => args.listed.breaks[cfg] === c.invariant); @@ -1320,17 +655,24 @@ if (args.story) { lines.push(`Rule broken: ${ruleOf(c)}`); if (c.cause) lines.push(`Why: ${c.cause.replace(/\.$/, "")}.`); if (args.listed) lines.push(listedLine(c)); - lines.push( - c.count === 1 - ? "This is the only way to get here in the model." - : `There are ${c.count} ways to get here in the model. This is the shortest.`, - "", - ); - c.path.forEach((p, j) => - lines.push( - j === 0 ? `0. Start: ${show(p.key)}` : `${j}. ${cap(p.label)}. Now: ${show(p.key)}`, - ), - ); + const ways = waysLine(c); + lines.push(...(ways ? [ways] : []), ""); + lines.push(`0. Start: ${show(c.path[0].key)}`); + const m = c.introduced ? second : main; + c.path.slice(1).forEach((p, j) => { + const from = c.path[j].modelStep + 1; + const hidden = c.trace.steps + .slice(from, p.modelStep) + .map( + (s, h) => + m.step(s.action, c.trace.steps[from + h - 1].vars, s.vars, "customer") || s.action, + ); + if (hidden.length) + lines.push( + ` (${hidden.length} hidden step${hidden.length === 1 ? "" : "s"}: ${[...new Set(hidden)].join(", ")})`, + ); + lines.push(`${j + 1}. ${cap(p.label)}. Now: ${show(p.key)}`); + }); lines.push(""); const said = secondLine(c); if (said) lines.push(said, ""); @@ -1352,12 +694,37 @@ const secondOf = (c) => : c.second ? c.second.sameEnd ? !c.second.checked - ? "same screens, rule not checked" + ? "same states, rule not checked" : c.second.ruleHolds - ? "same screens, but the rule holds" + ? "same states, but the rule holds" : "unchanged" : (c.second.reason ?? show(c.second.key)) : null; +function coverSummary() { + if (coverOff || regions?.mermaid) + return { drawn: false, why: coverOff ?? "the overview has parallel regions" }; + const pairs = new Set(); + const missed = new Map(); + for (const [id, e] of edges) { + const pair = e.apart ? id : edgeId(e.from, e.to); + pairs.add(pair); + if (coverEdges.has(id)) continue; + if (!missed.has(pair)) missed.set(pair, { from: e.from, to: e.to, labels: [] }); + missed.get(pair).labels.push(...[...e.labels].map((l) => guarded(e.guards, l))); + } + return { + drawn: true, + states: allNodes.size, + transitions: pairs.size, + leftOutStates: [...allNodes] + .filter((k) => !flatStates.has(k)) + .map(show) + .sort(), + leftOutTransitions: [...missed.values()] + .map((m) => `${show(m.from)} --> ${show(m.to)}: ${joinLabels(m.labels)}`) + .sort(), + }; +} const summary = { view: args.view, ...(role @@ -1377,11 +744,16 @@ const summary = { chartStatus: regions?.mermaid ? regions.status : Object.fromEntries( - ["works", "wrong", "fixed"].map((s) => [ + Object.keys(LEGEND).map((s) => [ s, - [...flatStates].filter((k) => status(k) === s).length, + [...(args.pathOnly && chosen ? onPath : allNodes)].filter((k) => status(k) === s).length, ]), ), + mixedStatus: [...allNodes] + .filter((k) => status(k) === "mixed") + .map(show) + .sort(), + ...(args.cover ? { cover: coverSummary() } : {}), ...(grouping ? { chartGroups: regions?.mermaid ? regions.groups : groupCounts(drawn), @@ -1409,7 +781,9 @@ const summary = { cause: c.cause, readerSteps: c.path.length - 1, modelSteps: c.trace.steps.length, - counterexamples: c.count, + counterexamples: c.ends.size, + lastSteps: c.lastSteps, + startsHere: c.startsHere, [role ?? "fix"]: secondOf(c), ...(args.listed ? { listed: listedIn(c) } : {}), })), diff --git a/pstack/skills/statechart/scripts/tlc-tools.mjs b/pstack/skills/statechart/scripts/tlc-tools.mjs new file mode 100644 index 000000000..09d46de4a --- /dev/null +++ b/pstack/skills/statechart/scripts/tlc-tools.mjs @@ -0,0 +1,69 @@ +import { spawnSync } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import process from "node:process"; + +// TLC's v1.8.0 is a rolling pre-release whose jar is re-uploaded, so pin the newest immutable +// release. The scripts read 1.7.4's output, which has no trace spec. +export const TLC_TOOL = + "github:tlaplus/tlaplus[asset_pattern=tla2tools.jar,checksum=sha256:936a262061c914694dfd669a543be24573c45d5aa0ff20a8b96b23d01e050e88]@1.7.4"; +export const JAVA_TOOL = "java@temurin-25"; + +const SET_TLC_VARS = + "set TLA2TOOLS to the path of tla2tools.jar, and JAVA to a java binary (Java 11 or later)."; +const indent = (text) => text.replace(/^/gm, " "); + +export function findTlc() { + const { TLA2TOOLS, JAVA } = process.env; + for (const [name, file] of Object.entries({ JAVA, TLA2TOOLS })) + if (file && !fs.existsSync(file)) throw new Error(`${name} is ${file}, which does not exist.`); + if (TLA2TOOLS) return { jar: TLA2TOOLS, java: JAVA || "java" }; + const command = (tools) => + `cd "$(mktemp -d)" && MISE_CEILING_PATHS="$PWD" mise install ${tools.map((t) => `'${t}'`).join(" ")}`; + const empty = fs.mkdtempSync(path.join(os.tmpdir(), "statechart-mise-")); + const env = { ...process.env, MISE_CEILING_PATHS: empty }; + const run = (...args) => spawnSync("mise", args, { cwd: empty, env, encoding: "utf8" }); + const mise = (...args) => { + const r = run(...args); + if (r.error) + throw new Error( + `Cannot find TLC. It installs with mise, which cannot start: ${r.error.message}. Install mise (https://mise.jdx.dev), or ${SET_TLC_VARS}`, + ); + if (r.status === 0) return r.stdout.trim(); + const said = (r.stderr ?? "").trim(); + if (!said || /not installed|mise bin/.test(said)) return ""; + throw new Error( + `Cannot find TLC, because mise failed:\n${indent(said)}\nThe install command: ${command(JAVA ? [TLC_TOOL] : [JAVA_TOOL, TLC_TOOL])}\nFix that, or ${SET_TLC_VARS}`, + ); + }; + const resolve = () => { + const home = mise("where", TLC_TOOL); + const jar = home && path.join(home, "tla2tools.jar"); + const java = JAVA || mise("which", "java", "--tool", JAVA_TOOL); + return { jar: jar && fs.existsSync(jar) ? jar : "", java }; + }; + try { + const found = resolve(); + if (found.jar && found.java) return found; + const tools = [!found.java && JAVA_TOOL, !found.jar && TLC_TOOL].filter(Boolean); + const names = [!found.jar && "TLC", !found.java && "Java"].filter(Boolean); + console.error(`Installing ${names.join(" and ")} with mise. This happens once.`); + const r = run("install", "--quiet", ...tools); + const installed = resolve(); + if (installed.jar && installed.java) return installed; + const said = + `${r.stderr ?? ""}${r.stdout ?? ""}`.trim() || + `mise install exited ${r.status}, with no output`; + throw new Error( + `Cannot find TLC, and mise could not install it:\n${indent(said)}\nThe install command: ${command(tools)}\nOr ${SET_TLC_VARS}`, + ); + } finally { + fs.rmSync(empty, { recursive: true, force: true }); + } +} + +export function cannotRunTlc({ java, jar }, r) { + const said = `java exited ${r.status ?? r.signal}:\n${indent(`${r.stderr}${r.stdout}`.trim())}`; + return `cannot run TLC with ${java} -cp ${jar}: ${r.error?.message ?? said}\nFix that, or ${SET_TLC_VARS}`; +} diff --git a/pstack/skills/statechart/scripts/ways-in.mjs b/pstack/skills/statechart/scripts/ways-in.mjs new file mode 100644 index 000000000..e94658c80 --- /dev/null +++ b/pstack/skills/statechart/scripts/ways-in.mjs @@ -0,0 +1,69 @@ +import { joinAnd } from "./lib.mjs"; +import { badOf, reachableAvoiding } from "./bug-states.mjs"; + +export function stepsIntoBugs(cs, m, raw, { story, summary, next }) { + if (!cs.length || (!story && !summary)) return cs; + const { graph: g, stateKeys } = m; + const unnamed = new Set(next); + const rules = new Set(cs.map((c) => c.invariant)); + const clean = new Map([...rules].map((r) => [r, reachableAvoiding(m, badOf(raw, r))])); + const byEnd = new Map(); + for (const c of cs) for (const k of c.ends) byEnd.set(k, [...(byEnd.get(k) ?? []), c]); + const ways = new Map(cs.map((c) => [c, { steps: new Map(), n: 0, nameless: 0, action: null }])); + for (const e of g.edges) { + const into = byEnd.get(stateKeys.get(e.to))?.filter((c) => clean.get(c.invariant).has(e.from)); + if (!into?.length) continue; + const [before, after] = [g.nodes.get(e.from).vars, g.nodes.get(e.to).vars]; + const label = m.step(e.action, before, after) || e.action; + const moves = m.key(before) !== m.key(after); + for (const c of into) { + const w = ways.get(c); + const s = w.steps.get(label) ?? { n: 0, moves: false }; + w.steps.set(label, { n: s.n + 1, moves: s.moves || moves }); + w.n++; + if (!unnamed.has(e.action)) continue; + w.nameless++; + w.action = e.action; + } + } + const starts = [...g.nodes.values()].filter((n) => n.init).map((n) => stateKeys.get(n.id)); + return cs.map((c) => { + const { steps, n, nameless, action } = ways.get(c); + const lastSteps = [...steps] + .sort((a, b) => b[1].n - a[1].n || (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0)) + .map(([l]) => l); + const onPath = new Set(c.path.slice(1).map((p) => p.label)); + const startsIn = starts.filter((k) => c.ends.has(k)).length; + return { + ...c, + lastSteps, + keptSteps: new Set(lastSteps.filter((l) => !steps.get(l).moves && !onPath.has(l))), + nameless: nameless ? { action, all: nameless === n } : null, + waysIn: n + startsIn, + startsHere: startsIn > 0, + }; + }); +} + +export function waysLine(c) { + const quoted = c.lastSteps.map( + (l) => `"${l}"${c.keptSteps.has(l) ? " (the state stays the same)" : ""}`, + ); + const named = quoted.slice(0, 5); + const more = quoted.length - named.length; + const list = joinAnd(more ? [...named, `${more} more`] : named); + const steps = c.nameless + ? `In the model, steps labeled ${list} lead into this bug. TLC names ${c.nameless.all ? "them" : "some of them"} ${c.nameless.action}, the whole next-state action, so the story cannot count them.` + : c.lastSteps.length === 1 + ? `In the model, one step leads into this bug: ${list}.` + : c.lastSteps.length + ? `In the model, ${c.lastSteps.length} different steps lead into this bug: ${list}.` + : ""; + const start = !c.startsHere + ? "" + : steps + ? "The model can also start in this bug." + : "The model starts in this bug."; + const shortest = c.waysIn > 1 ? "The steps below are the shortest path that TLC found." : ""; + return [steps, start, shortest].filter(Boolean).join(" "); +}