Skip to content

Sync ess- items verbatim and add ess-scan hook - #143

Merged
mattnorris merged 20 commits into
mainfrom
sync
Nov 3, 2026
Merged

mattnorris merged 20 commits into
mainfrom
sync

Conversation

@mattnorris

@mattnorris mattnorris commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Syncs every ess- skill, instruction, and package byte for byte from the source repo. This adds ess-load-test to the uv workspace, replaces skills/ess/{git-tidy,mcp-hide-secrets,pr-review} with their ess- versions, and renames the write-safety rule to ess-cli-write-safety.mdc.
  • Publishes an ess-scan pre-commit hook (root package.json, src/ess-scan.ts) that runs the deterministic scanners on changed ess-* items, so a source repo can block secrets, jargon, and PII before it syncs.
  • Fixes two ways secretlint in essentials-sync scan passed without scanning:
    • It rejected paths outside its working directory and scanned nothing. It now runs in the scanned directory.
    • On macOS its JSON report was cut off at 64 KiB in the stdout pipe, and the unparseable rest read as clean. The report is now written with --output and read from a file, and an unparseable report is an error.
    • Tests cover a directory, a single file, and a report larger than a pipe buffer.
  • essentials-sync status compares only the files git would ship, and the PII scanner accepts emails on example.com, example.net, example.org, and example.edu, or under the reserved .example, .invalid, and .test TLDs, in any case. Look-alikes such as x@test.com are still flagged.
  • The built-in jargon wordlist gains the internal team, platform, and tool names a source repo used to list in its own override file, so a source repo needs no override file.
  • Sync no longer copies an item's own LICENSE in either direction, matching status, which already skipped it. The open-source copies keep their LICENSE, and a source repo can go without one. Each ess- skill also declares license: Apache-2.0 in its SKILL.md frontmatter, which syncs verbatim.
  • Every installable item now pairs a license declaration with the LICENSE file. Each Python package and tool declares license = "Apache-2.0" (with hatchling>=1.27) and ships a LICENSE, which its wheel bundles. docs-docusaurus and essentials-sync gain the LICENSE their npm packages lacked.
  • ess-langsmith-client: keys create and keys delete gain --dry-run, prompt on stderr by default, and require --yes for JSON or non-interactive writes.
  • Bumps ruff-pre-commit to v0.15.6 (ruff-check) so synced files are not reformatted on push.

Test plan

  • essentials-sync status against the source repo: 30 of 30 items identical. A reverse dry-run from this repo changes no files.
  • npm test in tools/typescript/essentials-sync passes.
  • pytest passes for each ess- package and the skill scripts.
  • pre-commit run --hook-stage pre-push --from-ref origin/main --to-ref HEAD passes.
  • pre-commit try-repo with ess-scan: passes on clean files, and fails on planted jargon, a non-example email, a GitHub token, and a private key.

- Add ess-django-docs, ess-env-credentials, ess-gh-repo-mirror,
  ess-git-commits, ess-postgres-docs, and ess-shell-flags-literal.
- Replace summarize-change-log with ess-summarize-change-log.
- Replace pr-address-comments and pr-address-comments-all with
  ess-pr-address-comments, which handles one PR or many.
- Add portable rules under instructions/ess/.
- Scanners and the verbatim copy share one file list from git ls-files,
  so ignore rules in parent directories count (a root-ignored .env is
  neither scanned nor copied). TruffleHog and secretlint get that list.
- The PII check accepts subdomains of example.com, .net, and .org.
- Add read-only `scan <path>...` and `status --source-repo --target-repo`.
- Publish `scan` as the ess-scan pre-commit hook.
Domains are case-insensitive, so Alice.Smith@Example.COM is a placeholder too. Tests that check case handling no longer need to lowercase their domains.
IANA reserves example.edu for documentation alongside example.com, .net, and .org, so the PII scanner now allows it too. Registrable lookalikes such as example.io are still flagged.
secretlint ran from the tool directory and rejected every path outside it with exit 2, which the scanner read as no findings, so it never scanned another repo. It now runs in the scanned directory on relative paths, and any exit code other than 0 or 1 is reported as a scanner error.
A root package.json lets the ess-scan hook use pre-commit's node language. It carries only the scanners' dependencies, so the hook no longer installs @cursor/sdk, and a scan-only entry point replaces the shell launcher that built the whole tool on first run.
Every ess- item now matches its source repo byte for byte, and
essentials-sync status reports 30 of 30 identical in both directions.

- Add ess-load-test, the ess-cli-write-safety and ess-docs-example-urls
  instructions, and the ess-gh-copilot-review and ess-my-groups skills.
- Rename git-tidy, mcp-hide-secrets, and pr-review to their ess- names.
- Update ess-auth, ess-browser, ess-langsmith-client, ess-outlook,
  ess-passwords, ess-service-now-incident, ess-messages,
  ess-hello-jwt-auth-code, and ess-summarize-change-log. ess-browser no
  longer ships company IdP defaults.
- Replace the cli-write-safety rule with ess-cli-write-safety and register
  ess-load-test in the uv workspace.
keys create now confirms before creating a key, offers --dry-run, and takes --yes, which --format json and non-interactive runs require. ess-auth comments now state the general failure mode instead of a specific incident.
v0.8.6 formats asserts differently from v0.15.6, so files synced from a repo on the newer version were reformatted on push. The ruff hook id is now ruff-check.
Long strings are wrapped, and the remaining ruff waivers are noqa comments in the files rather than per-file ignores in a root config, so the package lints the same wherever it is copied.
The JSON report embeds each file's source, and secretlint exited before a report over 64 KiB had drained from the stdout pipe on macOS. The cut-off JSON failed to parse and was read as a clean scan. The report is now written with --output and read from a file, and an unparseable report is an error.
secretlint's basic-auth rule flagged the user:password URL that the _safe_origin test strips. The credentials are fake, so the line carries a secretlint-disable-next-line directive.
Internal team, platform, and tool names that a source repo kept in its own override file are now built in, so a source repo needs no override file to keep them out of synced items.
status already skipped each item's LICENSE, since the open-source copy carries the license text and a source repo may not. Sync copied it anyway, so a sync from this repo would add LICENSE files to the source. Both now share REPO_OWNED_ITEM_FILES.
Each ess- skill states its license in SKILL.md frontmatter as well as carrying the LICENSE file, so the source copies, which have no LICENSE, still name it.
Names under .example, .invalid, and .test are reserved and never resolve, so they are as safe for placeholder emails as the example.* documentation domains. A reserved name must still end the address, so x@test.com is flagged.
Names under .example, .invalid, and .test are reserved by RFC 2606 and never resolve, so they are as safe for placeholders as the example.* domains. .invalid suits a value that must never resolve, such as a throwaway git identity.
Every ess- Python package now declares license = "Apache-2.0", as ess-langsmith-client already did, and requires hatchling>=1.27, which reads the SPDX expression (PEP 639).
Each Python package and tool, docs-docusaurus, and essentials-sync now carries the Apache-2.0 LICENSE, so a built wheel or npm package includes the text, as the JavaScript packages already did. The non-ess Python projects also declare license = "Apache-2.0" and require hatchling>=1.27.
Every default jargon pattern now reports "References company systems." instead of a sentence describing each internal system, so the published source no longer explains what each term is. Findings still name the matched term. The compound org term is dropped because its two parts already match.
@mattnorris
mattnorris merged commit a88832f into main Nov 3, 2026
2 checks passed
@mattnorris
mattnorris deleted the sync branch November 3, 2026 14:14
mattnorris added a commit that referenced this pull request Nov 3, 2026
- **feat(ess-sync): scan files git would ship** — Scanners and the verbatim copy share one git ls-files list, so a root-ignored file is neither scanned nor copied. secretlint runs in the scan root and its report is read from a file, and any exit other than 0 or 1 is a scanner error. scan is the ess-scan pre-commit hook, installed from a root package.json that carries only scanner dependencies.
- **feat(ess-sync): allow safe placeholder emails** — The PII check accepts example.com, .net, .org, and .edu in any case, plus names that end in .example, .invalid, or .test. Built-in jargon covers internal names with one message, so a source repo needs no override file, and the docs allow those reserved names in examples.
- **feat(ess): sync items byte for byte** — Every ess- item matches its source, including renamed ess- skills and new instructions, and status reports them identical both ways. Sync skips each item's LICENSE, ruff is pinned to v0.15.6 with hook id ruff-check, and lint waivers stay in the files so a push does not reformat synced source.
- **build(ess): ship Apache-2.0 license text** — Python packages and tools declare license = "Apache-2.0" and require hatchling>=1.27, and each package carries the LICENSE so wheels and npm packages include it. ess- skills name Apache-2.0 in SKILL.md frontmatter.
- **feat(cli): confirm before creating keys** — keys create confirms before creating a key, offers --dry-run, and requires --yes for JSON output and non-interactive runs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant