Skip to content

Turn on --unicode by default to close #867 - #868

Merged
mathiasrw merged 2 commits into
mainfrom
fix/867
Oct 28, 2026
Merged

mathiasrw merged 2 commits into
mainfrom
fix/867

Conversation

@mathiasrw

Copy link
Copy Markdown
Owner

Closes #867
Also closes #776, the spike that decided it.

Change

  • The default flags of rr and -E runtime are gimu, so the flag u is on: . and [^…] take a whole emoji, [🦖🦕] and 🦖+ work, and \u{1F996} and \p{L} work without a flag.
  • --void-unicode (-U) turns it off, like the other --void-… flags, for the old reading as UTF-16 parts.
  • -u stays, so old scripts do not fail. With -E regex+ it keeps its meaning: u in place of v.
  • --flags still sets all the flags, so --flags gim is the old default. It can not go with -U either.
  • When a pattern only fails because of the strict syntax of u, the error says so:
> printf 'a"b' | rexreplace '\"' x
 ❌ Invalid regular expression: invalid escaped character for Unicode pattern
    It works without the flag u, which -U (--void-unicode) turns off.
    No further processing.
> printf 'a"b' | rexreplace '\"' x -U
axb

For the release note of v8: patterns with an escape that has no meaning, like \", \- or \#, a {, } or ] on its own, like a{ or foo], or an octal escape like \01 now stop the run. -U is the way back, and the error says it.

Tests

The tests that checked the old default now say voidUnicode: true, like the compatibility table of #854, which compares rr with the native engine with the same flags. --unicode "is not the default" is now "is the default, and --void-unicode turns it off", and a test checks the hint, and that a pattern that fails also without u gets no hint. The flags in the status report are gimu. The flag combinations of crossover.test.js use voidUnicode. yarn test-ci passes.

@mathiasrw
mathiasrw merged commit 4445544 into main Oct 28, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Turn on --unicode by default, with -U to turn it off Spike: turn on --unicode by default

1 participant