Skip to content

Fix compile() checking the source before an invalid mode - #1229

Open
jseop-lim wants to merge 1 commit into
oracle:masterfrom
jseop-lim:GH-1224-compile-mode-first
Open

jseop-lim wants to merge 1 commit into
oracle:masterfrom
jseop-lim:GH-1224-compile-mode-first

Conversation

@jseop-lim

Copy link
Copy Markdown
Contributor

Description

Fixes #1224

compile() read mode only after converting and checking the source, so an invalid source hid an invalid mode. The func_type mode's PyCF_ONLY_AST requirement was checked later still, so an AST source with that mode and no flag reached obj2sst and failed with ShouldNotReachHere. CPython's builtin_compile_impl dispatches on mode right after the flags and optimize checks (bltinmodule.c#L776-L798), before it converts the source (#L826).

AS-IS

>>> compile(None, 'f', 'x')
TypeError: compile() arg 1 must be a string, bytes or AST object
>>> compile('\x00', 'f', 'x')
SyntaxError: source code string cannot contain null bytes
>>> compile(b'\xff', 'f', 'x')
SyntaxError: (unicode error) 'utf-8' codec can't decode byte 0xff in position 0: malformed input (f, line 1)
>>> compile(ast.parse('1'), 'f', 'func_type')
com.oracle.truffle.api.CompilerDirectives$ShouldNotReachHere

TO-BE

>>> compile(None, 'f', 'x')
ValueError: compile() mode must be 'exec', 'eval' or 'single'
>>> compile('\x00', 'f', 'x')
ValueError: compile() mode must be 'exec', 'eval' or 'single'
>>> compile(b'\xff', 'f', 'x')
ValueError: compile() mode must be 'exec', 'eval' or 'single'
>>> compile(ast.parse('1'), 'f', 'func_type')
ValueError: compile() mode 'func_type' requires flag PyCF_ONLY_AST

(matching CPython)

Changes

  • In CompileNode.generic, resolve the input type with getParserInputType right after checkOptimize and reuse it for the AST path.
  • In CompileNode.compile, call getParserInputType before checkSource, which covers a str source that arrives through doCompile.
  • Move the func_type requires PyCF_ONLY_AST check from compile into getParserInputType, so both paths raise it with the other mode errors.
  • Add test_compile_checks_mode_before_source to tests/test_parser.py, with the file's first assert_raises helper.

Testing

mx python-jvm on linux-aarch64, then mx graalpytest test_parser.py test_exec.py test_ast.py test_code.py (108 passed). test.test_compile, test.test_builtin and test.test_ast from lib-python report the same failures and errors as on master. The new test fails on the unpatched build. Twenty compile() call shapes, including valid exec, eval, single and func_type compiles, AST sources, and func_type without the flag, print the same as CPython 3.12.13 except compile(None, 'f', 'x', optimize=5), which raises TypeError instead of ValueError before and after this change.

compile() resolved mode with getParserInputType only inside the
string compile path, after sourceAsString had converted the source and
checkSource had looked for null bytes, and checked that func_type comes
with PyCF_ONLY_AST later still. So compile(None, 'f', 'x') raised
TypeError, a source with a null byte or bytes that fail to decode
raised SyntaxError, and an AST source with mode 'func_type' but no
PyCF_ONLY_AST reached obj2sst and hit ShouldNotReachHere. CPython
reports all of these as ValueError.

CPython's builtin_compile_impl validates mode, including the func_type
flag requirement, right after the flags and optimize checks and before
it touches the source. getParserInputType now does the func_type
check, generic resolves the input type at that point and reuses it for
the AST path, and compile() resolves it before checkSource.

Signed-off-by: Jeongseop Lim <jeongseop_lim@korea.ac.kr>
@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Oct 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: compile() checks the source before mode, raising TypeError instead of ValueError

1 participant