Skip to content

Only run fork hooks for subprocesses with a preexec_fn - #781

Open
ybai08 wants to merge 1 commit into
MagicStack:masterfrom
ybai08:fix-at-fork-hooks-without-preexec-fn
Open

ybai08 wants to merge 1 commit into
MagicStack:masterfrom
ybai08:fix-at-fork-hooks-without-preexec-fn

Conversation

@ybai08

@ybai08 ybai08 commented Oct 18, 2026

Copy link
Copy Markdown

Fixes #749.

uvloop called PyOS_BeforeFork() / PyOS_AfterFork_Parent() for every spawned process, and PyOS_AfterFork_Child() in the child whenever libuv forked. As a result, callbacks registered with os.register_at_fork() ran on every create_subprocess_exec() / subprocess_exec(), even though the child never re-enters the interpreter before exec(). CPython's _posixsubprocess (and so the default asyncio loop) only does this when a preexec_fn is passed, which is also what the os.register_at_fork() docs describe.

This change does the same:

  • the child fork handler is installed, and PyOS_BeforeFork() / PyOS_AfterFork_Parent() are called, only when preexec_fn is set;
  • the error pipe's write end is made inheritable only in that case, since the child handler is what closes it (otherwise it stays close-on-exec).

Without a preexec_fn the child no longer runs any Python code between fork() and exec(). The steps the child handler did in that case were already covered by libuv: it resets all signal dispositions to SIG_DFL in the child (uv__process_child_init, or posix_spawnattr_setsigdefault on macOS), and uv_loop_fork() is not needed before exec(). This is also what already happens on macOS when libuv uses posix_spawn(), since pthread_atfork handlers don't run there.

On macOS the before / after_in_parent hooks fired for every spawn; with pass_fds (which forces a real fork) after_in_child fired too.

test_process_at_fork_hooks runs a small script in a fresh interpreter (the hooks can't be unregistered). The script registers all three hooks and spawns a child with no extra arguments, with pass_fds, and with preexec_fn. The test checks that the hooks run only for preexec_fn, under both the uvloop and the asyncio test cases.

Testing (macOS arm64, CPython 3.12, built with --use-system-libuv against Homebrew libuv 1.53.0):

  • With uvloop/handles/process.pyx reverted to master, Test_UV_Process.test_process_at_fork_hooks fails: AssertionError: Lists differ: ['before', 'after_in_parent'] != []. Test_AIO_Process passes.
  • With the change, both pass.
  • The suite was run with python -m unittest discover tests: 529 tests OK (40 skipped). With PYTHONASYNCIODEBUG=1, the only failure is test_fs_event_rename, which fails the same way on master.
  • flake8 and mypy (tests/test_sourcecode.py) pass.

Not run: Linux, the free-threaded build, and the bundled-libuv build.

uvloop called PyOS_BeforeFork(), PyOS_AfterFork_Parent() and (in the
child) PyOS_AfterFork_Child() for every spawned process, so the
callbacks registered with os.register_at_fork() ran even though the
child never re-enters the interpreter before exec().  CPython's
subprocess (and so the default asyncio loop) only does this when a
preexec_fn is given, as documented for os.register_at_fork().

Do the same: install the child fork handler and call the PyOS_*Fork
functions only when preexec_fn is set, and only make the error pipe
inheritable in that case, since the child handler is what closes it.

Fixes MagicStack#749.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

uvloop runs at_fork hook when running subprocess

1 participant