Repository navigation
Conversation
If creating a symlink fails, e.g. on Windows without the privilege to create symlinks, the venv that is created at runtime uses hard links, junctions or copies instead, and its pyvenv.cfg sets home, because Python can't find it from the interpreter otherwise. The RULES_PYTHON_TESTING_DISABLE_SYMLINKS environment variable forces the fallback, so that it can be tested where symlinks work.
|
This PR was written by an AI agent (Claude Code) working for @aignas. Review comments are read and addressed by the agent, and every reply and change is reviewed and approved by @aignas before it is posted or pushed. The new Windows CI job is experimental: it checks whether This comment was generated by an AI agent and approved by @aignas. |
Revert this before the PR leaves draft.
This reverts commit 0a739bf.
On Windows, the venv paths are Windows paths, which the bash checks couldn't handle.
Use the `? "--flag"` keys that presubmit_test requires, and download the remote cache outputs, because without symlinks Bazel creates a junction in the runfiles tree for an output that isn't downloaded yet.
|
I am not sure if it is the smallest possible change to fix the issue, but it is pretty small and if it works, I'll be chuffed. |
The fallback for when symlinks can't be created is now Windows only. Windows refuses to delete a read-only file and the attribute is shared by all the hard links of a file, so removing the venv made the hard linked Bazel outputs writable. The hard links are now removed first, and the attribute is restored on their originals.
Instead of restoring the read-only attribute of hard linked Bazel outputs after clearing it, delete the read-only files of the venv with FILE_DISPOSITION_FLAG_IGNORE_READONLY_ATTRIBUTE, which doesn't change the other hard links of a file. It needs Windows 10 1809 or Windows Server 2019, so older versions still require symlinks.
rickeylev
left a comment
There was a problem hiding this comment.
Several lines in the PR description reach 73–76 columns, exceeding the
72-column limit for merge commit messages in .editorconfig.
Suggested fix: Re-wrap the PR description body at 72 columns while preserving
the existing wording.
This is an agent-generated comment reviewed and approved by the user.
| if is_dir: | ||
| _link_dir_windows(from_, to) | ||
| else: | ||
| _link_file_windows(from_, to) | ||
|
|
||
| def _link_file_windows(from_, to): | ||
| # Hard links don't need any privileges, but only work within the same | ||
| # volume, so fall back to copying the file. | ||
| try: | ||
| os.link(to, from_) | ||
| except OSError as e: | ||
| print_verbose("hard link failed, copying:", to, "error:", e) | ||
| shutil.copy2(to, from_) |
There was a problem hiding this comment.
On Windows, os.link(to, from_) calls CreateHardLinkW without dereferencing
symlinks, hard-linking the reparse point itself when to is a .runfiles
symlink. That leaves sys.executable as a symlink, failing
run_binary_without_symlinks_test on the Default: Windows CI jobs and
breaking relative symlinks.
Suggested fix: Resolve to with os.path.realpath(to) before dispatching,
and pass to directly to CreateJunction in _link_dir_windows:
if os.path.lexists(from_):
return
to = os.path.realpath(to)
if is_dir:
_link_dir_windows(from_, to)
else:
_link_file_windows(from_, to)Where: Line 676 in python/private/python_bootstrap_template.txt.
This is an agent-generated comment reviewed and approved by the user.
There was a problem hiding this comment.
when to is a runfiles symlink ... leaves sys.executable as a symlink
If symlinks aren't enabled then they wouldn't be a symlink. I think the comment is mistaken unless there's some edge case that I'm not seeing
| try: | ||
| # Junctions don't need any privileges, but need an absolute target. | ||
| import _winapi | ||
| _winapi.CreateJunction(abspath(to), from_) |
There was a problem hiding this comment.
os.path.islink() returns False for NTFS junctions on Windows, so a dangling
parent junction in a cached RULES_PYTHON_EXTRACT_ROOT venv is not unlinked
and os.makedirs(venv_parent, exist_ok=True) raises FileExistsError.
Suggested fix: Check
os.path.lexists(venv_parent) and not os.path.exists(venv_parent) since
os.unlink removes both symlinks and junctions on Windows, and add a test
case with _SYMLINKS_SUPPORTED = False:
if os.path.lexists(venv_parent) and not os.path.exists(venv_parent):
os.unlink(venv_parent)Where: tests/bootstrap_impls/cached_venv_dangling_symlink_test.py
This is an agent-generated comment reviewed and approved by the user.
There was a problem hiding this comment.
Something to verify is how lexists() behaves on a junction. There was a lot of subtle behaviors when the unix-style understanding of things operated on windows things.
| def _link_file_windows(from_, to): | ||
| # Hard links don't need any privileges, but only work within the same | ||
| # volume, so fall back to copying the file. | ||
| try: | ||
| os.link(to, from_) | ||
| except OSError as e: | ||
| print_verbose("hard link failed, copying:", to, "error:", e) | ||
| shutil.copy2(to, from_) | ||
|
|
||
| def _link_dir_windows(from_, to): | ||
| try: | ||
| # Junctions don't need any privileges, but need an absolute target. | ||
| import _winapi | ||
| _winapi.CreateJunction(abspath(to), from_) | ||
| except (ImportError, AttributeError, OSError) as e: | ||
| print_verbose("junction failed, copying:", to, "error:", e) | ||
| shutil.copytree(to, from_) | ||
|
|
There was a problem hiding this comment.
os.path.lexists(from_) is non-atomic when concurrent processes share
RULES_PYTHON_EXTRACT_ROOT. If another process creates from_ before os.link
or CreateJunction runs, FileExistsError hits except OSError and calls
shutil.copy2 or shutil.copytree onto the existing destination.
Suggested fix: Catch FileExistsError and return before except OSError in
both _link_file_windows and _link_dir_windows:
except FileExistsError:
returnThis is an agent-generated comment reviewed and approved by the user.
| read_only_before=$(find "$runfiles" -type f ! -writable | sort) | ||
| actual=$(RULES_PYTHON_TESTING_DISABLE_SYMLINKS=1 $bin) | ||
| expect_match "Hello" "$actual" | ||
| expect_match "sys.executable is symlink: False" "$actual" | ||
| venv=$(to_unix "$(echo "$actual" | grep "^sys.prefix:" | cut -d' ' -f2-)") | ||
| if [[ -z "$venv" || -e "$venv" ]]; then | ||
| echo "expected the venv to be removed: $venv" | ||
| touch EXPECTATION_FAILED | ||
| fi | ||
| read_only_after=$(find "$runfiles" -type f ! -writable | sort) | ||
| if [[ "$read_only_before" != "$read_only_after" ]]; then | ||
| echo "expected the read-only runfiles to be unchanged" |
There was a problem hiding this comment.
Windows python.exe writes CRLF line endings, leaving a trailing \r on $venv
so [[ -e "$venv" ]] is always false even when cleanup fails. Running find
without -L also skips symlinked runfiles, letting an empty read_only_before
pass vacuously.
Suggested fix: Strip \r from $actual, pass -L to find for both
read_only_before and read_only_after, and verify read_only_before is
non-empty:
# -L: follow symlinks when Bazel materializes runfiles as symlinks.
read_only_before=$(find -L "$runfiles" -type f ! -writable | sort)
if [[ -z "$read_only_before" ]]; then
echo "expected read-only files in runfiles: $runfiles"
touch EXPECTATION_FAILED
fi
actual=$(RULES_PYTHON_TESTING_DISABLE_SYMLINKS=1 "$bin" | tr --delete '\r')Where: Also strip \r at line 45 and pass -L at line 64 in
tests/bootstrap_impls/run_binary_without_symlinks_test.sh.
This is an agent-generated comment reviewed and approved by the user.
| venvs_root=$(mktemp -d) | ||
| actual=$(RULES_PYTHON_TESTING_DISABLE_SYMLINKS=1 RULES_PYTHON_EXTRACT_ROOT=$venvs_root $bin) |
There was a problem hiding this comment.
Bare mktemp -d creates venvs_root in MSYS /tmp outside TEST_TMPDIR, leaking
a populated venv on every run. Lines 13, 45, and 56 also leave
$BIN_RLOCATION, $venvs_root, and $bin unquoted.
Suggested fix: Create venvs_root inside TEST_TMPDIR and quote the path
variables:
bin=$(rlocation "$BIN_RLOCATION")
...
venvs_root=$(mktemp --directory "$(to_unix "$TEST_TMPDIR")/venvs_root.XXXXXX")
actual=$(RULES_PYTHON_TESTING_DISABLE_SYMLINKS=1 RULES_PYTHON_EXTRACT_ROOT="$venvs_root" "$bin")Where: Lines 13 and 56 in
tests/bootstrap_impls/run_binary_without_symlinks_test.sh.
This is an agent-generated comment reviewed and approved by the user.
| import ctypes | ||
| from ctypes import wintypes | ||
|
|
||
| kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) | ||
| kernel32.CreateFileW.restype = wintypes.HANDLE | ||
| kernel32.CreateFileW.argtypes = [ | ||
| wintypes.LPCWSTR, wintypes.DWORD, wintypes.DWORD, wintypes.LPVOID, | ||
| wintypes.DWORD, wintypes.DWORD, wintypes.HANDLE, | ||
| ] | ||
| kernel32.SetFileInformationByHandle.restype = wintypes.BOOL | ||
| kernel32.SetFileInformationByHandle.argtypes = [ | ||
| wintypes.HANDLE, ctypes.c_int, wintypes.LPVOID, wintypes.DWORD, | ||
| ] | ||
| kernel32.CloseHandle.argtypes = [wintypes.HANDLE] | ||
|
|
There was a problem hiding this comment.
_remove_read_only_file_windows constructs
ctypes.WinDLL("kernel32", use_last_error=True) and rebinds prototypes on
every read-only file removed by _rmtree, while omitting
CloseHandle.restype = wintypes.BOOL.
Suggested fix: Lazily initialize and cache the kernel32 WinDLL handle and
prototypes once in a _get_kernel32() helper, and set
kernel32.CloseHandle.restype = wintypes.BOOL.
This is an agent-generated comment reviewed and approved by the user.
| target_compatible_with = select({ | ||
| "@platforms//os:windows": [], | ||
| "//conditions:default": ["@platforms//:incompatible"], | ||
| }), |
There was a problem hiding this comment.
@platforms//os:windows is already a constraint_value, so wrapping
@platforms//:incompatible in select() isn't needed here.
Suggested fix: Pass the constraint list directly:
target_compatible_with = ["@platforms//os:windows"],This is an agent-generated comment reviewed and approved by the user.
| @@ -0,0 +1,72 @@ | |||
| # --- begin runfiles.bash initialization v3 --- | |||
There was a problem hiding this comment.
This script has executable mode 100755 and uses Bash-specific syntax
(pipefail, [[ ... ]], and process substitution), but is missing a Bash
shebang on line 1.
Suggested fix: Add #!/usr/bin/env bash at the top of the file:
#!/usr/bin/env bash
# --- begin runfiles.bash initialization v3 ---This is an agent-generated comment reviewed and approved by the user.
| exit 1 | ||
| fi | ||
|
|
||
| function expect_match() { |
There was a problem hiding this comment.
expect_match mutates global test state by creating EXPECTATION_FAILED on
mismatch without documenting that side effect.
Suggested fix: Add a function doc comment with a Modifies: section:
# Checks that actual output matches expected_pattern.
#
# Modifies:
# Creates EXPECTATION_FAILED in the working directory on mismatch.
function expect_match() {This is an agent-generated comment reviewed and approved by the user.
| # Converts a Windows path to a Unix one under MSYS, so that bash can use it. | ||
| function to_unix() { | ||
| if command -v cygpath >/dev/null; then | ||
| cygpath -u "$1" |
There was a problem hiding this comment.
cygpath, mktemp, and cut are called with short flags instead of long option
names.
Suggested fix: Use long option names:
cygpath --unix "$1"mktemp --directory ...cut --delimiter=' ' --fields=2-
Where: Also at lines 44 and 59 in
tests/bootstrap_impls/run_binary_without_symlinks_test.sh.
This is an agent-generated comment reviewed and approved by the user.
… globals Move them next to the other top-level globals and use a lower case name for the one that changes at runtime.
When Bazel could create the runfiles symlinks, but the venv falls back to hard links, os.link() hard linked the symlink itself, so sys.executable was still a symlink. Resolve the target first.
When processes share RULES_PYTHON_EXTRACT_ROOT, another one can create the entry between the existence check and creating the link, which made the fallback copy onto it. Also only catch OSError from _winapi.CreateJunction, which every supported Python on Windows has.
They were set up again for every read-only file that is removed. Also declare the return type of CloseHandle.
The fallback creates junctions for directories, and islink() is False for them, so a dangling one wasn't removed and makedirs() failed.
Remove the CR of the Windows output, so that the check that the venv was removed doesn't always pass, follow the runfiles symlinks when looking for read-only outputs and require that there are some, create the venvs under TEST_TMPDIR, and quote the paths.
…oved Check the behavior of lexists(), exists() and unlink() for a dangling junction, which the bootstrap relies on, and run the cached venv test with the fallback for when symlinks can't be created.
…ed venv test Windows can't delete a hard link to a running executable, so cleaning up the temporary directory failed after the fallback hard linked sys.executable into the venv. The venv isn't run, so link to a stand-in file instead.
On Windows,
py_binaryandpy_testcreate a venv at runtime bysymlinking the interpreter, its
.dllfiles and thesite-packagesfilesinto a temporary directory. Without the privilege to create symlinks, this
fails with
WinError 1314, which is the default for users that can't enableDeveloper Mode.
On Windows, if creating a symlink fails, the bootstrap now uses hard links
for files, junctions for directories, and copies if those aren't possible,
e.g. when the temporary directory is on a different volume. The runfiles
files are already copies when Bazel can't create symlinks, so they aren't
moved, as that would break the next run that uses the same runfiles tree.
Other platforms are unchanged.
Windows refuses to delete read-only files, such as hard links to Bazel
outputs, and the read-only attribute is shared by all the hard links of a
file, so the venv's read-only files are deleted with
FILE_DISPOSITION_FLAG_IGNORE_READONLY_ATTRIBUTEinstead of clearing theattribute. This needs Windows 10 1809 or Windows Server 2019, so older
versions still require the privilege to create symlinks.
The
RULES_PYTHON_TESTING_DISABLE_SYMLINKSenvironment variable forces thefallback on Windows, which a new test uses, and a new CI job uses to build
and test
examples/bzlmodwith--nowindows_enable_symlinks.Fixes #3792