Skip to content

feat: support env from a file in bundle_fetch (env_file) - #433

Open
acecilia wants to merge 1 commit into
bazel-contrib:mainfrom
acecilia:feat/bundle-fetch-env-files
Open

acecilia wants to merge 1 commit into
bazel-contrib:mainfrom
acecilia:feat/bundle-fetch-env-files

Conversation

@acecilia

@acecilia acecilia commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem

rb_bundle_install compiles native gem extensions (json, prism, ffi, ...) against arbitrary host system components. On macOS mkmf resolves clang and the SDK through xcrun, but other bundles need CC, PKG_CONFIG_PATH, CMAKE, ... — the set of possible host dependencies is unbounded.

Consumers that need to control these values today face two dead ends:

  • The env tag attribute only accepts static strings.
  • Delegating resolution to the host's ambient environment is non-hermetic and invisible to the action cache: a toolchain change does not invalidate cached compiled extensions.

Solution

Optional env_file attribute on the bundle_fetch tag: a label pointing to a file with KEY=VALUE lines. The repository rule reads the file at evaluation time on the building machine and bakes the resolved values into the generated env as a plain static dict — the generated BUILD stays fully static, no load() of consumer starlark, no symbol conventions.

# MODULE.bazel
ruby.bundle_fetch(
    name = "ruby_bundle",
    env_file = "@ruby_build_env//:env",
    ...
)

The referenced file is produced by a single dedicated consumer repository rule that composes the whole installation environment (here one resolving the Xcode pinned by the project):

# Bazel/repository_rules/ruby_build_env.bzl
def _ruby_build_env_impl(ctx):
    result = ctx.execute(["zsh", "-c", str(ctx.path(ctx.attr._print_developer_dir_script))])
    ctx.file("env", "DEVELOPER_DIR=%s\n" % result.stdout)  # e.g. DEVELOPER_DIR=/Applications/Xcode-27.1.0.app
    ctx.file("BUILD.bazel", 'exports_files(["env"])')

Because the file is a label input to the repository rule:

  • values are resolved inside the build graph (no ambient env, no dependence on how bazel was invoked),
  • content changes re-run the repository, and the new static values change the rb_bundle_install action key, so compiled native extensions are correctly invalidated (Xcode bump → recompile),
  • per-machine values keep remote caches from sharing entries across differently-provisioned hosts.

The mechanism is provider-agnostic: the consumer rule decides what the environment contains (DEVELOPER_DIR today, PKG_CONFIG_PATH or a pinned CC tomorrow), with rules_ruby baking in no system knowledge.

Alternatives considered

  • repository_rule(environ = [...]): mirrors Bazel's native mechanism, but the variable list is a static per-rule declaration — a generic ruleset would need a hardcoded variable vocabulary, and values would depend on the ambient environment of the invocation.

Testing

examples/gem exercises env_file end-to-end: env_file_check_test asserts the value lands in the generated bundle repository (verified locally: passes with the correct value, fails with a tampered one), and the executed bundle_install_*.sh contains export RULES_RUBY_ENV_FILE_TEST=exercised. Also validated in a large consumer repo: DEVELOPER_DIR reaches the install script via a use_repo_rule repository, and changing the file content re-runs the bundle install action.

@acecilia
acecilia marked this pull request as draft October 27, 2026 21:06
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_files` mapping of environment variable name to a file
whose content is the value for that variable. The repository rule reads the
files at evaluation time on the building machine and bakes the resolved
values into the generated `env` as a plain static dict. Because the files are
label inputs, content changes (e.g. a machine-resolved toolchain path
changing) re-run the repository and change the install action key, keeping
compiled native extensions correctly invalidated. Values from `env` take
precedence over `env_files`.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from c9060d6 to 2a955db Compare October 27, 2026 21:07
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from 2a955db to 0e92745 Compare October 27, 2026 21:11
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from 0e92745 to 0e2526a Compare October 27, 2026 21:14
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from 0e2526a to 42eb327 Compare October 27, 2026 21:16
@acecilia acecilia changed the title feat: support env values from files in bundle_fetch (env_files) feat: support env from a file in bundle_fetch (env_file) Oct 27, 2026
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from 42eb327 to a94bac1 Compare October 27, 2026 21:18
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from a94bac1 to 10c3064 Compare October 27, 2026 21:20
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from 10c3064 to b116aea Compare October 27, 2026 21:22
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from b116aea to 5db79af Compare October 27, 2026 21:27
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch 2 times, most recently from 435ed2d to 5db79af Compare October 27, 2026 21:35
@acecilia
acecilia marked this pull request as ready for review October 27, 2026 21:44
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 27, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from 5db79af to a9c8739 Compare October 27, 2026 22:12
acecilia added a commit to acecilia/rules_ruby that referenced this pull request Oct 28, 2026
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from a9c8739 to 6612528 Compare October 28, 2026 13:00
Native gem extensions compile against arbitrary host system components,
which rules_ruby must not enumerate (Xcode toolchains, pkg-config paths,
compilers, ...). The static `env` attribute cannot express machine-resolved
values, and `load()` is not allowed in MODULE.bazel, so values resolved by
generated repositories cannot reach the generated rb_bundle_install action.

Add an optional `env_file` label pointing to a file with KEY=VALUE lines.
The repository rule reads it at evaluation time on the building machine and
bakes the resolved values into the generated `env` as a plain static dict,
so a consumer can compose the whole installation environment in a single
dedicated repository rule. Because the file is a label input, content
changes (e.g. a machine-resolved toolchain path changing) re-run the
repository and change the install action key, keeping compiled native
extensions correctly invalidated. Values from the static `env` take
precedence for the same variable.

Upstream: bazel-contrib#433
@acecilia
acecilia force-pushed the feat/bundle-fetch-env-files branch from 6612528 to 0e53d06 Compare October 29, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant