Skip to content

Keep the ruby crawl test off the host's Bundler env - #2056

Open
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
mainfrom
ci-janitor/ruby-crawler-test-hermeticity
Open

Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
mainfrom
ci-janitor/ruby-crawler-test-hermeticity

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

CI is healthy this run. The merge queue is empty, the last 20 merge_group runs passed, main's push runs are green, last night's scheduled CI passed (first green nightly since 2026-10-21, after #2049/#2050), and nothing is queued. Every PR failure in the last week traces to a real code change or to a release-504 class already fixed by #2039–#2042/#2046. So this run takes a priority-4 robustness target: the last unclaimed item from #1919 (unit tests that pass only because CI's environment is clean). #2054 (npm/yarn) and #2055 (python) cover the other items; this PR touches neither of their files.

ruby_crawler::tests::crawl_all_finds_flat_bundler1_project calls the public crawl_all, which reads BUNDLE_GEMFILE, BUNDLE_PATH__SYSTEM, BUNDLE_APP_CONFIG, and ~/.bundle/config from the process. Reproduced on origin/main (c09cb0e) with cargo test -p socket-patch-core --lib crawlers:::

env before after
clean 798 passed 798 passed
BUNDLE_PATH__SYSTEM=true 1 failed 798 passed
BUNDLE_GEMFILE=<other dir>/Gemfile 1 failed 798 passed
BUNDLE_APP_CONFIG=<dir with path.system config> 1 failed 798 passed
HOME with .bundle/config → BUNDLE_PATH__SYSTEM: "true" 1 failed 798 passed
BUNDLE_PATH, BUNDLE_DEPLOYMENT, BUNDLE_DISABLE_SHARED_GEMS, GEM_HOME, GEM_PATH, BUNDLE_IGNORE_CONFIG, BUNDLE_LOCKFILE ok ok

The failing test was the same one each time.

Root cause

The test exercised the ambient-env entry point instead of an injected-env seam. The test's doc only defended against BUNDLE_PATH.

Fix

  • I moved the body of crawl_all_with_discovery into a private crawl_all_with_env that takes the Bundler environment as arguments. This is the same argument list gem_paths_and_discovery and discover_bundle_stores_impl already take. crawl_all_with_discovery passes the ambient values exactly as before, so production behaviour doesn't change.
  • The test now calls crawl_all_with_env with an empty environment. It runs the same scan loop on the same fixture and makes the same assertion.
  • I injected the env instead of using EnvGuard plus #[serial], because mutating process env while other tests run in parallel is itself a flake source.
  • The public crawl_all / crawl_all_with_discovery stay covered by the other crawl_all unit tests in the module and by ecosystem_dispatch.

Proof

  • See the table above. All 6 hostile-env runs and the clean run pass after the change.
  • cargo clippy --locked -p socket-patch-core --all-features -- -D warnings (CI's clippy command, scoped to the crate) is clean.
  • rustfmt --check passes on the touched file.
  • No tests were removed or weakened.

Where tests run

Unchanged. The test still runs in the PR, merge-queue, and nightly unit test shards.


Generated by Claude Code


Generated by Claude Code

crawl_all_finds_flat_bundler1_project went through the public
crawl_all, which reads BUNDLE_GEMFILE, BUNDLE_PATH__SYSTEM,
BUNDLE_APP_CONFIG and ~/.bundle/config from the process. Any of those
set on a developer machine moved the install root away from
vendor/bundle and failed the test on a clean checkout (#1919).

Split the scan out of crawl_all_with_discovery into a private
crawl_all_with_env that takes the Bundler environment explicitly.
Production still passes the ambient values, so behaviour is unchanged;
the test now passes an empty environment.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011f9aePh5tehaLQ8MtWhUP5
@mikolalysenko Mikola Lysenko (mikolalysenko) added the ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) label Nov 11, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor

cursor Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Internal refactor plus test-only env injection; public crawl entry points still use the same ambient Bundler settings.

Overview
Fixes flakiness in crawl_all_finds_flat_bundler1_project by crawling with an injected, empty Bundler environment instead of reading the process’s BUNDLE_* vars and ~/.bundle/config.

The crawl loop is moved into a private crawl_all_with_env that takes the same Bundler inputs as gem_paths_and_discovery. crawl_all_with_discovery (and therefore crawl_all) still forwards the ambient environment, so runtime behavior is unchanged.

Reviewed by Cursor Bugbot for commit 13b445e. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 13b445e. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Nov 11, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review at 13b445e.

  • CI: 50/50 checks done, none failed (31 success incl. ci-ok and clippy, 19 skipped by path filters). No main-wide failures.
  • Bugbot: reviewed 13b445e, no findings.
  • Look at: crawl_all_with_env in ruby_crawler.rs — crawl_all_with_discovery should pass the ambient Bundler env exactly as before, so production behaviour is unchanged.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does: One Ruby unit test (crawl_all_finds_flat_bundler1_project) failed on a developer machine with BUNDLE_PATH__SYSTEM, BUNDLE_GEMFILE, BUNDLE_APP_CONFIG or a ~/.bundle/config set. The crawl body moves into a private crawl_all_with_env that takes the Bundler environment as arguments, and the test passes it an empty one. crawl_all_with_discovery still passes the same ambient values as before, so production behavior doesn't change.

Risk: low. It is a pure extract-function refactor with one test caller. The argument list is the same one gem_paths_and_discovery already takes, in the same order. There are no new dependencies.

Look here

  • ruby_crawler.rs:161: the public entry point now forwards the ambient env. Check that the arguments match the old call.
  • ruby_crawler.rs:183: the new crawl_all_with_env, which holds the old body unchanged.
  • ruby_crawler.rs:3390: the test now injects an empty env. The assertion is unchanged.

Verified: Locally, BUNDLE_PATH__SYSTEM=true BUNDLE_GEMFILE=/nonexistent/Gemfile cargo test -p socket-patch-core --lib -- ruby_crawler gives 108 passed. I also read the full diff (1 file, +49/−7) and compared the argument order against gem_paths_and_discovery. The only env reads left on this path go through the gem env fallback, which can only add paths and can't move vendor/bundle, so the find assertion still holds. CHANGELOG.md is untouched. CI: ci-ok and clippy are green, Bugbot is clean on 13b445e8, and there are no review threads.

Changes I made: none.

Open questions: none.

Auto-merge is armed, so approving sends this straight to the merge queue.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants