Skip to content

Roll back a vendored PDM 0.x dependency declared with extras - #2060

Open
Mikola Lysenko (mikolalysenko) wants to merge 2 commits into
mainfrom
fix/pdm-extras-vendor-revert
Open

Mikola Lysenko (mikolalysenko) wants to merge 2 commits into
mainfrom
fix/pdm-extras-vendor-revert

Conversation

@mikolalysenko

Copy link
Copy Markdown
Collaborator

What was wrong

In 5.1.0, rollback refuses to undo socket-patch's own vendoring of a PyPI package in a PDM 0.x project that declares it with extras, for example urllib3 = {version = "==1.26.18", extras = ["socks"]}:

refusing to revert pkg:pypi/urllib3@1.26.18: for pdm.lock, the ledger record for it would change
entries other than urllib3's own, which is not an edit socket-patch makes. Nothing was changed for
this package; restore .socket/vendor/state.json (or pdm.lock, if that is the damaged one) from
version control, or undo the vendoring by hand

Nothing is written, so no file is damaged, but the project stays vendored until its lock is restored from version control. scan --mode vendored on the same project succeeds, and the hosted flow is not affected.

Cause

PDM's lock_version = "2" format keeps file hashes in [metadata.files], keyed <name> <version>. A unit with extras has a key of its own, <name>[<extras>] <version>, and vendoring rewrites both:

"urllib3 1.26.18" = [...]
"urllib3[socks] 1.26.18" = [...]

The revert's confinement check, added in #2052, compares the lock before and after the restore with the package's own entries taken out of both. It read a key's package as its first word, so urllib3[socks] counted as another package's entry, the two sides differed, and the restore was refused.

Fix

  • pdm_lock::files_key_package reads a [metadata.files] key back to its package, dropping the [<extras>] suffix. It sits next to legacy_files_key, which builds the key, so the writer and the check share one definition.
  • drop_package uses it for files / hashes keys only. A sources key is read as before, and another package's extras key is still refused.

Testing

  • revert_round_trip_restores_a_legacy_lock_with_extras vendors and reverts urllib3 in the native PDM 0.12.3 extras lock through wire_pdm and revert_pdm, the path that failed. Without the fix it fails with the refusal above.
  • confinement_allows_a_pdm_revert_in_every_native_format runs the PDM lock writer over every native lock the rewriter takes and holds the check to accepting the revert. Without the fix it fails on 0.12.3-extras.
  • files_key_package_reads_back_every_legacy_key checks that every key the writer builds for the PDM 0.x fixtures reads back as its unit's package.
  • cargo test -p socket-patch-core --lib -- vendor::pypi utils::pdm_lock utils::poetry_lock vendor::common: 595 pass.
  • cargo clippy --locked --workspace --all-features -- -D warnings and cargo fmt --check are clean.
  • End to end, with depscan's PDM patch backtest (real PDM 0.12.3, the real urllib3 patch, scan --mode vendored, install, then rollback), shapes extras and direct: the released 5.1.0 binary fails 0.12.3 extras vendored, and a build of this branch passes all four cells.

Review findings not fixed here

/code-review high found no bug in the change. Two findings are left out:

  • A hole that predates this change, in the same function. drop_package decides that a table is keyed by package from its parent key alone, so it also treats the inline {file, hash} tables of a files = [...] array, and a pylock hashes = { sha256 = … } table, as keyed by package. When the vendored package is itself named file, hash, url or sha256, those keys are taken out of every package's entries on both sides, and a tampered ledger record that changes another package's hash would pass the check. It needs a patch for a package with one of those names and a tampered committed ledger. Closing it changes what the check accepts, so it should be its own PR with its own tests; I read the code path but did not write a failing test.
  • Reusing pep508_name instead of a new helper. It would also accept six@x or six/dir as six's key. The new helper accepts only what legacy_files_key writes.

How it was found

depscan's PDM patch backtest failed 0.12.3 extras vendored on Linux and Windows when the submodule moved to v5.1.0 (SocketDev/depscan#30174). That bump needs this fix before its CI can pass.

🤖 Generated with Claude Code

`rollback` refused to undo socket-patch's own vendoring of a PyPI
package in a PDM 0.x project that declares it with extras
(`urllib3 = {version = "==1.26.18", extras = ["socks"]}`):

    refusing to revert pkg:pypi/urllib3@1.26.18: for pdm.lock, the
    ledger record for it would change entries other than urllib3's
    own, which is not an edit socket-patch makes

PDM's lock_version 2 format keeps file hashes in `[metadata.files]`,
keyed `<name> <version>`, and a unit with extras has a key of its own,
`<name>[<extras>] <version>`. Vendoring rewrites both. The revert's
confinement check, new in 5.1.0 (#2052), read a key's package as its
first word, so `urllib3[socks]` counted as another package's entry
and the restore was refused. Nothing was written: the project stayed
vendored until its lock was restored from version control.

The check now drops the `[<extras>]` suffix before it compares names.
Another package's extras key is still refused.

Test: confinement_allows_a_pdm_revert_in_every_native_format runs the
PDM lock writer over every native lock fixture and holds the check to
accepting the revert. It fails on 0.12.3-extras without this change.

Found by depscan's PDM patch backtest on the 5.1.0 bump (PDM 0.12.3,
extras, vendored). The same cell passes with this build.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Review follow-up for the extras rollback fix.

The check's reading of a `[metadata.files]` key is now
`pdm_lock::files_key_package`, next to `legacy_files_key`, which builds
the key, with a test that every key the writer builds for the PDM 0.x
fixtures reads back as its unit's package. The two can no longer drift
apart unnoticed, which is how the 5.1.0 refusal shipped.

The `[<extras>]` suffix is dropped only for `files` / `hashes` keys.
A `sources` key is read as before, so `"six[socks]"` under
`[tool.uv.sources]` is not six's own entry.

Tests: revert_round_trip_restores_a_legacy_lock_with_extras vendors
and reverts urllib3 in the native PDM 0.12.3 extras lock through
wire_pdm and revert_pdm, the path that failed. Without the fix it
fails with the refusal users saw. The confinement sweep takes its
fixture list from pdm_lock's tests instead of a copy.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@cursor

cursor Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches revert confinement logic for lock file keys; scope is narrow (PDM legacy files/hashes parsing) but incorrect parsing could weaken tamper detection for metadata tables.

Overview
Fixes rollback refusing to undo vendoring in PDM 0.x projects when a dependency is declared with extras (e.g. urllib3[socks]). Vendoring rewrites both the base and extras-specific [metadata.files] keys (urllib3 1.26.18 and urllib3[socks] 1.26.18); the revert confinement check previously treated the extras key as another package because it only used the first whitespace-delimited token.

Adds files_key_package next to legacy_files_key so keys round-trip to the package name by stripping [extras]. drop_package in the PyPI lock vendor path now uses that helper for files / hashes keys ( sources keys are unchanged).

Adds unit and integration tests: key parsing over native fixtures, confinement accepting reverts across all rewritable PDM lock formats, and a revert_pdm round-trip on the 0.12.3-extras fixture.

Reviewed by Cursor Bugbot for commit 0baea26. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Nov 11, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does: rollback refused to undo socket-patch's own vendoring of a PyPI dependency declared with extras in a PDM 0.x (lock_version = "2") project. The extras unit's [metadata.files] key urllib3[socks] 1.26.18 was read as another package's entry by the revert confinement check. A new files_key_package helper reads the key back to its package, and drop_package uses it for files / hashes keys only.

Risk: low. The change is confined to how the confinement check names files/hashes keys. sources keys are parsed as before, and another package's extras key is still refused (tested). No writer behavior changes.

Look here:

  • pdm_lock.rs:120: files_key_package, next to legacy_files_key, which writes the key
  • [pypi_lock.rs:1221-1226](https://github.com/SocketDev/socket-patch/blob/0baea26923fca499443cf0325615f500bc36da67/crates/socket-patch-core/src/vendor/pypi_lock.rs#L1221-L1226): the per-parent key matching in drop_package
  • pypi_lock.rs:3687: confinement accepts the revert for every native PDM format; another package's extras key and a sources key are still refused
  • pypi_pdm.rs:1336: end-to-end wire_pdm → revert_pdm on the 0.12.3-extras lock, checked for a byte-identical result

Verified: I read the full diff. CHANGELOG.md is untouched. Locally, the 4 new or changed tests pass, and cargo clippy --locked -p socket-patch-core --all-features -D warnings is clean. The only failures in the wider vendor::pypi test filter are 2 permission-denial tests that fail when run as root, a pre-existing issue that #2058 fixes. CI: 31/31 checks passed (19 skipped). Cursor Bugbot passed with no findings. The PR description matches the code. It also lists a pre-existing hole in drop_package (packages named file/hash/url/sha256) that is out of scope and deserves its own PR.

Changes I made: none.

Open questions: none blocking. Consider a follow-up issue for the pre-existing drop_package hole described in the PR body.

Auto-merge is armed, so approving sends it straight to the merge queue.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant