Experiment with bundled wheel metadata

[charliermarsh]

Summary

Backtracking-heavy warm resolutions can open thousands of tiny wheel-METADATA cache files. In the boto3 case, boto3/botocore alone require 1,903 files and 1.42 MB, of which roughly 1.1 MB is repeated HTTP-cache-policy trailers. Warm-page-cache measurements put opening/reading those files around 67 ms, versus roughly 1-2 ms for a sequential package-sized read.

This experiments with an additive, offline-only per-package metadata bundle. The first request loads a bundle through an Fx-hashed concurrent once-map; later requests reuse the buffer and an offset index. Wheel keys, exact request URLs, and decoded name/version are checked, and any bundle/key/URL/decode miss falls back to the existing per-wheel cache. Online freshness behavior is unchanged. A small benchmark packer is included so the cache can be seeded reproducibly.

PEP 658 metadata is per distribution, not per version, so this intentionally keeps one record per wheel rather than hydrating the version-level Simple metadata. The boto3/botocore bundles shrink the two directories from 1,424,072 B across 1,903 files to 725,861 B across two files.

Performance

Measured incrementally on the stale-cache and one-pass-prefetch changes, using profiling binaries and a prewarmed PyPI cache with every cached package bundled. All runs were offline uv pip compile, Python 3.12, fixed UV_EXCLUDE_NEWER=2024-08-08T00:00:00Z, UV_CONCURRENT_CACHE_READS=4, CPUs 8-15, 10 warmups, and 40 alternating pairs. Every generated requirements file matched byte-for-byte.

workload	wall before → after	wall change (95% CI)	CPU before → after	CPU change (95% CI)
jupyter (97 packages)	98.1 → 103.1 ms	+4.1% [-0.9, +9.4]	53.2 → 60.3 ms	+12.8% [+11.3, +14.4]
boto3 (7 packages, backtracking)	158.2 → 139.3 ms	-10.0% [-12.7, -7.2]	213.0 → 102.2 ms	-53.0% [-54.0, -52.0]
airflow (559 packages)	233.3 → 223.1 ms	-3.8% [-7.0, -0.8]	352.1 → 314.0 ms	-11.2% [-12.3, -10.1]
large-index (26 large Simple pages, no deps)	102.5 → 103.8 ms	+10.8% [+1.5, +21.4]	57.6 → 57.5 ms	-0.7% [-2.1, +0.7]

An 80-pair comparison of the combined changes against latest main independently reproduced the large effects: boto3 -11.2% wall/-55.0% CPU and Airflow -5.5% wall/-14.8% CPU.

This is explicitly experimental and not ready to merge: singleton-heavy Jupyter regresses, and a missing bundle is worse still because its first lookup adds an extra open. A production version needs either a bundle-presence manifest or resolver-driven threshold (for example, only after repeated attempts/batch prefetch), plus an atomic package-scoped writer and an artifact/metadata-hash guard for refreshed mutable indexes. The current packer/reader demonstrates the ceiling without changing online cache semantics.

The full uv-client unit suite (85 tests), strict clippy including tests, packer syntax check, formatting, and diff checks pass.