perf(pm): execute resolver manifest provider jobs#3039
Conversation
There was a problem hiding this comment.
Code Review
This pull request implements the ManifestProvider trait for UnifiedRegistry, enabling it to execute manifest-related jobs such as fetching full manifests, specific versions, or extracting versions from a full manifest. It introduces the ProviderFullManifestBytes enum to handle raw manifest data and adds a registry_url method to the RegistryClient trait. Feedback was provided to optimize the store_version_manifest helper by using a deduplication set to prevent redundant writes to the persistent store when multiple jobs resolve to the same version.
| fn store_version_manifest(&self, name: &str, manifest: Arc<CoreVersionManifest>) { | ||
| let version = manifest.version.clone(); | ||
| self.store.store_version_manifest(name, &version, manifest); | ||
| } |
There was a problem hiding this comment.
The store_version_manifest helper should utilize the stored_version dedup set to prevent redundant writes to the persistent store. This ensures that if multiple jobs resolve to the same version (e.g., through different specs), the manifest is only written once, which is consistent with the de-duplication logic used in resolve_version_manifest.
fn store_version_manifest(&self, name: &str, manifest: Arc<CoreVersionManifest>) {
let version = &manifest.version;
if self.stored_version.insert((name.to_string(), version.clone())) {
self.store.store_version_manifest(name, version, manifest);
}
}70806c8 to
d1db0d2
Compare
fada6f8 to
3dcc5f7
Compare
d1db0d2 to
94579a5
Compare
3dcc5f7 to
3c19175
Compare
3c19175 to
8b32ee5
Compare
94579a5 to
56101d8
Compare
📊 pm-bench-phases ·
|
| PM | wall | ±σ | user | sys | RSS | pgMinor |
|---|---|---|---|---|---|---|
| bun | 9.02s | 0.22s | 10.32s | 9.81s | 722M | 330.1K |
| utoo-next | 7.70s | 0.44s | 10.19s | 11.74s | 1004M | 119.7K |
| utoo-npm | 7.68s | 0.12s | 10.49s | 12.04s | 972M | 130.5K |
| utoo | 7.94s | 0.17s | 10.57s | 11.78s | 988M | 123.4K |
| PM | vCtx | iCtx | netRX | netTX | cache | node_mod | lock |
|---|---|---|---|---|---|---|---|
| bun | 15.4K | 17.9K | 1.19G | 6M | 1.86G | 1.75G | 1M |
| utoo-next | 113.1K | 82.9K | 1.16G | 5M | 1.71G | 1.70G | 2M |
| utoo-npm | 131.2K | 97.1K | 1.16G | 5M | 1.71G | 1.70G | 2M |
| utoo | 122.6K | 69.0K | 1.17G | 6M | 1.71G | 1.70G | 2M |
p1_resolve
| PM | wall | ±σ | user | sys | RSS | pgMinor |
|---|---|---|---|---|---|---|
| bun | 1.92s | 0.01s | 3.98s | 1.03s | 520M | 161.4K |
| utoo-next | 2.80s | 0.05s | 4.92s | 1.79s | 616M | 91.5K |
| utoo-npm | 2.98s | 0.01s | 5.09s | 2.10s | 616M | 90.0K |
| utoo | 3.14s | 0.06s | 5.53s | 1.92s | 625M | 90.8K |
| PM | vCtx | iCtx | netRX | netTX | cache | node_mod | lock |
|---|---|---|---|---|---|---|---|
| bun | 8.5K | 4.6K | 203M | 3M | 108M | - | 1M |
| utoo-next | 48.1K | 67.5K | 200M | 2M | 7M | 3M | 2M |
| utoo-npm | 71.7K | 88.8K | 200M | 2M | 7M | 3M | 2M |
| utoo | 54.6K | 76.1K | 202M | 3M | 7M | 3M | 2M |
p3_cold_install
| PM | wall | ±σ | user | sys | RSS | pgMinor |
|---|---|---|---|---|---|---|
| bun | 6.54s | 0.27s | 6.29s | 9.47s | 615M | 203.6K |
| utoo-next | 5.97s | 0.23s | 4.93s | 10.31s | 461M | 60.5K |
| utoo-npm | 6.89s | 1.87s | 5.14s | 10.50s | 464M | 61.4K |
| utoo | 7.55s | 3.26s | 4.96s | 10.43s | 460M | 65.7K |
| PM | vCtx | iCtx | netRX | netTX | cache | node_mod | lock |
|---|---|---|---|---|---|---|---|
| bun | 4.9K | 6.7K | 1019M | 4M | 1.76G | 1.76G | 1M |
| utoo-next | 101.0K | 50.7K | 988M | 3M | 1.70G | 1.70G | 2M |
| utoo-npm | 114.0K | 51.0K | 988M | 3M | 1.70G | 1.70G | 2M |
| utoo | 114.0K | 51.9K | 988M | 3M | 1.70G | 1.70G | 2M |
p4_warm_link
| PM | wall | ±σ | user | sys | RSS | pgMinor |
|---|---|---|---|---|---|---|
| bun | 3.43s | 0.02s | 0.20s | 2.42s | 135M | 32.2K |
| utoo-next | 2.23s | 0.12s | 0.47s | 3.69s | 79M | 18.1K |
| utoo-npm | 2.38s | 0.02s | 0.48s | 3.67s | 79M | 18.5K |
| utoo | 2.19s | 0.01s | 0.45s | 3.73s | 79M | 18.1K |
| PM | vCtx | iCtx | netRX | netTX | cache | node_mod | lock |
|---|---|---|---|---|---|---|---|
| bun | 221 | 26 | 5M | 30K | 1.91G | 1.75G | 1M |
| utoo-next | 41.7K | 19.4K | 26K | 6K | 1.70G | 1.70G | 2M |
| utoo-npm | 38.3K | 17.3K | 29K | 12K | 1.70G | 1.70G | 2M |
| utoo | 39.4K | 17.9K | 26K | 24K | 1.71G | 1.70G | 2M |
npmmirror.com: no output captured.
Summary
ManifestProviderforUnifiedRegistryas a manifest job executorRegistryClientfor follow-up scheduler workValidation
cargo fmtcargo check -p utoo-ruboristcargo test -p utoo-ruborist service::registry::tests::test_unified_registry_executes_extract_manifest_provider_job -- --nocapturecargo clippy --all-targets -- -D warnings --no-depsSplit Plan
Part of the resolver stack split from source PR #3028. This PR prepares the registry/provider execution boundary; the next resolver PR can move the BFS loop to
ManifestProviderjobs without mixing registry I/O changes into the builder rewrite.