env probe: recover Buck-target torch native libs + capture triton/fbgemm commits (schema 1.8)

[vivekkhandelwal1]

Summary

Two recoverability gaps surfaced while confirming aorta env probe against a Buck torch target (fbcode//caffe2:torch), where import torch succeeds in-process but the Python package is a link-tree and the C++ runtime is dlopen'd from a build-artifact dir:

• Buck/monorepo native-lib recovery. libtorch_hip.so was located only at <torch.__file__>/../lib, which does not exist in a Buck par layout, so composable_kernel.pytorch_bundled, aotriton.*, and pytorch_build.binary_introspection.* came back null. Added a /proc/self/maps fallback (_loaded_lib_path_from_maps / _torch_native_lib_dir) that recovers the real lib directory from the process's mapped libraries (torch is already imported). The original on-disk path and reason strings are preserved as the last-resort fallback, so existing "missing lib" diagnostics are unchanged when nothing is locatable. Linux-only, fail-soft.
• Package commit capture. Generalized aiter's +g<sha> parse into _extract_commit_from_version / _capture_python_package_commit and added a commit field to the triton and fbgemm blocks. Best-effort: setuptools_scm +g<sha>, ROCm fork .git<sha>, or a git_version/__commit__ module attr; null when no SHA (e.g. "3.5.0+fb").

Schema 1.7 → 1.8, additive nested keys only — EnvSnapshot.from_dict still round-trips pre-1.8 snapshots. Updates docs/env-probe.md.

Test plan

• tests/instrumentation/test_environment.py — added TestPackageCommitExtraction and TestTorchNativeLibDir (incl. a test proving the symbol dump recovers via /proc/self/maps for a Buck-style torch with no <torch>/lib); updated the 3 existing triton/fbgemm assertions + schema-version pin.
• env + probe e2e + recipe suites pass (one unrelated pre-existing failure: test_partial_false_on_clean_full_probe, which requires triton installed in the venv and fails identically on main).
• Live validation against a real ROCm libtorch_hip.so on a gfx950 Buck torch target (unit-tested with a fake mapped lib; not yet exercised against a live HIP build).

Made with Cursor

[Copilot]

Pull request overview

Improves aorta env probe recoverability and metadata capture when running against Buck-target PyTorch builds by (1) finding torch native libraries via /proc/self/maps when <torch>/lib is absent, and (2) capturing best-effort git commits for triton and fbgemm_gpu into the env snapshot (schema bump 1.7 → 1.8).

Changes:

• Bump env snapshot schema version to 1.8 and add nested commit fields under triton and fbgemm.
• Add /proc/self/maps fallback to locate torch native shared libraries for Buck/monorepo layouts.
• Update unit tests and documentation to reflect the new schema and recovery behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
src/aorta/instrumentation/environment.py	Adds commit extraction helpers, adds triton.commit/fbgemm.commit, and introduces /proc/self/maps fallback for torch native lib discovery.
tests/instrumentation/test_environment.py	Updates schema pin and assertions; adds new tests for commit parsing and Buck torch native-lib recovery.
docs/env-probe.md	Documents schema 1.8 additions and the Buck/monorepo native-lib recovery behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[vivekkhandelwal1]

Live-validated on a gfx950 node against a real ROCm libtorch_hip.so (torch 2.9.1+rocm6.4, torch.version.hip=6.4.43484-..., 371 MB lib):

• maps locator found the loaded libtorch_hip.so from /proc/self/maps.
• Primary path (real <torch>/lib): symbol counts populated (pytorch_flash::=4, _efficient_attention=18, aotriton::=3, mha_fwd_aot=1), torch_lib_bundled={libaotriton_v2.so: true}, aotriton bundled_version=0.11.0 with real hash — confirms the refactor doesn't regress the normal layout.
• Fallback path (bogus <torch>/lib, lib still mapped — i.e. the Buck-target scenario): _torch_native_lib_dir recovered the real lib dir via /proc/self/maps, the nm | c++filt dump processed 30,482 demangled symbols, no "not found" reasons, and symbol counts were identical to the primary path.
• triton commit: this ROCm wheel reports package_version=3.5.1 with no SHA → commit: null (correctly no false-positive).

So the previously-null composable_kernel.pytorch_bundled / aotriton.* / pytorch_build.binary_introspection.* fields now populate for a Buck torch target. Checking the live-validation box.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[amd-vivekag]

Review summary

Verdict: ship (two Low nits, nothing blocking)

Clean, well-scoped change. The two mechanisms — /proc/self/maps recovery of torch's native lib dir for Buck layouts, and best-effort commit extraction for triton/fbgemm — are both fail-soft, Linux-guarded, and don't touch the common wheel path (so no perf/behaviour regression when <torch>/lib exists). Schema bump is genuinely additive: only nested commit keys added, from_dict still round-trips pre-1.8, and _disaster_snapshot + the example snapshot were both updated to match. Test coverage is strong, including the key Buck-recovery case proving nm gets pointed at the maps-recovered path and no "not found" reason is emitted.

Re-review of prior Copilot comments — both Fixed (correct):

• Non-SHA module attr leaking into commit: resolved via _commit_from_attr_value (validates embed-or-is a 7-40 hex SHA), with tests rejecting unknown/dirty/tag and accepting+lowercasing a bare SHA. Verified.
• Docs still on 1.7: schema_version row, sample output, and changelog all bumped to 1.8. Verified.

Findings: 0 High · 0 Medium · 2 Low

Nice touch: preserving the original <torch>/lib path in the fallback reason string so the "missing lib" diagnostic still names a concrete path.

[amd-vivekag]

[Low] line.split() + parts[5] truncates any mapped path that contains a space (e.g. /opt/my libs/libtorch_hip.so becomes /opt/my), so the recovery silently misses the lib.

The pathname is always the trailing field, so split with a bounded count and take the remainder:

Fix:

Suggested change

	path_str = parts[5]
	parts = line.split(maxsplit=5)
	if len(parts) < 6:
	continue
	path_str = parts[5]

(the len(parts) < 6 check just above can be dropped since it's now folded in; adjust as needed). Real-world impact is small (torch libs rarely live under spaced paths), hence Low.

[amd-vivekag]

[Low] The PR body says aiter's +g<sha> parse was "generalized into _extract_commit_from_version", but _capture_aiter still has its own inline re.search(r"\+g([0-9a-f]{7,40})", version) (~line 3432). So there are now two divergent commit parsers, and aiter's is the weaker one (it won't catch the .git<sha> fork form this new helper handles).

Fix: have _capture_aiter call _extract_commit_from_version(version) instead of its own regex, or drop the "generalized" wording from the PR description. Not blocking — aiter's format is +g<sha> so it still works today.