Hardware detection: AMD unified-memory rigs fall through to 4K Ollama budget

[antoinezambelli]

Hardware detection: AMD unified-memory rigs fall through to 4K Ollama budget

Problem

forge.context.hardware.detect_hardware() shells out to nvidia-smi only. On AMD unified-memory rigs (Ryzen AI 300 / Strix Halo class — Corsair AI 300, MSI Claw, similar) there is no nvidia-smi, so detection returns None.

Downstream, ServerManager._ollama_vram_tier_budget() in src/forge/server.py:383-394 calls detect_hardware() and treats None as the lowest tier — falling through to 4096 tokens. This applies silently to BudgetMode.FORGE_FULL and BudgetMode.FORGE_FAST whenever the backend is Ollama.

The llamaserver / llamafile path is unaffected — those resolve via /props after the server auto-tunes (src/forge/server.py:285-289), which works fine on any GPU vendor. This issue is therefore Ollama-scoped: llama-server users on AMD rigs are unaffected.

Current state

• src/forge/context/hardware.py:41 — detect_hardware() calls nvidia-smi and returns None on any failure (FileNotFoundError, non-zero exit, etc.).
• src/forge/server.py:383-394 — _ollama_vram_tier_budget():

  hw = detect_hardware()
  if hw is None:
      return 4096
  vram_gb = hw.vram_total_gb
  if vram_gb >= 48:    return 262_144
  elif vram_gb >= 24:  return 32_768
  else:                return 4_096

• No log, no warning. Caller has no signal that detection failed vs. succeeded with a low-tier GPU.

Concrete situation

Observed on rig-03 (Corsair AI 300, AMD Vulkan/RADV, 128GB unified memory, Fedora 43):

• nvidia-smi not present → detect_hardware() returns None
• BudgetMode.FORGE_FULL on Ollama backend silently resolves to num_ctx=4096
• llama-server backend on the same rig allocates n_ctx_train from GGUF metadata (per llama.cpp/src/llama-context.cpp:57: cparams.n_ctx = params.n_ctx == 0 ? hparams.n_ctx_train : params.n_ctx;), which can be 128K+ for models like Granite 4.0
• /sys/class/drm/card1/device/mem_info_vram_total reports 68719476736 bytes (64 GB — the BIOS-carved chunk of the 128GB unified pool). That lands cleanly in the ≥48GB → 262_144 tier.

The 4K floor is a defensible default when detection genuinely fails, but on a 128GB unified rig it understates what's actually available by ~2 orders of magnitude. And the silent fallback is a Principle-#1 violation — caller has no way to distinguish "no GPU" from "AMD GPU we can't probe" from "tiny GPU."

Decision (2026-05-02): Vulkan-only AMD backend

forge's chosen AMD GPU backend is Vulkan/RADV. ROCm is intentionally out of scope (not installed on rig-03 by design — Vulkan covers the supported workloads with simpler setup). This means the probe ladder does not rely on rocm-smi. The original sketch listed it as the first AMD probe; that's been dropped.

Proposed change

Scope: detect AMD unified-memory rigs specifically, since that's the gap that's biting. Discrete AMD GPUs are a separate question.

Two parts:

1. Add an AMD detection path in detect_hardware() via sysfs.

   • Read /sys/class/drm/card*/device/vendor to identify AMD (0x1002).
   • Read /sys/class/drm/card*/device/mem_info_vram_total for the kernel-reported VRAM/unified-pool size in bytes.
   • Read GPU name from /sys/class/drm/card*/device/uevent (or lspci as fallback) for a human-readable identifier.
   • No userspace dependency — purely kernel-exposed.

   Return a HardwareProfile that distinguishes "discrete VRAM" from "unified system RAM" so downstream tier logic can choose appropriately.

2. Loud failure for genuine "no GPU" detection. When all probes fail, log at WARN with the probes that were attempted and the reason each failed. The 4K floor still applies, but at least it's not silent.

Out of scope (separate issues if pursued):

• Apple Silicon detection — same shape (unified memory, no nvidia-smi) but Metal-specific quirks.
• Multi-GPU detection — current detect_hardware() only reports the first GPU.
• Ollama-side context override — Ollama config concern, not forge.
• ROCm/rocm-smi probing — Vulkan-only decision above.
• vainfo probing — superseded by sysfs (no userspace dep).

Acceptance criteria

• On rig-03 (or equivalent AMD unified-memory rig), detect_hardware() returns a HardwareProfile with a non-None indicator of available memory.
• _ollama_vram_tier_budget() resolves to a sensible budget (≥32K) on a 128GB unified rig.
• Existing nvidia-smi path is unchanged — same call, same behavior, same return.
• When all detection probes fail, a single WARN log fires (one-shot per session) listing what was tried.
• Tests cover: nvidia-smi present, nvidia-smi absent + AMD sysfs present, all probes fail.

Notes for implementation

• HardwareProfile currently has gpu_name: str and vram_total_mb: int. Adding a memory_kind: Literal["discrete", "unified"] field (and a gpu_vendor field) lets _ollama_vram_tier_budget() and logs distinguish 64GB-unified from 64GB-discrete without forcing every caller to know the difference.
• The 4K-fallback when truly nothing detects should remain — it's a safety net, not a bug. The issue is the silent fallback on detectable-but-unsupported hardware.

[antoinezambelli]

Fixed on main by 992e346.

detect_hardware() now probes AMD via sysfs after the nvidia-smi path: it walks /sys/class/drm/card*/device/, matches vendor == 0x1002 (AMD's PCI vendor ID), and reads mem_info_vram_total for the
BIOS-carved memory size. On a Ryzen AI 300 / Strix Halo-class APU, that's the unified-memory VRAM carve-out — so the Ollama VRAM-tier budget now sees the real memory instead of silently falling through to
the 4096-token floor.

For context on why this only bit Ollama: llama.cpp and llamafile already read their effective context from the server's /props endpoint, so they were never affected by the hardware-detection fallback —
only the Ollama tier-budget path depended on detect_hardware() reporting real memory.