FIBO (base briaai/FIBO) txt2img produces pure noise — transformer velocity ~5–8× too small

[gethyper]

Summary

mflux-generate-fibo (base FIBO, briaai/FIBO) produces pure rainbow-confetti noise for every prompt — the final latent never leaves its initial noise. This reproduces via the stock CLI with no custom code, at both q8 and full precision, and at both guidance=4.0 and guidance=1.0. I've isolated it to the transformer's velocity prediction (VAE, latent packing, weight-loading and scheduler are all fine), and I'm sharing the diagnostics below in the hope it saves someone time.

Environment

• mflux 0.18.0, installed from git main (97ac5e62)
• mlx 0.31.2
• macOS / Apple Silicon (M-series)
• Model: briaai/FIBO (access-granted, weights ~17 GB downloaded fine)

Reproduction

mflux-generate-fibo --prompt '{"caption":"a red cube on a white table"}' --steps 50 --seed 42
# also: -q 8, and guidance 1.0 -> identical confetti

Output is smooth multicolored blobs with no semantic structure (same for --steps 8).

What is NOT the cause (verified)

• VAE — an encode → decode round-trip of a real image is near-perfect (e.g. a photo with mean/std 249.9/35.6 comes back 249.9/34.6). De-normalization (latents*std + mean) and patchify/unpatchify are correct.
• Latent packing — FiboLatentCreator.pack_latents / unpack_latents are exact inverses (transpose+reshape).
• Weight loading — all transformer params load with real trained statistics (x_embedder.weight std ≈ 0.083, context_embedder 0.009, proj_out 0.023, norm_out.linear 0.040), i.e. not left at default init.
• Quantization — full-precision run (peak ~37 GB) is identical to -q 8.
• CFG — same at guidance=4.0 and guidance=1.0.
• Timestep conditioning — sweeping the value fed to BriaFiboTimestepProjEmbeddings (sigma, sigma*1000, (1-sigma)*1000, 1000) barely moves the result.

Root cause — transformer velocity is far too small / poorly directed

Diagnostic: encode a real image → x0 (packed), sample noise n, form x_t = (1-σ)·x0 + σ·n, run transformer(...) and compare the predicted velocity v to the rectified-flow target n - x0:

σ	expected	measured
1.0 (pure noise, t=0)	std(v) ≈ 1.0	std(v) ≈ 0.19
0.5	cos(v, n−x0) ≈ 0.7+, std(v) ≈ O(1)	cos ≈ 0.2–0.37, std(v) ≈ 0.15 vs std(n−x0) ≈ 1.46 (~8× too small)

Because v is ~5–8× too small (and only weakly correlated with the true direction), the Euler step latents += (σ_{t+1}−σ_t)·v barely moves the latent, so after all 50 steps it is still essentially the initial noise → confetti. The magnitude deficit is invariant to the timestep value, which rules out the AdaLayerNorm-gate/timestep-embedding hypothesis. All transformer blocks (joint + single) have correct residual connections.

So the discrepancy appears to be somewhere in the transformer forward — candidates I haven't yet pinned down: the RoPE image/text position ids, the per-block SmolLM3 text_encoder_layers injection (encoder_hidden_states[:, :, :1536] concatenated with caption_projection(layer_i)), or an attention scale — i.e. something that corrupts the predicted direction/magnitude without touching the VAE or scheduler.

Possibly-related minor inconsistency

variants/txt2img/fibo.py never applies a sigma shift, whereas variants/edit/fibo_edit.py does (it calls config.scheduler.set_mu(FlowMatchEulerDiscreteScheduler._compute_linear_mu(config.image_seq_len))). Both configs have requires_sigma_shift=False, so the generic Config.scheduler path skips set_image_seq_len for txt2img. Adding the set_mu call to fibo.py changes the schedule but does not fix the confetti — noting it only in case it's a real oversight.

Ask

Could a maintainer confirm the intended FIBO txt2img flow (timestep convention + whether txt2img should also apply the sigma shift), and whether the per-block text_encoder_layers injection / RoPE ids match the reference BriaFiboTransformer2DModel? Happy to run further numerical probes or test a patch — I have the model loaded and the diagnostic scripts ready.