Core8: Enzyme-native DAE-init gradient crashes on MTK __apply_copy_template (BoundsError); upstream Enzyme#3259

[ChrisRackauckas-Claude]

Summary

Core8 (test/Core8/mtk.jl, "MTK Forward Mode") fails because reverse-mode Enzyme crashes differentiating MTK's DAE-initialization parameter promotion (ModelingToolkitBase.__apply_copy_template) on the DefaultInit setup — CI shows BoundsError: attempt to access MemoryRef{Float64} at index [6/7].

Triggered by #1498 (3907835), which restored an Enzyme-native init-sensitivity gradient: _init_originator_gradient(::EnzymeOriginator) (src/concrete_solve.jl:384) → Enzyme.gradient(set_runtime_activity(Reverse), Const(init_loss), tunables). That nested Enzyme.gradient is what hits the Enzyme bug.

Upstream

Filed: EnzymeAD/Enzyme.jl#3259 (reverse-mode over __apply_copy_template's Union-typed-index mapreduce/vcat → undersized shadow Memory{Float64} BoundsError / TypeAnalysis hang / symbol-multiply-defined, depending on compile state). CheckInit/BrownFullBasicInit work; only DefaultInit breaks.

Options

• Wait for the upstream Enzyme fix (#3259) — keeps the init gradient Enzyme-native (the Restore Enzyme-native init-sensitivity gradient (revert #1467 Zygote workaround) #1498 direction). Core8 stays red until then.
• SciMLSensitivity-side stopgap to get green now: compute the tiny scalar init-sensitivity gradient with ForwardDiff instead of Enzyme.gradient in _init_originator_gradient(::EnzymeOriginator) (the init nonlinear solve already runs on AutoForwardDiff). This routes around the Enzyme bug for users + tests without reintroducing the old Zygote workaround (Enzyme: route MTK DAE init gradient through Zygote; enable Enzyme-outer mtk.jl test #1467). Changes Restore Enzyme-native init-sensitivity gradient (revert #1467 Zygote workaround) #1498's choice for the init sub-gradient.

Per @ChrisRackauckas's call, currently filing upstream and leaving the Enzyme-native path; the ForwardDiff stopgap is available if we want Core8 green before Enzyme lands the fix.

[ChrisRackauckas-Claude]

Resolved with the Zygote-pullback revert (the #1467 workaround) rather than the ForwardDiff option: PR #1515. Verified locally that the DefaultInit setup now returns the correct gradient ([-88324.833, 46528.856, 99324.874], matching CheckInit/BrownFullBasicInit). Stopgap until EnzymeAD/Enzyme.jl#3259 is fixed, then #1498's Enzyme-native init path can be restored.

[wsmoses]

@ChrisRackauckas-Claude please re-evaluate

[ChrisRackauckas-Claude]

Re-evaluated as requested — the upstream fix works; ready to re-land the Enzyme-native path.

EnzymeAD/Enzyme.jl#3259 was fixed by EnzymeAD/Enzyme.jl#3279 (link_split_existing!), merged 2026-07-05 and released in Enzyme v0.13.174 — which is already exactly this package's [compat] floor.

Verified locally: Julia 1.12.6, Enzyme v0.13.186, ModelingToolkit v11.34.0, against a checkout with the #1498 Enzyme-native _init_originator_gradient(::EnzymeOriginator) re-applied (i.e. #1515 reverted). Ran the Enzyme#3259 reproducer (MTK DAE with algebraic constraint, GaussAdjoint(EnzymeVJP()), outer Enzyme.autodiff with set_runtime_activity), warm-compile sweep in one process:

BrownFullBasicInit:   dtun = [-88396.24382131331, 46556.72765802712, 99402.34301657784]
DefaultInit:          dtun = [-88396.24382131331, 46556.72765802712, 99402.34301657784]
DefaultInit-2nd-call: dtun = [-88396.24382131331, 46556.72765802712, 99402.34301657784]

The previously-crashing DefaultInit setup (shadow-Memory BoundsError / TypeAnalysis hang / symbol-multiply-defined) now compiles and returns the same gradient as BrownFullBasicInit, stable across repeated outer Enzyme.autodiff calls (so the #1469-style repeated-call corruption also does not recur).

Currently running the full GROUP=Core8 suite locally on the re-land branch; PR to follow once it's green. That PR supersedes #1497 (whose compat gate at 0.13.163 predates the #3259 fix).

[ChrisRackauckas-Claude]

Update after the full local GROUP=Core8 run on the re-land branch: the upstream fix is only partial — re-land is still blocked.

What I verified (Julia 1.12.6, Enzyme v0.13.186, MTK v11.34.0, Enzyme-native _init_originator_gradient re-applied locally):

• ✅ The #3259 reproducer (BrownFullBasicInit → DefaultInit → DefaultInit, one process) passes, gradients identical and repeatable — #3279 fixed that form.
• ✅ 9/15 setups of the mtk.jl sweep pass with identical gradients [-88396.244, 46556.728, 99402.343].
• ❌ Setup 10 (correctu0 / NoInit()) hangs in Enzyme compilation >50 min warm, while the identical setup completes in ~6 min in a fresh process — the order-dependent TypeAnalysis blowup form from Enzyme#3259.
• ❌ The actual GROUP=Core8 test run dies instead with the runtime form: BoundsError: attempt to access MemoryRef{Float64} at index [5] in __apply_copy_template / override_bc_mapreduce (Fix1 over the init NonlinearSolution, i.e. the OverrideInit promotion path). 37 passed, 1 errored, 5 broken.

Filed the follow-up upstream issue @wsmoses asked for in Enzyme#3259: EnzymeAD/Enzyme.jl#3350 (with the per-setup sweep script and cold-vs-warm evidence).

State of the re-land: branch reland-enzyme-native-init-gradient is ready locally (no compat change needed — the floor 0.13.174 already includes #3279) and I'm holding it until Enzyme#3350 is fixed; #1497 was closed as superseded. This issue stays open tracking Enzyme#3350.

[wsmoses]

@ChrisRackauckas-Claude / @ChrisRackauckas please re-evaluate on latest release