Not sure if this is the appropriate repository to post at, but when I try to get a gradient using Zygote and the ODE is solved on GPU I just get an error message. If I run the same code without having the ODE solved on GPU I get a gradient and no errors.
That zygote returns a gradient.
Initializing z0Float32[0.0, 0.0, 0.0, 0.0]
ERROR: LLVM error: Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i32, i64, i64, i64, i64, float*, i64, float*, i64, i64, float*, i64)* @cublasSgemm_v2_64
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i32, i64, i64, i64, i64, float*, i64, float*, i64, i64, float*, i64)* @cublasSgemm_v2_64
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i32, i32, i32, i32, i64, float*, i32, float*, i32, i64, float*, i32)* @cublasSgemm_v2
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i32, i32, i32, i32, i64, float*, i32, float*, i32, i64, float*, i32)* @cublasSgemm_v2
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i32, i32, i64, float*, i32, float*, i32, i64, float*, i32)* @cublasSgemv_v2
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i32, i32, i64, float*, i32, float*, i32, i64, float*, i32)* @cublasSgemv_v2
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i64, i64, i64, float*, i64, float*, i64, i64, float*, i64)* @cublasSgemv_v2_64
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i64, i64, i64, float*, i64, float*, i64, i64, float*, i64)* @cublasSgemv_v2_64
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i64, float*, i32)* @cublasSscal_v2
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i64, i64, float*, i64)* @cublasSscal_v2_64
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i64, i64, float*, i64, float*, i64, i64, float*, i64)* @cublasSsymv_v2_64
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i64, i64, float*, i64, float*, i64, i64, float*, i64)* @cublasSsymv_v2_64
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i32, i64, float*, i32, float*, i32, i64, float*, i32)* @cublasSsymv_v2
Attribute 'nocapture' applied to incompatible type!
i32 (i64, i32, i32, i64, float*, i32, float*, i32, i64, float*, i32)* @cublasSsymv_v2
Stacktrace:
[1] verify(mod::LLVM.Module)
@ LLVM C:\Users\mabun\.julia\packages\LLVM\UFrs4\src\analysis.jl:19
[2] enzyme!(job::GPUCompiler.CompilerJob{…}, interp::Enzyme.Compiler.Interpreter.EnzymeInterpreter{…}, mod::LLVM.Module, primalf::LLVM.Function, TT::Type, mode::Enzyme.API.CDerivativeMode, width::Int64, parallel::Bool, actualRetType::Type, wrap::Bool, modifiedBetween::NTuple{…} where N, returnPrimal::Bool, expectedTapeType::Type, loweredArgs::Set{…}, boxedArgs::Set{…})
@ Enzyme.Compiler C:\Users\mabun\.julia\packages\Enzyme\X4pFp\src\compiler.jl:2628
[3] compile_unhooked(output::Symbol, job::GPUCompiler.CompilerJob{…})
@ Enzyme.Compiler C:\Users\mabun\.julia\packages\Enzyme\X4pFp\src\compiler.jl:4972
[4] compile(target::Symbol, job::GPUCompiler.CompilerJob; kwargs::@Kwargs{})
@ GPUCompiler C:\Users\mabun\.julia\packages\GPUCompiler\bTNLD\src\driver.jl:67
[5] compile
@ C:\Users\mabun\.julia\packages\GPUCompiler\bTNLD\src\driver.jl:55 [inlined]
[6] _thunk(job::GPUCompiler.CompilerJob{…}, postopt::Bool)
@ Enzyme.Compiler C:\Users\mabun\.julia\packages\Enzyme\X4pFp\src\compiler.jl:5826
[7] _thunk
@ C:\Users\mabun\.julia\packages\Enzyme\X4pFp\src\compiler.jl:5824 [inlined]
[8] cached_compilation
@ C:\Users\mabun\.julia\packages\Enzyme\X4pFp\src\compiler.jl:5881 [inlined]
[9] thunkbase(mi::Core.MethodInstance, World::UInt64, FA::Type{…}, A::Type{…}, TT::Type, Mode::Enzyme.API.CDerivativeMode, width::Int64, ModifiedBetween::NTuple{…} where N, ReturnPrimal::Bool, ShadowInit::Bool, ABI::Type, ErrIfFuncWritten::Bool, RuntimeActivity::Bool, StrongZero::Bool, edges::Vector{…})
@ Enzyme.Compiler C:\Users\mabun\.julia\packages\Enzyme\X4pFp\src\compiler.jl:5997
[10] thunk_generator(world::UInt64, source::Union{…}, FA::Type, A::Type, TT::Type, Mode::Enzyme.API.CDerivativeMode, Width::Int64, ModifiedBetween::NTuple{…} where N, ReturnPrimal::Bool, ShadowInit::Bool, ABI::Type, ErrIfFuncWritten::Bool, RuntimeActivity::Bool, StrongZero::Bool, self::Any, fakeworld::Any, fa::Type, a::Type, tt::Type, mode::Type, width::Type, modifiedbetween::Type, returnprimal::Type, shadowinit::Type, abi::Type, erriffuncwritten::Type, runtimeactivity::Type, strongzero::Type)
@ Enzyme.Compiler C:\Users\mabun\.julia\packages\Enzyme\X4pFp\src\compiler.jl:6192
[11] macro expansion
@ C:\Users\mabun\.julia\packages\Enzyme\X4pFp\src\rules\jitrules.jl:464 [inlined]
[12] runtime_generic_augfwd(::Type{…}, ::Val{…}, ::Val{…}, ::Val{…}, ::Val{…}, ::Val{…}, ::typeof(*), ::Nothing, ::Float32, ::Base.RefValue{…}, ::CuArray{…}, ::Nothing, ::CuArray{…}, ::CuArray{…})
@ Enzyme.Compiler C:\Users\mabun\.julia\packages\Enzyme\X4pFp\src\rules\jitrules.jl:574
[13] du
@ c:\Users\mabun\OneDrive - Danmarks Tekniske Universitet\Projects\QuantumOptimalControl\scripts\gradient_cuda_QT.jl:48
Some type information was truncated. Use `show(err)` to see complete types.Add any other context about the problem here.
Describe the bug 🐞
Not sure if this is the appropriate repository to post at, but when I try to get a gradient using Zygote and the ODE is solved on GPU I just get an error message. If I run the same code without having the ODE solved on GPU I get a gradient and no errors.
Expected behavior
That zygote returns a gradient.
Minimal Reproducible Example 👇
Error & Stacktrace⚠️
Environment (please complete the following information):
using Pkg; Pkg.status()using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)versioninfo()Additional context
Add any other context about the problem here.