oneMKL.hemv does not respect queue order

[maleadt]

MWE:

using oneAPI, LinearAlgebra, Test

function main()
    m = 2
    n = 2

    T = ComplexF32
    alpha = one(T)
    beta = one(T)

    A = ones(T,m,n)
    dA = oneArray(A)
    sA = ones(T,m,m)
    sA = sA + transpose(sA)
    dsA = oneArray(sA)
    hA = ones(T,m,m)
    hA = hA + hA'
    dhA = oneArray(hA)
    x = ones(T,m)
    dx = oneArray(x)
    y = ones(T,m)
    dy = oneArray(y)

    #synchronize()

    BLAS.hemv!('U',alpha,hA,x,beta,y)
    oneMKL.hemv!('U',alpha,dhA,dx,beta,dy)

    hy = Array(dy)
    @test y ≈ hy
end

Note the commented-out call to synchronize(). Without it, the calculation is wrong:

Test Failed at /home/tim/Julia/pkg/oneAPI/wip.jl:31
  Expression: y ≈ hy
   Evaluated: ComplexF32[5.0f0 + 0.0f0im, 5.0f0 + 0.0f0im] ≈ ComplexF32[9.0f0 + 0.0f0im, 9.0f0 + 0.0f0im]

Adding an additional synchronization (which shouldn't be necessary, as we invoke onemklChemv and thus oneapi::mkl::blas::column_major::hemv passing a SYCL queue based on the currently-active oneL0 command queue) makes the example succeed. This looks like an MKL bug?

cc @kballeda

[maleadt]

Ah, no, this is a bug in the test suite: We call BLAS.hemv! before the copy to device memory has finished.