Unexpected nans in complex cholfact

[tkelman]

This is related to #245 but I can get it to happen on julia.mit.edu without setting OPENBLAS_CORETYPE to use old kernels, and it persists even with the latest develop branch of openblas:

using Base.Test

apd = [5.8525753f0 + 0.0f0im -0.79540455f0 + 0.7066077f0im 0.98274714f0 + 1.3824869f0im 2.619998f0 + 1.8532984f0im -1.8306153f0 - 1.2336911f0im 0.32275113f0 + 0.015575029f0im 2.1968813f0 + 1.0640624f0im 0.27894387f0 + 0.97911835f0im 3.0476584f0 + 0.18548489f0im 0.3842994f0 + 0.7050991f0im
                 -0.79540455f0 - 0.7066077f0im 8.313246f0 + 0.0f0im -1.8076122f0 - 0.8882447f0im 0.47806996f0 + 0.48494184f0im 0.5096429f0 - 0.5395974f0im -0.7285097f0 - 0.10360408f0im -1.1760061f0 - 2.7146957f0im -0.4271084f0 + 0.042899966f0im -1.7228563f0 + 2.8335886f0im 1.8942566f0 + 0.6389735f0im
                 0.98274714f0 - 1.3824869f0im -1.8076122f0 + 0.8882447f0im 9.367975f0 + 0.0f0im -0.1838578f0 + 0.6468568f0im -1.8338387f0 + 0.7064959f0im 0.041852742f0 - 0.6556877f0im 2.5673025f0 + 1.9732997f0im -1.1148382f0 - 0.15693812f0im 2.4704504f0 - 1.0389464f0im 1.0858271f0 - 1.298006f0im
                 2.619998f0 - 1.8532984f0im 0.47806996f0 - 0.48494184f0im -0.1838578f0 - 0.6468568f0im 3.1117508f0 + 0.0f0im -1.956626f0 + 0.22825956f0im 0.07081801f0 - 0.31801307f0im 0.3698375f0 - 0.5400855f0im 0.80686307f0 + 1.5315914f0im 1.5649154f0 - 1.6229297f0im -0.112077385f0 + 1.2014246f0im
                 -1.8306153f0 + 1.2336911f0im 0.5096429f0 + 0.5395974f0im -1.8338387f0 - 0.7064959f0im -1.956626f0 - 0.22825956f0im 3.6439795f0 + 0.0f0im -0.2594722f0 + 0.48786148f0im -0.47636223f0 - 0.27821827f0im -0.61608654f0 - 2.01858f0im -2.7767487f0 + 1.7693765f0im 0.048102796f0 - 0.9741874f0im
                 0.32275113f0 - 0.015575029f0im -0.7285097f0 + 0.10360408f0im 0.041852742f0 + 0.6556877f0im 0.07081801f0 + 0.31801307f0im -0.2594722f0 - 0.48786148f0im 3.624376f0 + 0.0f0im -1.6697118f0 + 0.4017511f0im -1.4397877f0 - 0.7550918f0im -0.31456697f0 - 1.0403451f0im -0.31978557f0 + 0.13701046f0im
                 2.1968813f0 - 1.0640624f0im -1.1760061f0 + 2.7146957f0im 2.5673025f0 - 1.9732997f0im 0.3698375f0 + 0.5400855f0im -0.47636223f0 + 0.27821827f0im -1.6697118f0 - 0.4017511f0im 6.8273163f0 + 0.0f0im -0.10051322f0 + 0.24303961f0im 1.4415971f0 + 0.29750675f0im 1.221786f0 - 0.85654986f0im
                 0.27894387f0 - 0.97911835f0im -0.4271084f0 - 0.042899966f0im -1.1148382f0 + 0.15693812f0im 0.80686307f0 - 1.5315914f0im -0.61608654f0 + 2.01858f0im -1.4397877f0 + 0.7550918f0im -0.10051322f0 - 0.24303961f0im 3.4057708f0 + 0.0f0im -0.5856801f0 - 1.0203559f0im 0.7103452f0 + 0.8422135f0im
                 3.0476584f0 - 0.18548489f0im -1.7228563f0 - 2.8335886f0im 2.4704504f0 + 1.0389464f0im 1.5649154f0 + 1.6229297f0im -2.7767487f0 - 1.7693765f0im -0.31456697f0 + 1.0403451f0im 1.4415971f0 - 0.29750675f0im -0.5856801f0 + 1.0203559f0im 7.005772f0 + 0.0f0im -0.9617417f0 - 1.2486815f0im
                 0.3842994f0 - 0.7050991f0im 1.8942566f0 - 0.6389735f0im 1.0858271f0 + 1.298006f0im -0.112077385f0 - 1.2014246f0im 0.048102796f0 + 0.9741874f0im -0.31978557f0 - 0.13701046f0im 1.221786f0 + 0.85654986f0im 0.7103452f0 - 0.8422135f0im -0.9617417f0 + 1.2486815f0im 3.4629636f0 + 0.0f0im]
b = [-0.905011814118756 + 0.2847570854574069im -0.7122162951294634 - 0.630289556702497im
                 -0.7620356655676837 + 0.15533508334193666im 0.39947219167701153 - 0.4576746001199889im
                 -0.21782716937787788 - 0.9222220085490986im -0.727775859267237 + 0.50638268521728im
                 -1.0509472322215125 + 0.5022165705328413im -0.7264975746431271 + 0.31670415674097235im
                 -0.6650468984506477 - 0.5000967284800251im -0.023682508769195098 + 0.18093440285319276im
                 -0.20604111555491242 + 0.10570814584017311im 0.562377322638969 - 0.2578030745663871im
                 -0.3451346708401685 + 1.076948486041297im 0.9870834574024372 - 0.2825689605519449im
                 0.25336108035924787 + 0.975317836492159im 0.0628393808469436 - 0.1253397353973715im
                 0.11192755545114 - 0.1603741874112385im 0.8439562576196216 + 1.0850814110398734im
                 -1.0568488936791578 - 0.06025820467086475im 0.12696236014017806 - 0.09853584666755086im]
cholfact(apd, :L, Val{true}) \ b
r = factorize(apd)[:U]
E = abs(apd - r'*r)
ε = eps(abs(float(one(Complex64))))
n = 10
for i=1:n, j=1:n
    @test E[i,j] <= (n+1)ε/(1-(n+1)ε)*real(sqrt(apd[i,i]*apd[j,j]))
end

This results in test failed: NaN32 <= 9.709564f-6. This was reduced from a test failure that was happening repeatably in b512505cacc69fd1ba2cf04ecca79cee587a4cfd, which was in a WIP version of JuliaLang/julia#12870 (you can add and fetch from my fork as a remote for an easier time checking out that commit, since it is no longer present on any branch here), ref comments at https://github.com/JuliaLang/julia/blob/8681313fbab1dcd88b8a9fe9b56f91edd0b75f9b/test/linalg/cholesky.jl#L41-L43. I think this qualifies as "worth reporting upstream" but might make their lives easier to translate the test case to C or Fortran first.

[andreasnoack]

julia.mit.edu has old processors so maybe that is the reason. I don't get the NaNs on julia.mit.edu when using MKL so it could be the OpenBLAS kernels for older architectures.

[tkelman]

How about on recent macs or newer processor linux?

[andreasnoack]

Just tried on my Haswell MacBook Pro and a Haswell Dell running Ubuntu 15.04 and there are no problems.

[tkelman]

Can you reproduce by setting OPENBLAS_CORETYPE to any of the following?

    "Katmai",
    "Coppermine",
    "Northwood",
    "Prescott",
    "Banias",
    "Atom",
    "Core2",
    "Penryn",
    "Dunnington",
    "Nehalem",
    "Athlon",
    "Opteron",
    "Opteron(SSE3)",
    "Barcelona",
    "Nano",
    "Sandybridge",
    "Bobcat",
    "Bulldozer",
    "Piledriver",
    "Haswell",
    "Steamroller",

[andreasnoack]

My Haswell based Macbook Pro gives

Arch	isnan
Katmai	true
Coppermine	true
Northwood	true
Prescott	true
Banias	true
Atom	true
Core2	true
Penryn	true
Dunnington	true
Nehalem	true
Athlon	true
Opteron	true
Opteron(SSE3)	true
Barcelona	true
Nano	true
Sandybridge	false
Bobcat	true
Bulldozer	segfault on startup
Piledriver	segfault on startup
Haswell	false
Steamroller	segfault on BLAS call

[tkelman]

Great, thanks for running those. On a few different machines I tried I was either always getting the nan or never getting it. So Sandybridge and Haswell kernels aren't hitting the bug, but earlier ones are.

[tkelman]

Worked around upstream with the latest openblas release