Checking for overflow in matrix_mod2_dense not working on OS X 10.12.6

[koffie]

With OS X 10.12.6, Xcode 9.1 (and the same happens when building with clang, #12426):

sage -t --long --warn-long 64.0 src/sage/matrix/matrix_mod2_dense.pyx  # Killed due to kill signal
...
...
...
sage: sig_on_count() # check sig_on/off pairings (virtual doctest) ## line 90 ##
0
sage: type(random_matrix(GF(2),2,2)) ## line 165 ##
<type 'sage.matrix.matrix_mod2_dense.Matrix_mod2_dense'>
sage: Matrix(GF(2),3,3,1) # indirect doctest ## line 168 ##
[1 0 0]
[0 1 0]
[0 0 1]
sage: matrix(GF(2),0,[]) * vector(GF(2),0,[]) ## line 177 ##
()
sage: MatrixSpace(GF(2), 2^30)(1) ## line 182 ##

The failing doctest was introduced at #23742

Component: linear algebra

Author: Jeroen Demeyer

Branch: 145f9df

Reviewer: Volker Braun

Issue created by migration from https://trac.sagemath.org/ticket/24190

[jdemeyer]

comment:1

I'm currently building this on vbraun's testing machine. In the worst case, we just have to remove the test.

[jdemeyer]

comment:2

It seems that OS X simply doesn't support ulimit -v.

[jdemeyer]

Branch: u/jdemeyer/checking_for_overflow_in_matrix_mod2_dense_not_working_on_os_x_10_12_6

[jdemeyer]

Commit: 145f9df

[jdemeyer]

Author: Jeroen Demeyer

[jdemeyer]

New commits:

145f9df

Skip test on systems without RLIMIT_AS support

[vbraun]

Reviewer: Volker Braun

[vbraun]

Changed branch from u/jdemeyer/checking_for_overflow_in_matrix_mod2_dense_not_working_on_os_x_10_12_6 to 145f9df

[embray]

comment:7

I think this fix was incomplete, because it does not provide a workaround for the next two tests in the example:

            sage: import resource
            sage: if resource.RLIMIT_AS == getattr(resource, 'RLIMIT_RSS', None):
            ....:     # Skip this test if RLIMIT_AS is not properly
            ....:     # supported like on OS X, see Trac #24190
            ....:     raise RuntimeError("matrix allocation failed")
            ....: else:  # Real test
            ....:     MatrixSpace(GF(2), 2^30)(1)
            Traceback (most recent call last):
            ...
            RuntimeError: matrix allocation failed
            sage: MatrixSpace(GF(2), 1, 2^40).zero()
            Traceback (most recent call last):
            ...
            OverflowError: ...
            sage: MatrixSpace(GF(2), 2^40, 1).zero()
            Traceback (most recent call last):
            ...
            OverflowError: ...

The MatrixSpace(GF(2), 1, 2^40).zero() and so on still happily gobble up as much memory as they can if RLIMIT_AS was not set. See #25884 and #23979. In the latter, I dealt with the fact that setting RLIMIT_AS doesn't work on Cygwin, but in that case it raises a ValueError. I'm guessing, from this ticket, that on OSX the setrlimit call doesn't raise an exception, but doesn't actually work either.

I think we need a better solution for dealing with tests on systems where setting memory limits by the test runner doesn't work properly...

[embray]

Changed commit from 145f9df to none

[embray]

comment:8

I take it back--the other tests do immediately raise OverflowError as they should. Question is then why the MatrixSpace(GF(2), 2^30)(1) is being run at all on Cygwin. It shouldn't, and yet it seems it does...