Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP/RFH: Deprecate generalized linear indexing #20040

Closed
wants to merge 6 commits into from

Conversation

mbauman
Copy link
Sponsor Member

@mbauman mbauman commented Jan 14, 2017

This deprecates generalized linear indexing, meaning that after this patch, the only two ways to index into an AbstractArray{T,N} is either with one index or with N indices. This removes support for what I've called "partial linear indexing" (e.g., rand(2,3,4)[1, 7]) as well as trailing 1 indices (e.g., (1:10)[2,1,1]).

Note that until #18457 is merged, these deprecated definitions need to occur before the real definitions in order to get the method sorting right.

There are tons of deprecation warnings, and I need help in chasing them all down. Please feel free to push to this branch or open PRs targeted at it. My computer is slowly chugging through the test suite and saving the warnings, but here's an initial list:

  • test/subarray.jl: Many of these tests are explicitly designed to test this behavior; needs significant refactor or lots of reshapes.
  • test/abstractarray.jl: Same as subarray.

And then there are a bunch of places within the standard library that need refactored. I'll post the full depwarn log once I finally get to the end of the test suite, but here's an initial list. Here is the full log from make test. I modified depwarn to try and print where it thought the deprecation was coming from in an easy-to-grep-for way; that's how I've populated the list below. So if something there doesn't make sense, you can refer to the full log and find the full stack trace for it. While there are thousands of warnings, it's not as bad as I thought it might be… It seems like many come from the same areas:

  • .../test/arrayops.jl at 171
  • .../test/dsp.jl at 10
  • .../test/linalg/diagonal.jl at 9
  • .../test/linalg/generic.jl at 148
  • .../test/linalg/lu.jl at 31
  • .../test/show.jl at 12
  • .../test/show.jl at 422
  • .../test/show.jl at 512
  • .../test/show.jl at 539
  • .../test/show.jl at 619
  • .../test/sparse/umfpack.jl at 15
  • abstractarray.jl:995
  • arnoldi.jl:302
  • arnoldi.jl:303
  • arrayops.jl:328 [inlined]
  • dsp.jl:19
  • dsp.jl:71
  • dsp.jl:73
  • error.jl:88
  • error.jl:99 [inlined] (may be libgit/error.jl)
  • factorization.jl:48
  • factorization.jl:60
  • generic.jl:757
  • givens.jl:10
  • linalg.jl:85
  • linalg.jl:869
  • linalg.jl:96
  • matmul.jl:200
  • matmul.jl:225
  • matmul.jl:379
  • matmul.jl:387
  • matmul.jl:83
  • qr.jl:367
  • qr.jl:409
  • reference.jl:179
  • repository.jl:11
  • sparse.jl:1127 [inlined]
  • sparse.jl:155 [inlined]
  • sparse.jl:1565 [inlined]
  • sparse.jl:1567 [inlined]
  • sparse.jl:157 [inlined]
  • sparse.jl:163 [inlined]
  • sparse.jl:165 [inlined]
  • triangular.jl:1559
  • umfpack.jl:400
  • umfpack.jl:419

@Sacha0
Copy link
Member

Sacha0 commented Jan 15, 2017

(Apologies, accidentally pushed directly to this branch. Should be corrected now. Best!)

@mbauman
Copy link
Sponsor Member Author

mbauman commented Jan 15, 2017

Re: @andreasnoack in #20043:

I thought that generalized linear indexing was about indexing an N dimensional array with n indices where 1 < n < N. The old behavior (which I though we called trailing singleton dimension) is pretty convenient in this case where it allows us to avoid duplication of the kernel.

Yes, this deprecates both behaviors. You now need to specifically opt in to using vectors as matrices. You can still share the same kernel. See, for example, how I changed triangular.jl. I'm still not 100% on this change; it doesn't really simplify implementations. But it does tighten up our behaviors and make vectors more different from matrices, and it does allow catching a few more mistakes on the user side.

Of course, trailing singleton dimensions aren't nearly as error-prone, since they're bounds checked and must be 1. But we cannot currently deprecate just one of these behaviors… that'd require support for putting bound Varargs in the middle of an argument list.

### Multiplication with triangle to the left and hence rhs cannot be transposed.
for (f, g) in ((:*, :A_mul_B!), (:Ac_mul_B, :Ac_mul_B!), (:At_mul_B, :At_mul_B!))
@eval begin
function ($f)(A::AbstractTriangular, B::$mat)
($f)(A::AbstractTriangular, B::AbstractVector) = ($f)(A, reshape(B, Val{2}))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I read this correctly, I believe this changes the behavior such that the rhs is now promoted to a matrix, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Echoing @andreasnoack, these changes result in a matrix being returned rather than a vector, e.g.

julia> UpperTriangular(rand(3, 3)) * rand(3)
3×1 Array{Float64,2}:
 0.978125
 0.687285
 0.568977

Best!

@andreasnoack
Copy link
Member

@mbauman Thanks for explaining. I didn't read your original post carefully enough and I didn't anticipate how much this behavior was actually used in Base. Often you'll have to implement the change to realize that.

@mbauman
Copy link
Sponsor Member Author

mbauman commented Jan 15, 2017

Ah, you know what? Let's do this as bounds-checking instead of changing the implementation. I'll need some help on the C side, but it'll mean that we can separate these behaviors and make this a simpler fix.

@Sacha0
Copy link
Member

Sacha0 commented Jan 16, 2017

we can separate these behaviors and make this a simpler fix.

Do you mean (1) sequentially deprecate partial linear indexing and indexing with trailing singleton dimensions (as opposed to simultaneous deprecation) or (2) only deprecate partial linear indexing at this time? (Trying to determine how best to help.) Thanks!

@mbauman
Copy link
Sponsor Member Author

mbauman commented Jan 16, 2017

Yeah, I was thinking about just doing partial linear indexing first, and then seeing if there's still a stomach for trailing singletons. I'll need some help with the codegen modifications for Array.

@mbauman
Copy link
Sponsor Member Author

mbauman commented Jan 16, 2017

Closed in favor of the smaller, simpler, and more incremental process in #20079.

@mbauman mbauman closed this Jan 16, 2017
@mbauman mbauman mentioned this pull request Jan 26, 2017
27 tasks
@tkelman tkelman deleted the mb/deprecate-17440 branch February 11, 2017 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrays [a, r, r, a, y, s] deprecation This change introduces or involves a deprecation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants