Rules for cumsum
#573
Rules for cumsum
#573
Conversation
| ##### `cumsum` | ||
| ##### | ||
|
|
||
| function frule((_, xdot), ::typeof(cumsum), x::AbstractArray; dims::Integer) |
There was a problem hiding this comment.
Perhaps an frule for cumsum! here as well?
There was a problem hiding this comment.
Are forward rlues for mutating functions safe? I haven't thought that through.
There was a problem hiding this comment.
For reverse mode, #521 has thorny examples.
There was a problem hiding this comment.
Yes, I believe so, since the primal and tangent are mutated simultaneously. We already have several frules for mutating functions. See e.g. https://github.com/JuliaDiff/ChainRules.jl/blob/main/src/rulesets/LinearAlgebra/factorization.jl.
There was a problem hiding this comment.
Ok, nice.
Then someone should do a big PR adding this to half the Base functions, right? sum! and circshift! et. al.
There was a problem hiding this comment.
Must we define methods like cumsum!([0,0,0], ZeroTangent())?
There was a problem hiding this comment.
Hm, not certain. @oxinabox didn't we decide at some point that it's up to the AD to handle cases where all (co)tangents are zero tangents?
There was a problem hiding this comment.
I guess we can solve this when or if it comes up.
There was a problem hiding this comment.
yeah we expect the AD to handle cases in reverse mode when inputs are all ZeroTangent.
For forwards mode, i would need to think more, but i think even with mutation https://en.wikipedia.org/wiki/Nothing_comes_from_nothing
We can absolutely add fixes as things come up.
No description provided.