-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add nondiff rules #252
add nondiff rules #252
Conversation
idk if these are worth testing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very nice.
I'm not of the opinion that tests would be particularly helpful here. We'll probably want to add regression tests when we find issues with these rules / decide to add extra rules, but for now I think this is completely fine.
Things from Base
@nograd
ed in Zygote but not @non_differentiable
d here:
Base.gc_num
Base.time_ns
Channel
schedule
floor
ceil
trunc
round
div
eachindex
Base.OneTo
axes
Colon()
ones
zeros
one
zero
all
any
Things in submodules:
Broadcast.combine_styles
Broadcast.result_style
These all look broadly correct to me. Maybe we want to implement them before merging this?
@non_differentiable write(::IO, ::Any, ::Any, ::Any) | ||
@non_differentiable write(::IO, ::Any, ::Any, ::Any, ::Any) | ||
|
||
@non_differentiable Libc.free(::Any) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like Libc
has quite a lot of methods that certainly don't appear differentiable that don't appear on this list. Is there a particular reason for the omission, or just because you've not had time to write them out yet?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We only grabbed once that had particular types uses in their type signatures.
In particular we grabbed nothing with Number
in the type-sig
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh interesting. Just looking through the methods there are definitely some good candidates e.g. calloc
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you want to do about this one? I'm happy to leave for future work for the sake of getting this in provided that we document what we think is missing in an issue somewhere.
I think the following we already have as returning
I am dubious about these:
We should definately have these:
I think these make sense to have
Also i guess the other logical operators
|
Hmmm yeah, maybe. There are definitely some methods of them that involve clearly non-differentiable types that we can be pretty sure are Assuming that no one is punning on them, I think I'm correct in saying that they all have zero derivative everywhere, except at the points where the functions change value. At those points the derivative is unbounded / undefined / whatever we're calling |
Right but |
Good point. |
perhaps we can generalize the code. Could be something like |
Good point -- a constant rule would be helpful. I vote that we get this in and then add a |
Co-authored-by: willtebbutt <wt0881@my.bristol.ac.uk>
Co-authored-by: willtebbutt <wt0881@my.bristol.ac.uk>
Other zyogte ones:
|
Co-authored-by: Nick Robinson <npr251@gmail.com>
There are lots more we could add, but lets add them as they come up. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
780: Move a bunch of no_grad to ChainRules r=oxinabox a=oxinabox this is the partner to JuliaDiff/ChainRules.jl#252 It will fail til that is merged and tagged What is left is: - Types (because JuliaDiff/ChainRulesCore.jl#213) (e.g. `Colon`, `OneTo` `Channel`) - Things to which the derivative is `Zero()` not `DoesNotExist()` (e.g. `one`, `ones`, `zero`, `zeros`) - Things that felt too magic: e.g. `Base.eval` Should I bump patch version and tag a release? Co-authored-by: Lyndon White <lyndon.white@invenialabs.co.uk> Co-authored-by: Lyndon White <oxinabox@ucc.asn.au>
780: Move a bunch of no_grad to ChainRules r=oxinabox a=oxinabox this is the partner to JuliaDiff/ChainRules.jl#252 It will fail til that is merged and tagged What is left is: - Types (because JuliaDiff/ChainRulesCore.jl#213) (e.g. `Colon`, `OneTo` `Channel`) - Things to which the derivative is `Zero()` not `DoesNotExist()` (e.g. `one`, `ones`, `zero`, `zeros`) - Things that felt too magic: e.g. `Base.eval` Should I bump patch version and tag a release? Co-authored-by: Lyndon White <lyndon.white@invenialabs.co.uk> Co-authored-by: Lyndon White <oxinabox@ucc.asn.au>
Requires JuliaDiff/ChainRulesCore.jl#207
@Keno 🎁
Here is several hundred methods you don't have to AD any more.
About 438 methods at last count. (though that is only counting methods added to our rules, and in many cases they are more general than the primal methods, so it probably covers a few hundred more than that)