🐱 Neural Network from a very scratch - backpropagation, gradient descent, activation functions
install anaconda, so you have conda
available in shell
# To activate this environment, use
#
# $ conda activate nn
#
# To deactivate an active environment, use
#
# $ conda deactivate
export env settings to .yml:
conda env export --from-history > environment.yml
softmax derivative:
we need two derivatives of softmax - with respect to
let's say
apply the quotient rule of differentiation
so then
derivative of sum
now another derivative of
once again let's apply the quotient rule of differentiation
first term in numerator is
then there's a sum, from which all terms are
So