Differentiable finite-difference gradient transform #1476

josh146 · 2021-07-26T14:16:00Z

Context: As part of the roadmap for supporting differentiable batch execution, gradient logic will be moved out of the subclasses and into a module of pure functions. This is the first such PR; here, we move the finite-difference logic out of JacobianTape and into a new gradients package.

Description of the Change:

Adds a function qml.gradients.finite_diff_stencil, for generating the coefficients and shifts required for finite-difference rules of various order, accuracy, and form.
Adds a function qml.gradients.finite_diff for generating the finite-difference tapes of an input tape. The transform is fully differentiable, and may be differentiated to get higher-order derivatives.

Benefits: The finite-difference logic is now much more user and dev accessible, and supports higher order derivatives.

Possible Drawbacks:

Currently, when executing a device, the output is always cast to a tensor. For example, expval(), expval() -> tensor[2], and probs[0], probs[1] -> tensor[2, 2]. However, this results in ragged arrays in cases such as expval(), probs().

This has two drawbacks:

NumPy is deprecating ragged arrays shortly (!!)
Some ML frameworks don't support ragged arrays

In the current codebase, we get around the latter issue by always hstack-ing the ragged arrays when we compute the Jacobian. E.g., for N input parameters and output expval(), probs([0, 1]) -> tensor[[1], [4]], the Jacobian will be returned as a [5, N] array (the 1 and 4 dimensions of the output have been concatenated).

However, long term, we should move to treating the output as a tuple expval(), probs([0, 1]) -> tuple[tensor[1], tensor[4]]. This avoids the issues listed in the bullet points above, and also makes the jacobian computation easier; we simply return tuple[tensor[1, N], tensor[4, N]].

Related GitHub Issues: n/a

github-actions · 2021-07-26T14:16:17Z

Hello. You may have forgotten to update the changelog!
Please edit .github/CHANGELOG.md with:

A one-to-two sentence description of the change. You may include a small working example for new features.
A link back to this PR.
Your name (or GitHub username) in the contributors section.

…inite-diff

codecov · 2021-07-26T18:22:44Z

Codecov Report

Merging #1476 (49fe608) into master (e8925e5) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1476      +/-   ##
==========================================
+ Coverage   98.30%   98.31%   +0.01%     
==========================================
  Files         170      172       +2     
  Lines       12483    12579      +96     
==========================================
+ Hits        12271    12367      +96     
  Misses        212      212

Impacted Files	Coverage Δ
pennylane/__init__.py	`98.59% <100.00%> (+0.02%)`	⬆️
pennylane/gradients/__init__.py	`100.00% <100.00%> (ø)`
pennylane/gradients/finite_difference.py	`100.00% <100.00%> (ø)`
pennylane/math/single_dispatch.py	`99.43% <100.00%> (+0.01%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e8925e5...49fe608. Read the comment docs.

glassnotes

Very slick transform! 😎

Mostly minor points within, then two more general suggestions:

How user-facing is this transform? Is it something that would also be nice to have a QNode transform wrapper for so users don't have to work directly with tapes?
While there is a parameter n to set the order of the gradient, I mostly noticed n=1 and a few n=2 cases in the tests. It may be valuable to show a larger example; also, the transform can be applied to something multiple times, right? If so would be cool to see an example of that as well.

.github/CHANGELOG.md

glassnotes · 2021-07-27T12:21:29Z

pennylane/gradients/finite_difference.py

+"""
+# pylint: disable=protected-access,too-many-arguments
+import numpy as np
+from scipy.special import factorial


Oh cool, I didn't know about this function 😁 (or that it would have applications here1)

😆 actually terrifying now that I realize there is a factorial scaling in here somewhere lol

pennylane/gradients/finite_difference.py

tests/gradients/test_finite_difference.py

glassnotes · 2021-07-27T12:48:00Z

tests/gradients/test_finite_difference.py

+
+    def test_autograd(self, order, form, tol):
+        """Tests that the output of the finite-difference transform
+        can be differentiated using autograd, yielding second derivatives."""


I'm a bit confused about where the second derivative comes in since n=1 in the transform. Is it because the base tape is a JacobianTape?

Nope, it's because the device is default.qubit.autograd!

I am writing a cost function that computes the finite-difference gradient of this QNode using this transform.

Since this transform is differentiable, I can differentiate the cost function

When the autodiff framework performs backpropagation, it will end up needing to compute various d tape/d parameters as part of the chain rule. However, since the device is written using autograd, more chain rules are applied to simply get to d state/d parameters!

So it's a mixture of (a) the transform being differentiable, and (b) the device execution being differentiable.

Basically, you can differentiate any qfunc(qnode) cost function, to get the gradient d qfunc(qnode)/d parameters. In this case, since qfunc itself is the first derivative, we end up with the second derivative 😆

Co-authored-by: Olivia Di Matteo <2068515+glassnotes@users.noreply.github.com>

josh146 · 2021-07-27T13:59:35Z

How user-facing is this transform? Is it something that would also be nice to have a QNode transform wrapper for so users don't have to work directly with tapes?

At some point I would like to make it more user-friendly, but at the moment it is purely a port of JacobianTape.jacobian from a method to a transform. The plan is:

The new, differentiable batch_execute will use this transform internally when computing gradients during backpropagation.
Later, wrap this up as a qnode transform so it can also be used by users. However, this is blocked by us putting the QNode transform API on hold!

While there is a parameter n to set the order of the gradient, I mostly noticed n=1 and a few n=2 cases in the tests. It may be valuable to show a larger example; also, the transform can be applied to something multiple times, right? If so would be cool to see an example of that as well.

I have to admit, I planned to only port the code in JacobianTape.jacobian as-is to reduce my workload. It currently only supports first derivatives, and only 2nd-order center and 1st-order forward.

However.... I went a bit down a rabbit hole while porting it, and instead implemented support for arbitrary finite-diff rules 🙁

…inite-diff

josh146 · 2021-07-28T12:19:46Z

but I requested changes because I was often quite confused, and ideally we want to make the code base self-explanatory.

@mariaschuld these were good catches. I've updated the PR accordingly, hopefully it should be easier to follow now!

glassnotes

@josh146 updated version looks great, new parameter names and examples are really helpful!

glassnotes · 2021-07-28T12:27:39Z

.github/CHANGELOG.md

+  ...     qml.RX(params[2], wires=0)
+  ...     qml.expval(qml.PauliZ(0))
+  ...     qml.var(qml.PauliZ(0))
+  >>> tape.trainable_params = {0, 1, 2}


Just noticed this, why is it { } instead of [ ] for trainable_params?

Ah - trainable_params is a set! Since trainable parameters must be unique, and order doesn't matter

pennylane/gradients/finite_difference.py

tests/gradients/test_finite_difference.py

Co-authored-by: Olivia Di Matteo <2068515+glassnotes@users.noreply.github.com>

glassnotes

Awesome, looks good to me! 👍

pennylane/gradients/finite_difference.py

mariaschuld · 2021-07-29T06:05:28Z

pennylane/gradients/finite_difference.py

+def finite_diff_coeffs(n, approx, strategy):
+    r"""Generate the finite difference shift values and corresponding
+    term coefficients for a given derivative order, approximation accuracy,
+    and strategy.


This is also voluntary, but you may make future developer's life sweet and easy if you mention the equation for the coefficients here - at least to me this piece of theory is not obvious.

Have added the following underneath examples:

pennylane/gradients/finite_difference.py

mariaschuld · 2021-07-29T06:10:00Z

pennylane/gradients/finite_difference.py

+    """
+    # TODO: replace the JacobianTape._grad_method_validation
+    # functionality before deprecation.
+    diff_methods = tape._grad_method_validation("numeric")


Just to learn about this - would it return a list like ["F", "F",...]? Isn't at this stage clear that all parameters should do finite-diff?

this method also does two other things:

Returns "0" if the output is independent of the parameter

Raises an error if the parameter has grad_method=None.

This is purely to retain backwards compatibility with JacobianTape.jacobian. My main goal of this PR was that,

>>> tapes, fn = finite_diff(tape) >>> j = fn(dev.batch_execute(tapes))

should behave exactly like

>>> j = tape.jacobian(dev)

does in master 🙂

pennylane/gradients/finite_difference.py

tests/gradients/test_finite_difference.py

mariaschuld

Very nice, the code now explains itself!

The approval is of course conditioned on fixing CodeFactor issues and testing those two new untested lines, but I trust you :)

Co-authored-by: Maria Schuld <mariaschuld@gmail.com>

…inite-diff

josh146 added 4 commits July 25, 2021 23:19

more

fc67b6c

more

e57f624

more

8786425

more

3f924b2

josh146 added the WIP 🚧 Work-in-progress label Jul 26, 2021

josh146 added 8 commits July 26, 2021 23:56

more

0542bd4

add tests

9f6b53c

more tests

ce46343

more tests

1a28703

Merge branch 'master' into finite-diff

c436a42

changelog

a449fef

Merge branch 'finite-diff' of github.com:PennyLaneAI/pennylane into f…

a61b3d9

…inite-diff

add todo

e63ffbd

josh146 changed the title ~~[WIP] Differentiable finite-difference gradient transform~~ Differentiable finite-difference gradient transform Jul 26, 2021

josh146 added review-ready 👌 PRs which are ready for review by someone from the core team. and removed WIP 🚧 Work-in-progress labels Jul 26, 2021

josh146 requested review from glassnotes and mariaschuld July 26, 2021 17:54

josh146 added 3 commits July 27, 2021 01:59

fix

83bad03

fix

148cd45

fix

c3a5b41

glassnotes requested changes Jul 27, 2021

View reviewed changes

josh146 and others added 2 commits July 27, 2021 21:15

Apply suggestions from code review

dacc22a

Co-authored-by: Olivia Di Matteo <2068515+glassnotes@users.noreply.github.com>

Merge branch 'master' into finite-diff

e460d2e

josh146 added 2 commits July 27, 2021 23:39

fixes

23454fa

Merge branch 'master' into finite-diff

c855407

josh146 mentioned this pull request Jul 27, 2021

Differentiable parameter-shift gradient transform #1479

Merged

josh146 added 4 commits July 28, 2021 18:09

Update pennylane/gradients/finite_difference.py

d0d7848

more

e4cb204

Merge branch 'finite-diff' of github.com:PennyLaneAI/pennylane into f…

fa8caed

…inite-diff

suggested changes

cdae87f

suggested changes

657fec2

josh146 requested a review from mariaschuld July 28, 2021 12:20

typo

bf96beb

glassnotes requested changes Jul 28, 2021

View reviewed changes

Apply suggestions from code review

1142511

Co-authored-by: Olivia Di Matteo <2068515+glassnotes@users.noreply.github.com>

josh146 requested a review from glassnotes July 28, 2021 13:49

glassnotes approved these changes Jul 28, 2021

View reviewed changes

mariaschuld reviewed Jul 29, 2021

View reviewed changes

pennylane/gradients/finite_difference.py Outdated Show resolved Hide resolved

mariaschuld reviewed Jul 29, 2021

View reviewed changes

pennylane/gradients/finite_difference.py Show resolved Hide resolved

mariaschuld reviewed Jul 29, 2021

View reviewed changes

pennylane/gradients/finite_difference.py Show resolved Hide resolved

mariaschuld reviewed Jul 29, 2021

View reviewed changes

pennylane/gradients/finite_difference.py Outdated Show resolved Hide resolved

mariaschuld reviewed Jul 29, 2021

View reviewed changes

tests/gradients/test_finite_difference.py Outdated Show resolved Hide resolved

mariaschuld approved these changes Jul 29, 2021

View reviewed changes

josh146 and others added 7 commits July 29, 2021 15:48

suggested changes

6621b79

suggested changes

d0eb58d

suggested changes

1952277

suggested changes

781fb6b

Apply suggestions from code review

215a6d9

Co-authored-by: Maria Schuld <mariaschuld@gmail.com>

Merge branch 'finite-diff' of github.com:PennyLaneAI/pennylane into f…

5614fd5

…inite-diff

added multiplier test

49fe608

josh146 merged commit a562747 into master Jul 29, 2021

josh146 deleted the finite-diff branch July 29, 2021 08:19

josh146 mentioned this pull request Jul 30, 2021

(non)Differentiable CV parameter-shift gradient transform #1486

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Differentiable finite-difference gradient transform #1476

Differentiable finite-difference gradient transform #1476

josh146 commented Jul 26, 2021 •

edited

Loading

github-actions bot commented Jul 26, 2021

codecov bot commented Jul 26, 2021 •

edited

Loading

glassnotes left a comment

glassnotes Jul 27, 2021

josh146 Jul 27, 2021

glassnotes Jul 27, 2021

josh146 Jul 27, 2021

josh146 Jul 27, 2021

josh146 commented Jul 27, 2021

josh146 commented Jul 28, 2021

glassnotes left a comment

glassnotes Jul 28, 2021

josh146 Jul 29, 2021

glassnotes left a comment

mariaschuld Jul 29, 2021

josh146 Jul 29, 2021 •

edited

Loading

mariaschuld Jul 29, 2021

josh146 Jul 29, 2021

mariaschuld left a comment •

edited

Loading

Differentiable finite-difference gradient transform #1476

Differentiable finite-difference gradient transform #1476

Conversation

josh146 commented Jul 26, 2021 • edited Loading

github-actions bot commented Jul 26, 2021

codecov bot commented Jul 26, 2021 • edited Loading

Codecov Report

glassnotes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josh146 commented Jul 27, 2021

josh146 commented Jul 28, 2021

glassnotes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

glassnotes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josh146 Jul 29, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mariaschuld left a comment • edited Loading

Choose a reason for hiding this comment

josh146 commented Jul 26, 2021 •

edited

Loading

codecov bot commented Jul 26, 2021 •

edited

Loading

josh146 Jul 29, 2021 •

edited

Loading

mariaschuld left a comment •

edited

Loading