Upgrade null.qubit to the new device API

[timmysilv]

Context:
null.qubit doesn't quite do everything as it should, so it needs an upgrade of sorts.

Description of the Change:
Port null.qubit to the new Device API. The diff for the device file can be tough to parse, so let me share some context for what I did. In the fist commit, I just copy-pasted the contents of default_qubit.py, then replaced all actual computation methods with dummy methods (eg. _simulate(), _vjp(), etc.). It's probably best to filter out that commit using the github UI when reviewing the changes. I'd even recommend reviewing commits 2+3 together, then commits 4-end together as well, just so it's clear to see what I've actually done.

As well, I defined ClassicalShadow.shape according to what it returns with default.qubit. While testing, I found that it ignores batching entirely. We might want to raise an error for that, but for now I'm just raising the error in null.qubit when it tries to go down that road.

Benefits:
null.qubit does what it's supposed to!

Possible Drawbacks:
NullQubit.compute_vjp doesn't work in all cases, particularly with jax. this can be improved in the future, but the reason appears to be that jax is doing some smart stuff behind the scenes with inputs and outputs that I'm not anticipating

[sc-51495]

[github-actions]

Hello. You may have forgotten to update the changelog!
Please edit doc/releases/changelog-dev.md with:

• A one-to-two sentence description of the change. You may include a small working example for new features.
• A link back to this PR.
• Your name (or GitHub username) in the contributors section.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.65%. Comparing base (f9bdb32) to head (1d991e5).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5211      +/-   ##
==========================================
- Coverage   99.65%   99.65%   -0.01%     
==========================================
  Files         399      399              
  Lines       36901    36703     -198     
==========================================
- Hits        36774    36575     -199     
- Misses        127      128       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[albi3ro]

Potentially worth seeing how this PR would change with #5200 .

[albi3ro]

Note: I replaced null qubit preprocessing with default qubit preprocessing.

Some initial results from running:

n_wires = 40
n_layers = 5

shape = qml.StronglyEntanglingLayers.shape(n_layers=n_layers, n_wires=n_wires)
rng = np.random.default_rng(seed=1239594)
params = qml.numpy.array(rng.random(shape))

@qml.qnode(dev, diff_method="parameter-shift")
def circuit(params):
    qml.StronglyEntanglingLayers(params, wires=range(n_wires))
    return [qml.expval(qml.Z(i)) for i in range(n_wires)]


def cost(params):
    out = circuit(params)
    return qml.math.sum(out)

qml.grad(cost)(params)

Half of the time is in QuantumScript.hash, and 33% is in convert_to_numpy_parameters .... hmmm... We might want to make caching only default with higher order derivatives.

[timmysilv]

also plenty of time spent evaluating the batch size : ) So.. this is null.qubit doing what it does best, right? helping us benchmark and improve PennyLane-at-large?

[albi3ro]

Still needs a changelog entry, but looks great to me!

I'm a little puzzled by the failing tensorflow default qubit test, but I can't imagine it would be because of anything that happened in this PR, so I just retriggered it.

[PietropaoloFrisoni]

Great, I'll start reviewing it today. Realistically, it's going to take a few days for me to properly go over this PR, but I'll do it asap

[PietropaoloFrisoni]

Amazing! Thank you for requesting my review (I actually 'studied' this new part of the code and definitely learned a lot).