[bug] Declaring provides_jacobian=True capability breaks device with current master

I have observed (with great delight!) that tape mode now allows observables returning non-standard types beyond floats, e.g., dicts or classes.

I am also pleasantly surprised, this was not something we intended/expected when coding it 😆 Glad to know it works!

We should probably add tests for this behaviour, to make sure we don’t accidentally break it.

_Originally posted by @josh146 in https://github.com/PennyLaneAI/pennylane/issues/1109#issuecomment-785616046_

It seems that this “feature” is now broken in the latest master.

More precisely the line here https://github.com/PennyLaneAI/pennylane/blob/1a13bef8adfa85a86a7567f41294b3f73285a294/pennylane/_qubit_device.py#L299 makes it impossible to have an observable that returns something that is not castable to float.

Would it be possible to have this line in a try/catch block and return either the plain result or a dtype=object array in case result cannot be cast to float?

[cvjjm]

Hi, as far as I can tell the test is still in the “vandalized” state in the current master. Is there a chance we can get this functionality restored in the next release?

[cvjjm]

Oh! I guess my take home form the above is then:

1. Checking whether the device declares provides_jacobian=True is not a good indicator for whether the jacobian actually came from the device. I guess the wrapping in a tuple should at least be moved to inside this else statement: https://github.com/PennyLaneAI/pennylane/blob/bcd837b1ed1187895c327abfe62aea71fbeba02f/pennylane/interfaces/autograd.py#L220-L232

2. Is there a more robust way of checking whether the jacobian actually came from a device and thus needs to be wrapped? Maybe the shape or type of the jacobian can be used to decide whether wrapping is needed? This would even allow some room for error on the plugin developer side, which would be nice.

3. There needs to be a test in core PL with at least a “fake” device that implements a “fake” device jacobian feature (like returning a fixed number) so that such things can be caught. Currently there is no device that ships with PL that implements the device gradient, isn’t there?

I would highly appreciate a 0.23.X bugfix release, not because I cannot wait until the 21st but because it is nice having at least some v0.23.X version that I do not need to black list because of this 😃