Figure out the best way to handle our git-lfs build going forward

🧰 Task

Our git-lfs setup is already causing headaches due to GitHub’s bandwidth limitations on LFS. We need to consider sustainable options going forward.

1. Based on our burn rate, 10x the bandwidth should cover us easily and would cost $600/y, which is not that terrible. So we (ie CZI 😂 Thank you!!! 😂🙏) could just fork it out.
2. We could urgently add more dynamically generated screenshots and remove a large number of objects from LFS, which could decrease our burn rate, while at the same time ensuring our documentation is up to date.
3. The burn rate should be very small if we can correctly implement caching. The build-docs workflow has some breadcrumbs on how to implement that here. ie in theory it’s already implemented but clearly it’s not working correctly.
4. It looks like some non-docs builds are running into the lfs quotas, which should never happen. This I don’t understand currently (it doesn’t look like the checkout action has lfs enabled without adding the with: lfs: true parameter to the action), but it’s definitely happening, see e.g. this line in a failed build.
5. We can drop LFS and go back to “GHIFS”, ie having images served by GitHub links. @neuromusic has strongly opposed this approach because it’s not officially supported by Github, though. It’s a fair objection but the LFS approach presents its own headaches. (One could say that Github already doesn’t really support LFS! 😂)
6. On that note, it also makes no sense that the same bandwidth with a git submodule stored in github would be completely fine. We could try contacting github to ask them whether they can update their LFS bandwidth policy to something more reasonable.

[jni]

Ah, looks like it’s to test pytest --pyargs napari while not having any napari in the local directory. We could/should fix this by putting our code in a src/ directory, but in the meantime, perhaps we can come up with a more lfs-compatible solution? For example, checkout, install, then delete.

Come to think of it, maybe going to point 3 above, caching is working in the build-docs (which would explain why build-docs didn’t fail), and all our bandwidth was consumed by this pip install line???

[jni]

Indeed, no build-docs or build-and-deploy docs actions have failed:

https://github.com/napari/napari/actions/workflows/build_docs.yml

So #4114 is not the right way forward, and we need to fix this ubuntu-latest action.