Reading in DVID volumes

🚀 Feature

We’ve been chatting with the FlyEM folks at Janelia about putting their data in napari, whether it would scale, the various ways to interact with it, etc. As a first step, it would be great to build a proof of concept accessing a DVID volume as a dask array. @DocSavage shared the following DVID endpoint URLs with me as a first pass, with the caveat that they might move in ~2-3 months. But it should be enough to get us started.

---

Here are the relevant endpoints to access DVID directly:

MUSHROOM BODY dataset – grayscale only https://emdata.janelia.org/api/repo/ee789/info This will give you JSON describing the metadata for the Mushroom body dataset, which is 7615 x 11139 x 14655 voxels. So it’s a reasonable size.

For the grayscale (uint8blk) you can access documentation for the HTTP API here: https://emdata.janelia.org/api/node/ee789/grayscale/help If you scroll down to the /raw or /blocks endpoint, that will let you download the raw data.

There’s a JPEG (3D) compression version with multiscale under the names grayscalejpeg, grayscalejpeg_1, … grayscalejpeg_7.

7 medulla column (version 82252). If you do http://emdata.janelia.org/api/repo/82252/info you will see the metadata for it. It’s a bit smaller at about 5k x 4.5k x 8k. The grayscale is called grayscaleview, grayscaleview_1, etc.

Segmentation is available under the name “groundtruth”. You can see the REST API for that type of data here: https://emdata.janelia.org/api/node/82252/groundtruth/help

Like above, you can use /raw or /blocks endpoints to grab the data. Pseudocolored segmentation: https://emdata.janelia.org/api/node/82252/groundtruth/pseudocolor/xy/512_512/2500_2600_4000 The matching grayscale at high-res: https://emdata.janelia.org/api/node/82252/grayscaleview/raw/xy/512_512/2500_2600_4000

---

The first step would be to read those docs, use da.from_delayed to build an array from DVID endpoints, and then produce a usage example in the examples/ folder that reads in that data.

[hungyiwu]

I made a working prototype for the first data set and demo’ed it with the following GIFs:

3D volume view

3D layer view

here’s the repo https://github.com/hungyiwu/napari_issue_789

what would be a good way to wrap it into the examples/ folder?

[emmazhou]

@jni @hungyiwu made a plugin! https://github.com/emmazhou/napari-dvid

this was almost entirely cannibalized from some sample code @DocSavage sent me 😃

try it out and lemme know if it works for you!