Custom control over segment colors

Currently, users have extraordinarily flexible control over the rendering of image layers, thanks to custom fragment shaders. It would be great to have similar flexibility for segmentation layers. Is that a realistic request? Right now, I have to choose between custom rendering (with image layers) or the selection/deselection UI (segmentation layers). Is it possible to have both?

If full flexibility is not possible in the near term, then the ability to specify a single color for all segments in a layer would be appreciated. This feature request was discussed in a previous thread. In the past I’ve used the saturation trick, but now I want to create multiple layers with distinct uniform segment colors (for millions of segments).

More brainstorming… Perhaps a simple compromise solution is to support a color table. I have millions of segments, so I can’t list each one via a segmentColors mapping. But it would be possible for me to custom-tailor my segment IDs to target a color table of N colors (assuming color = table[segment % N]). Bonus points if the color table can be defined by the user, which would also provide an implicit solution to the “single color” request above. But even a hard-coded color-table would still be useful (assuming it contains a reasonable variety of colors to choose from).

[jbms]

I am planning to implement this feature request when I have chance, hopefully soon.

Regarding the performance issue with uint64 image layers, though:

Neuroglancer has two ways to represent volumetric data in GPU memory: uncompressed (which supports all data types), and compressed_segmentation (which supports uint32 and uint64). If the data is originally in compressed_segmentation format, then Neuroglancer always just uses it directly without any transcoding. If the data is in any other format (e.g. raw, jpeg, compressed via an n5 compression method), Neuroglancer first decodes it to uncompressed, then in certain cases may re-encode as compressed_segmentation. Currently, re-encoding to compressed_segmentation only happens when using a segmentation layer, because compressed_segmentation is only effective if the same ids are repeated within local blocks. The assumption is that the data for an image layer is likely to be continuous and compressed_segmentation won’t be helpful.

Uncompressed uint64 data takes up a lot of memory, which means only a small region can fit in the cache. That may be the reason for the bad performance that you are observing.

If you export your uint64 label volumes as neuroglancer precomputed with compressed_segmentation encoding, that should be a lot faster since it will greatly reduce the memory required, and will also avoid the need to transcode the data as it is downloaded.

[jbms]

I would be happy to include this change. You would add something like:

builder.addUniform('highp uint', 'uSegmentId', 2);

here:

https://github.com/google/neuroglancer/blob/60c8c036d202cecc7bc09e18b07dc86757bef200/src/neuroglancer/skeleton/frontend.ts#L75

And then make sure to set that uniform when running the shader here (will need to pass in the segment id):

https://github.com/google/neuroglancer/blob/60c8c036d202cecc7bc09e18b07dc86757bef200/src/neuroglancer/skeleton/frontend.ts#L243

Then add some code to be included elsewhere in the shaders to convert uSegmentId into a uint64_t (which is a custom type defined by neuroglancer in shader_lib.ts).