Integration with OpenLayers

We have been using OpenLayers for displaying images in some of the ImJoy plugins (e.g.: https://imjoy.io/lite?plugin=oeway/ImJoy-Plugins:ImageAnnotator), and we have been benefiting from a well equipped 2D annotation features provided by the library.

Now I started to think whether we can combine the itk-vtk-viewer with OpenLayers, say, create a new CanvasLayer in OpenLayers, and use itk-vtk-viewer to manipulate that layer. Here is an example about d3 integration in OpenLayers: https://openlayers.org/en/latest/examples/d3.html Essentially, we need to provide a render function to get synchronized.

Could you please tell me what do you think about that? Or do you have a native way of supporting annotation in the viewer?

If that would work, I think it’s gonna be a very powerful viewer with annotation support. Annotating 2D images should work without much tweaking, and for 3D data, we can at least support plane based annotation.

@thewtex

[oeway]

Hey @thewtex Thanks for your code snippets! I finally figured it out! It was because I passed an array instead of three arguments 😭

//wrong!
camera.setFocalPoint([xf, yf, zf])
camera.setPosition([xp, yp, zp]);

//correct
camera.setFocalPoint(xf, yf, zf)
camera.setPosition(xp, yp, zp);

Now it works like a charm : https://imjoy.io/lite?plugin=https://gist.githubusercontent.com/oeway/531c2f72ff712414f2229f9f8b522464/raw/9d750604cefed030df83a86646fa537f1423915e/ImageViewer.imjoy.html

A huge thanks to you!

[oeway]

Hey @thewtex I made it! A preliminary version which synchronizes the two coordinate system:

https://imjoy.io/lite?plugin=https://gist.githubusercontent.com/oeway/531c2f72ff712414f2229f9f8b522464/raw/6598d390172ef3ad10300145b7e978e2e6753d85/ImageViewer.imjoy.html

However, while I can do camera. setParallelScale to set the scale of the viewer, I cannot find a way to shift/translate the camera, I tried:

const viewFocus = camera.getFocalPoint();
const viewPoint = camera.getPosition();
camera.setFocalPoint([viewFocus[0]-diff_x, viewFocus[1]-diff_y, viewFocus[2]]);
camera.setPosition([viewPoint[0]-diff_x, viewPoint[1]-diff_y, viewPoint[2]]);
viewProxy.getRenderer().getRenderWindow().render();

( code: https://gist.github.com/oeway/531c2f72ff712414f2229f9f8b522464/533a0226c86cc14f7995b9f0c72c71ce23a92a8e/#file-imageviewer-imjoy-html-L207-L208 ) The issue of the above code is that the entire display gone if call render(). Do you know why?

Another thing is for the Zplane, openlayers uses lower-left corner as it’s origin, but the viewer uses top-left. I had to do:

viewer.setViewMode('ZPlane');
viewProxy.updateOrientation(2, 1, [0, 1, 0])

But I think I can fix this if I find a way to set the orientation in openlayers.