Interactive View Screenshots in Jupyter Notebook

I can use the code below in a jupyter notebook to create an interactive view embedded in the notebook:

from vtkplotter import *
from brainrender.scene import Scene

embedWindow('panel') 
scene = Scene(jupyter=True) # Create a scene

scene.add_brain_regions(["XII"], colors="ivory", alpha= 0.2, use_original_color=False)

scene.render(interactive = True) # <- this wont actually render things in a notebook

vp = Plotter(axes=0)
vp.show(scene.get_actors())

I can take a screenshot of the brain, but I am not able to update the camera for a different view:

scene.take_screenshot() #Image with camera position at 0 1 0

# position 1
bespoke_camera = dict(
    position = [70000, -30000, 50000], # yaw, pitch -30000 or -80000, roll 
    focal = [6587.835, 3849.085, 5688.164],
    viewup = [0, -1.2, 0],
    distance = 61226.68,
    clipping = [47007.899, 79272.88],
)

scene.render(interactive=False,camera=bespoke_camera) #This should move the camera and re-render
scene.take_screenshot() #Image is the same as first image

I tried a bunch of permutations of this, and could not get an altered view while using jupyter. From what I can tell, the problem seems to be that the renderer object created with the scene method does not always communicate with the renderer object created for Jupyter. somewhere in this process the camera is always reset to the starting position.

I made the following function to implement camera rotations in the vtkplotter object that is embedded in jupyter. This lets me alter the camera and take new images:

#Modified from https://vtk.org/Wiki/VTK/Examples/Python/Screenshot
import vtk
from brainrender.Utils import camera

#Screen Shot Methods
def take_screenshot2(vp,cam):

    #Change Camera
    camera.set_camera_params(vp.renderer.GetActiveCamera(),cam)
    vp.window.Render()

    w2if = vtk.vtkWindowToImageFilter()
    w2if.SetInput(vp.window)
    w2if.SetInputBufferTypeToRGB()
    w2if.ReadFrontBufferOff()
    w2if.Update()

    writer = vtk.vtkPNGWriter()
    now = datetime.now()
    date_time = now.strftime("%m%d%Y_%H:%M:%S")
    writer.SetFileName(date_time)
    writer.SetInputConnection(w2if.GetOutputPort())
    writer.Write()

#Now I can move the camera and get a screenshot

take_screenshot2(vp,bespoke_camera)

What I really wanted to do was be able to interact with the 3D brain, set it how I liked, then take a high quality screenshot. These functions below allow me to grab the camera position of the interaction window, and then take a screenshot using that same camera:

def interactive_screenshot(vp):
    new_cam=get_camera_position(settings.notebook_plotter.camera)
    take_screenshot2(vp,new_cam)

def get_camera_position(camera):

    camera_pos = dict(
        position = camera['position'],
        focal = camera['focalPoint'],
        viewup = camera['viewUp'],
        distance = camera['distance'],
        clipping = camera['clippingRange'],
    )
    return camera_pos

The method above isn’t perfect. I’m only partially transferring options from the interactive view camera, so the screenshots look 95% the same (the screenshots seem to have a smaller FOV), but it’s a start.

What would you think of adding this functionality? It may be nice if the jupyter flag you give scene was used to determine which renderer should be used to take a screenshot, but I’m not sure if the scene object has a reference to the vp object to do this.

[FedeClaudi]

hey, this is the example from Ben: https://github.com/brainglobe/brainrender/issues/111#issuecomment-706398892 and the notebook is the one you’ve seen already: https://github.com/brainglobe/brainrender/blob/master/examples/notebook_workflow.ipynb

I’ve recently had some problem with using itkwidgets, but I’m sure that we can fix it. It’s a very exciting development and probably the way to go for notebooks in the future

[FedeClaudi]

That’s awesome! Would you might giving me a shout once it’s implemented in vedo so that I can update the code and docs for brainrender please?