Notes from meeting with Tim

I met with @timkpaine today to see they are up to at JP Morgan in exposing SQL databases and virtual filesystems in JupyterLab.

Currently, they are using something like this multi contents manager (I can’t remember if they are using that exactly or an internal fork of it). My impression from it is that the Jupyter server supports only one contents manager. So this project lets you combine multiple contents managers into one contents manager. You still specify a default that shows up normally, so the filebrowser looks the same in JL. But, you add some special root paths, like /s3:// to redirect to other content managers under the hood.

Then in JL, using https://github.com/youngthejames/jupyterlab_filetree/, they display these other roots in different browsers in JL. They wanna use this to make a “SQL” contents manager that shows different tables as files.

My takeaways was that building a good tree view UI is a PITA and so the data registry should be the place we do that and then others don’t have to re-implement it. Tim pointed out a bunch of subtle behavior that they copied from the Finder on mac in terms of single and double clicking on files and folders. Also it would be nice to have both a tree view and a multipanel view the user could switch between.

We should support the contents manager API in the data registry, with the end goal of replacing JL’s existing file browser with the data browser. A bunch of things need to happen before that’s feasible including:

• Recreate right click menu from filebrowser in data register, move views to “open with” submenu.
• Add priority and icons for views so we have a default one
• Fix tree view UI so it isn’t as ugly with “show/hide” button.
• Add mutation features, including renaming, adding folder, uploads, and drag and drop.

I also got a nice demo of perspective. In our world, it functions a bit like voyager or plotly. However, it is much more performant for large datasets. So we discussed how it could use the data registry mimetypes to represent different pointers to different types of data like parquet, and use that.

[quigleyj-mavenomics]

If you’re interested, I’ve written a tree control to address some of these use cases that’s being used internally. We’re about to open-source a larger dashboarding project that encompasses it, but I can separate out the tree control and open-source that sooner.

There’s some rough edges that need to be cleaned up, but it was designed to address a lot of the shortcomings of other tree controls (inflexibility, no async support, bad drag UX, no tree-to-tree, etc). It’s written in React, but wraps nicely in a Phosphor widget.

Here it is in action, with a backend making updates to the tree asynchronously (in this case, doing some tree pruning to remove useless containers)

[telamonian]

@quigleyj-mavenomics What’s the status of React tree component? Have you open sourced it anywhere?

For context, I’m starting to work on a refactor of the filebrowser UI and underlying stack in jlab core. One of my goals is to make the file listing a tree view. I’m deciding right now whether to use an existing React component for the tree, or if I should roll a new one. And then I remembered this thread, and thought I’d check in with you