Provide metadata service as separate server extension

I’m loving the metadata service inside of elyra!

I’d really like to tap into this extension without installing all of elyra’s dependencies. Right now, installing the Python package leads to the installation of a few other server extensions, such as jupyterlab-git and nbdime, because they are required in elyra’s setup.py. However, it would be nice to install this extension without these extra dependencies.

One option is to create a new repo and provide the metadata service as a separate PyPI package; the second option is to make these other dependencies optional (though this is likely a problem for the elyra distribution).

Let me know your thoughts. I’d be happy to help move this part of the code out into a separate repo if you think this make sense. Thanks!

cc @kevin-bates

[lresende]

Hi @Zsailer,

I’d really like to tap into this extension without installing all of elyra’s dependencies. Right now, installing the Python package leads to the installation of a few other server extensions, such as jupyterlab-git and nbdime, because they are required in elyra’s setup.py. However, it would be nice to install this extension without these extra dependencies.

In development mode, make install-server on the Elyra repo should do what you need.

In a release, we want to simplify the user experience for the user, and thus we drop these extensions into the JupyterLab folder which makes the pip install elyra && jupyter lab build the only thing the user needs to do and that’s very useful.

Separating the modules is definitely something we might be open to, but as we are still having a lot of changes on the metadata services, having a separate module might increase the work necessary to make quick progress, as changes on the new module need to be released to be used with the main Elyra repo.

One thing I was thinking about, at least for now, is to customize the setup.py to allow something like ELYRA_DISABLE_LAB_EXTENSIONS=true pip install elyra which then would only install the backend.

My immediate reason is that our “code-snippets” team (@jupytercalpoly) is building a stand-alone code-snippets extension on top of Elyra’s metadata service backend (i.e. using the metadata service for validating+storing snippets). Their plan is to work in a sandbox, outside of Elyra, for a little while and contribute upstream later when they make some progress.

Don’t be shy 😃 we can create a branch for the interns so they can work in isolation and in a more stable environment, which will give them some flexibility to experiment but also provide the real open source experience working with the community.

[kevin-bates]

I think there are a number of additional functional items for making the metadata service a separate extension as well:

• Namespace objects that hold schemas and define storage characteristics
• Ability to bring your own namespaces and schemas
• Proper definitions for specifying “scopes” or “storage areas” of instances (e.g., user, shared, system) or simplification of the current hierarchy
• Addition of some meta-behaviors (e.g., “system-owned”, “default-instance”)