question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Plugin Broker refactoring

See original GitHub issue

This issue is to collect the various open design-level issues for the plugin broker and discuss the path forward.

Current Issues

Current broker functionality

The current version of the plugin broker (v0.20), broadly, does two things:

  • Compute workspace metadata changes necessary to run the requested plugins (e.g. env vars, sidecars, etc.)
  • Download extension artifacts so that Theia/VSCode plugins can be run.

This is done in one image (ignoring the init broker for now); the broker takes plugin FQNs, downloads meta.yamls, converts those meta.yamls to Che Plugins, and downloads their artifacts.

Proposed changes

The plugin broker should be refactored into two images with separate concerns;

  • One image (a metadata broker) is only responsible for computing Che Plugins from meta.yamls and returning that information to the Che server so that the workspace can be provisioned
  • One image (an “artifact” broker) is only responsible for ensuring any files needed for plugins are present in their appropriate folders (/plugins or /plugins/sidecars)

A proof-of-concept implementation is available in my fork: https://github.com/eclipse/che-plugin-broker/compare/master...amisevsk:refactor-brokers?ts=4

These changes would address the issues listed above

  • Plugin broker as separate service shared among workspaces: Separating out the metadata actions is a necessary step in having a shared “plugin broker” service; the artifact broker will always have to exist, as something has to mount /plugins to download artifacts. The metadata broker could easily be adapted into a library or service that runs separate from workspace starts/stops
  • REST backend/merge plugin and devfile registries: The metadata broker could be integrated into the plugin-registry REST backend, allowing the registry to precompute/cache Che Plugins without running a separate image.
  • Broker’s role as a library in Che Operator: The adapted brokers are API compatible with the unified broker, so the metadata broker can be used as a library. It also avoids the need to specify “only metadata actions” when running the broker, since the metadata broker is always “only metadata actions”
  • Merge plugin brokers into a single image: The goal of a single image has issues with the other goals listed above; the metadata broker, if integrated into e.g. the registry, could be removed, leaving us with one image as hoped
  • Plugin broker should only make necessary changes to plugins: Since the refactor removes the init broker, it would be fairly simple to add this functionality to the artifact broker.

Additionally, one improvement with this approach is that it removes the clumsy way we run the broker in ephemeral workspaces (run the entire broker and download everything twice). The metadata broker can run separately to get the workspace configuration information, and then the artifact broker can run as an init container on the workspace pod.

Alternatives

One goal would be to get rid of the plugin broker altogether. This would likely involve reworking Theia to do this fetching at startup. Even in this case, the refactor above would be useful, as it would easily allow separate deprecation of broker functions (i.e. we could deprecate/move the metadata broker to the registry and deprecate the artifact broker when Theia has this functionality).

Summary

Workspace startup with and without changes (ephemeral workspace to show worst-case process): broker-diagram

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:6
  • Comments:12 (12 by maintainers)

github_iconTop GitHub Comments

1reaction
amisevskcommented, Dec 4, 2019

The relevant PRs are finally open:

Changes can be tested by starting a Che server using the CI image eclipseche/che-server:15386 and adding env vars:

CHE_WORKSPACE_PLUGIN__BROKER_METADATA_IMAGE=amisevsk/che-metadata-plugin-broker:dev
CHE_WORKSPACE_PLUGIN__BROKER_ARTIFACTS_IMAGE=amisevsk/che-artifacts-plugin-broker:dev
0reactions
nickboldtcommented, Jan 21, 2020

Moved to milestone 7.8 as that’s where it was delivered. “Backlog” makes it seem like it wasn’t done. 😃

Read more comments on GitHub >

github_iconTop Results From Across the Web

refactor broker plugins with new tracing base plugin · DataDog/dd ...
refactor broker plugins with new tracing base plugin · Run details · Usage · Workflow file.
Read more >
4 Common Mistakes You Make in Refactoring | by Juntao Qiu
4 Common Mistakes You Make in Refactoring · 1. Make changes without tests · 2. Restructure instead of refactoring · 3. Dedicated Refactoring...
Read more >
Design Patterns - Refactoring.Guru
Check out our ebook on design patterns and principles. It's available in PDF/ePUB/MOBI formats and includes the archive with code examples in Java,...
Read more >
Migration approaches - Amazon MQ - AWS Documentation
The first step in migrating your commercial message broker to Amazon MQ is determining the ... Rehost; Replatform; Refactor (re-architect); Phased migration ...
Read more >
10 JetBrains Extensions to Fight Technical Debt
Refactoring & tech debt tools. 1. RefactorInsight ... Plugin has possibility to define custom user tokens for commented line highlighting.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found