Publish smaller packages

CesiumJS is an extensive library with a lot of functionality that, depending on the app or use case, goes unused. Installing the cesium npm package also syncs third party libraries which may go unused. All features are tied to the monthly Cesium versioning, whether or not anything has changed.

Many users should continue to use the “traditional” CesiumJS releases, but there are also many use cases that would benefit from ingesting smaller, more specific packages.

See this community forum thread for some seeds of this discussion.

• Maintain the traditional combined release as before. There may be some internal refactoring, but from an end user standpoint, the cesium npm package and release would remain as-is
• Maintain all code from within the same repo. We don’t want to introduce additional development burden with additional repos or submodules
  • workspaces are built into npm for use cases such as this. It allows dependency management to work similar to before with all modules installed to one top-level node_modules directory
  • All packages would continue to share the same tooling
• Publish individual packages along architectural boundaries. In theory, there should be nothing stopping us from breaking up and publishing code as-is with some additional configuration, however due to some interconnections in the architecture, this may require some refactoring so that we are not “cheating” by using private APIs.
  • Each new package would be able to be published individually
  • Each new package would only require relevant dependencies-- This means dependencies would only be synced if they are strictly needed, rather than the approach we take now where all dependencies are installed regardless
  • Each new package would follow semver, while traditional CesiumJS release could remain with its monthly versioning strategy
• Most major JS projects implement a variation of this (React, Babel, BabylonJS, Angular…)
• While not needed for an initial implementation, eventually, this could allow us take advantage of tooling for dependency graphs and more, allowing us to do things like run only the unit tests relevant to a change
  • lerna is a tool recommended by npm for managing monorepos, also nx for more advanced use cases

A few use cases:

• Widgets
  • CesiumViewer and other Widgets work well for simple use cases that want to get something really basic up quickly. But a more sophisticated app does not need them, and they can actively be a hindrance as they bring in many static assets.
  • Separating widgets would isolate the dependency on knockout, which is generally incompatible with other UI/Component stacks, and itself can be a security issue. Notably, the basic CesiumWidget does not rely on knockout.
• Core/Math
  • Could be used by gltf-pipeline, where we depend on CesiumJS for utilities but don’t use it for rendering (gltf-pipeline is also a dependency back in CesiumJS, which is currently a circular dependency).
• KML, GeoJson/TopoJson, and Google Earth support each rely on specific dependencies that are not used elsewhere in the API

[ggetz]

Code from Scene or Renderer would end up in cesium/engine, at least for this first iteration

Yes. I think it would be cleaner to move source files directly into this directory structure rather than having scripts copy or reference code from Source, but let me know if that’s a concern.

Future iterations might further divide cesium/engine into smaller packages, perhaps separating graphics/geospatial math from rendering details

Yes, and this would be split along architectural boundaries such that units of code which require additional dependencies are separated into packages. To do so, we would increment a major version to denote the breaking change of modules moving to a separate package, and potentially cesium/engine would then depend on these new packages (for instance, if we publish cesium/core).

[kring]

@kring might have some thoughts here

I don’t have too much to add. Since you’re planning to stick with one repo (makes sense to me), there’s not a lot of cost to extra libraries, so generally going “too fine” is better than “too coarse” IMO. For example, this long-ish list of libraries in cesium-native: https://github.com/CesiumGS/cesium-native#card_file_boxlibraries-overview

(and that’s even though CesiumJS does a lot that cesium-native doesn’t, like all the GeoJSON / KML / Google Earth stuff Matt mentioned doesn’t exist at all in native)

But it also seems reasonable to me to start by breaking out some coarse pieces - even as coarse as engine and widgets, potentially - with the plan to break each of them up further into smaller pieces in the future. I don’t think it will be too hard to maintain (temporary) backward compatibility during this evolution.