ceylon assemble

Currently ceylon fat-jar is a great assembly tool that lets you run a Ceylon app using javac. It has one limitation: it implies --flat-classpath, and doesn’t use JBoss Modules for classloader isolation.

We need a similar ceylon assemble command that similarly produces a fat jar, but with “russian doll” packaging, and which uses JBoss Modules for classloading.

This, together with #5955 satisfies the need for “assemblies”.

[quintesse]

Ok, I pushed my implementation of this to a branch so people can comment on what I created before merging it with master.

Introduction

So the most important part of this design is the realization that if we’re talking about putting a bunch of .car (and .jar) files in a single file and calling it an Assembly then basically assemblies are zipped/jarred repositories.

From that came the idea that if they are “basically zipped repositories” why not make them actual repositories? Meaning that zipped repositories would be first class citizens of the CMR. So that’s the first pillar:

Zipped repositories

So the Ceylon now support a new special kind of Assembly Repository that lets it look up modules in a zip or jar file. The syntax for the repository URI is the following:

assembly:path/to/a/zip/or/jar/containing/modules.zip[!subfolder]

So using the helloworld example code from the samples folder in the Ceylon distribution we can now literally do:

$ ceylon compile com.example.helloworld
$ cd modules
$ zip -r ../modules.zip *
$ cd ..
$ rm -rf modules # just to show the module isn't read from the modules folder
$ ceylon run --rep assembly:modules.zip com.example.helloworld
Hello, World!

By default Ceylon searches for modules in the root of the zip/jar file, you can change this behaviour by adding the name of a sub folder to search to the end of the assembly path separated by a exclamation mark (!). So to do the same as above a bit more simply we can do:

$ ceylon compile com.example.helloworld
$ zip -r modules.zip modules
$ ceylon run --rep assembly:modules.zip!modules com.example.helloworld
Hello, World!

Of course we’re not supposed to go around manually creating our own zip files, but it certainly an interesting feature to have and it’s the basis for the second pillar:

Assemble an assembly

Instead of creating your own zips and perhaps having to figure out which modules to include and which to leave out you can use the new ceylon assemble command that was specifically created for that purpose. In its most basic form it works just like the ceylon fat-jar command:

ceylon assemble <module>

This creates a module-version.cas file in the local directory.

NB: Yes, a CAS file. Where JAR stands for Java ARchive, and CAR stands for Ceylon ARchive CAS stands for Ceylon ASsembly, logically

This CAS file by default will only contain the module mentioned in the assemble command together with the dependencies explicitly mentioned in its module descriptor and the explicit dependencies of those dependencies etc.

So using this new command the above example now becomes:

$ ceylon compile com.example.helloworld
$ ceylon assemble com.example.helloworld
$ ceylon run --rep assembly:com.example.helloworld-1.0.cas!modules com.example.helloworld
Hello, World!

But of course this isn’t much better. But luckily the assemble command isn’t just a simple archiver, it doesn’t just zip the modules, it also adds some meta-information that the ceylon run command can use to make executing an assembly a lot easier. So that’s where we come to the third pillar:

Running assemblies

The ceylon run command has been improved and knows about assemblies. There’s a new --assembly option that’s the equivalent of Java’s -jar option and it works like this:

ceylon run --assembly <path-to-assembly>

So again using the example above we can now do the following:

$ ceylon compile com.example.helloworld
$ ceylon assemble com.example.helloworld
$ ceylon run --assembly com.example.helloworld-1.0.cas
Hello, World!

Now this is much simpler! The assemble command has stored some important information in the CAS file’s MANIFEST.MF telling the run command where to look for the modules and which module is the “main” module that should be executed.

NB: in all the above cases the assembly is run using JBoss Modules with full module isolation. The only difference from the norm is that we have a new kind of repository.

So these are the basics for creating and using some very light-weight assemblies. They are as small as possible and can easily be distributed and used in places where They could possibly become Ceylon’s WAR files where an application server could provide a single Ceylon environment where multiple application assemblies could be installed.

But what if you want to use them in places where you do not have Ceylon installed? Well we can do that too.

Stand-alone assemblies

There are actually two ways of creating stand-alone assemblies that don’t need a local Ceylon installation to run. The first option is very similar to the situation we have when using fat jars:

ceylon assemble --include-language <module>

This creates an assembly that, besides the main module and its explicit dependencies, also includes the Ceylon language module and the minimal set of Ceylon supporting system modules to be able to run with only Java installed on the system. But like fat jars this means running with a flat classpath and with a dynamic meta model. But it works and results in an assembly that’s only somewhat bigger (~2MB).

$ ceylon compile com.example.helloworld
$ ceylon assemble --include-language com.example.helloworld
$ java -jar com.example.helloworld-1.0.cas
Hello, World!

But what if you want a stand-alone assembly but still want to use the full module isolation that JBoss Module provides? Well for that we have a different option:

ceylon assemble --include-runtime <module>

This creates an assembly that, besides the main module and its explicit dependencies, also includes as much of the Ceylon system modules to be able to run using JBoss Modules. It does result in an assembly that’s quite a bit bigger (~7MB).

$ ceylon compile com.example.helloworld
$ ceylon assemble --include-runtime com.example.helloworld
$ java -jar com.example.helloworld-1.0.cas
Hello, World!

Advanced topics

• Like for the fat-jar and run commands you can specify which toplevel declaration of an assembly’s main module should be executed:

   ceylon assemble --run <toplevel> <module>
  

• If an overrides file was used while creating the assembly then that file will be included in the assembly and used when executed:

   ceylon assemble --overrides <overrides_file> <module>
  

Caveats

There is no support for Maven dependencies. (Things will work when run using ceylon run but the dependencies won’t be part of the assembly itself. Not sure if that’s a realistic future expansion)

Comments?

So what do you guys think? @gavinking @FroMage @tombentley @chochos @davidfestal @bjansen @lucaswerkmeister @jvasileff I think this does everything that was expected for this issue while perhaps adding some additional interesting possibilities. But perhaps you guys can think of some improvements? Maybe you don’t agree with some of the decisions I made?

Edit: changed # to !

[quintesse]

Closing this now and opening a new issue for Maven support,