[feature request] can module systems be supported?

Goal

I’m just starting out here, I read the docs start to finish and don’t believe such functionality exists yet. I am interested in helping tackle support for module systems (I use LMod, but traditional TCL modules would be obligatory).

I believe the best way to implement this would be some kind of addition (probably to a kit?). I don’t know how this would pan out though, because as far as I can tell you do not get consistent access to the same terminal session.

Overview of Typical Workflow

Say I have project foo/ I’m just building from my terminal.

$ mkdir gcc_7.3.0_build/ && cd gcc_7.3.0_build/
$ ml load gcc/7.3.0 # the thing I am trying to figure out how to support here
$ cmake ..

In a traditional module based workflow, if we go back to the repo root and want to test another stack I would say

$ mkdir clang_5.0.0_build && cd clang_5.0.0_build/
$ ml switch clang/5.0.0
$ cmake ..

But for the purposes of this tool, trying to add in a comprehensive listing of

1. Tracking currently loaded modules
2. Deciding whether load vs switch needs to happen
3. …

I think that what could potentially be successful is an addition to the kit options that just runs a raw command (so not actually module specific).

{
    "name": "My Compiler Kit",
    "commands": [
        "ml load gcc/7.3.0",
        "ml load openmpi/3.1.2",
        "ml load hdf5/1.10.2"
    ],
    "compilers": {
        "__comment__": "properly crafted compiler modules should set these",
        "__comment__": "doing this '__comment__' stuff for syntax highlighter xD",
        "C": "${env:CC}",
        "CXX": "${env:CXX}",
        "Fortran": "${env:FC}"
    }
}

Where each command in commands is executed verbatim (aka user at fault for invalid commands), and equally important: in order.

1. Since the modules themselves should set CC, CXX, and FC respectively, maybe instead of finding a way to make sure that commands are run for evaluating "compilers", it would be better to just introduce a “trap” e.g., "C": false to indicate to CMake Tools please do not set CC environment variable or -DCMAKE_C_COMPILER=....
   • Or can we just omit "compilers" altogether?
2. Order matters for LMod specifically, it’s a hierarchical module system. So hdf5 won’t be ‘visible’ until I load OpenMPI.
3. If you aren’t aware, what the module systems do are basically just prepend to the important variables: $PATH, $LD_LIBRARY_PATH, $CMAKE_MODULE_PATH, and friends. So they really do need to be loaded for everything: configure, build, running targets, all of it 😕

Anticipated Difficulty

The reason a CMake Toolchain file cannot be used for this is because the module needs to be loaded before CMake even starts executing. For example, the module system may change which cmake shows up in $PATH first. Even if that wasn’t a problem, the compiler modules should be setting CC, CXX, and FC which needs to happen before CMake runs.

Basically, the problem is that executing the module commands in a subshell will not affect the parent shell. So say we were talking about the configure stage. You would basically need take

$ cd ${build_dir}
$ cmake .. -DCMAKE_......

and wrap it

$ commands[0]
$ commands[1]
$ commands[N]
$ cd ${build_dir}
$ cmake .. -DCMAKE_......

all being executed in the same subshell.

Is this something a VS Code extension can do without becoming overbearingly difficult to maintain?

Other Notes/Information

• The reason I like the commands approach as to doing something module specific is there are a lot of nuances / edge cases for modules you would have to bake in for no real reason. Having commands makes it so that this feature could potentially be used for a myriad of other applications, and we wouldn’t have to be bothered by details such as the module command printing to stderr for example.
• I don’t know if supporting this on Windows consoles will be easy. For *nix you could basically just ";".join(commands) so to speak.
• I am entirely unattached to this being specific to a kit. I don’t really understand how all of the pieces fit together, but this seemed to be the right location.
• I am absolutely willing to help implement this (though I’ll probably need some hand-holding on how to actually test the extension in VSCode xD).

Thanks for being you, the CMake Guru / champion for the people. We are lucky to have you! I look forward to your thoughts, even if it’s just “that’s beyond the scope of this extension”.

P.S. I did promise you I’d “break” your extension 😉

[vector-of-bool]

Take a peek in kit.ts, which does all the kits stuff. It’s getting kind of large, so you may want to put it in a new .ts file (I don’t recommend calling it module.ts, since that is nightmare confusing 😛).

See if you can get the environment variables loaded from a module system, there you’ll go. It may take a bit of poking to figure out what the scatter/gather thing is doing where I start a dozen kit-checks in parallel, but it shouldn’t be completely inscrutable? Use proc.ts to execute subprocesses. It’ll handle a lot of the piping and stuff for you (including splitting of stdout/stderr).

Before worrying about the docs or anything I’d say to just take a stab at it and see what you can do.

Thanks!

[vector-of-bool]

Good to hear you’ve solved your issue. But do note that VSCode uses ‘unique’-style processes. If you already have a code instance running and run code <path>, it may or may not start a new VSCode window with the environment from the shell. It may re-use the same environment from the already running instances. This is why I added the VS kits rather than just asking people to start VSCode from a dev command prompt.