How to reuse previous compilations?

As far as I can see, test dependencies only allow you to reuse compilations for multiple tests within a single run of reframe, not between runs, is that right? I can see why, but is there an easy way to reuse a binary built in a previous run? (assuming I have some way to know its actually the correct binary, which is not trivial.)

What I’ve come up with so far is:

1. Define a build system which doesn’t do anything:

class NoBuild(BuildSystem):
    def __init__(self):
        super().__init__()
    def emit_build_commands(self, environ):
        return []

1. Create a CompileOnlyRegressionTest with a run_before('compile')-tagged method which

• checks for a binary in a builds tree.
• if it exists, copy it to the stage directory then set the build_system to NoBuild()

Just wondering if there’s an easier way!

[victorusu]

@sjpb, I have played a bit with this sometime ago. I have shared this gist to show you more or less the concept.

The class that runs GROMACS is not shown. But it can be easily created from the GROMACS test

The downside of this gist method is that it relies on an external tool like EasyBuild. I also have played with Spack and I have a similar set of classes. The concept, though, does not require any tool. I have also a test that compiles GROMACS using the CMake build system, but I haven’t combined it with the GROMACS test. It should be simple, though.

[vkarak]

Here is a possible implementation for this CachedRegressionTest (I haven’t tested it):

class CachedRegressionTest(rfm.RegressionTest, special=True):
    def compile(self):
        if not build_in_cache():
            super().compile(self)
        else:
            self.sourcesdir = fetch_builddir_from_cache()
            return

    def compile_wait(self):
        if not build_in_cache():
            super().compile_wait(self)
        else:
            return

    def run(self):
        if build_in_cache():
            self._copy_to_stagedir(os.path.join(self._prefix,
                                                self.sourcesdir))

        super().run(self)

It would also be a lot easier to debug both this type of code and issues with the compile not producing what reframe expects if there was a way to stop reframe deleting & recreating the state dir - then at least the actual “compile” would be really quick. But I guess that’s a separate feature request.

Actually, there is this possibility internally, but not exposed. I think that’s a valid feature request and we could implement it easily now that the new configuration mechanism is in place. What we usually do for debugging such issues is to switch to the stage directory of the failing test and manually change the build script generated by ReFrame, then port back these changes to ReFrame.

Have you considered using EasyBuild or Spack from ReFrame to compile GROMACS? I know that @victorusu has experimented quite extensively with that.