"Clone Module" API Idea and Request for Comment

As designs grow larger and larger, we are starting to hit fundamental performance problems in both Firrtl and Chisel. There are many techniques to deal with this, and one useful one I think we should consider is allowing users to explicitly “clone” Modules. This would actually partially address both #472 and https://github.com/ucb-bar/firrtl/issues/538.

Cloning would essentially create another Firrtl instance of an already elaborated Module. There are some subtle issues to think about with annotations (Module.component annotations would probably be fine but hierarchical ones would need duplication).

I propose something like:

val inst0 = Module(new MyModule)
val inst1 = inst0.cloneModule
// or
val inst2 = Module.cloneInst(inst0)
// or 
val inst3 = Module.duplicate(inst0)

This does present some serious problems for systems like rocket-chips’s LazyModule that utilize their own data structures to emit information about the configuration and the like. However, as a strictly opt-in feature such libraries could better leverage annotations or provide their own APIs that call ours to take advantage.

Thoughts? Questions? Concerns?

[jchang0]

An expansion to this which is related that might use the same underlying code in the future is incremental chisel compile. As the design gets larger, we might need incremental compile capability

I imagine that it might be related since we have to guarantee that the module we generated before is exactly the same as the module we are compiling now to be able to skip the compilation step. Granted that some firrtl optimization can no longer work across boundaries, it might be something that the user have to specify a particular module is subject to incremental compile and known optimization might not be applied.

[chick]

Do you have some estimate of the gain here? Your example was of an unparameterized Module, is that relevant or just a simpler example. It seems to me that requiring users to do this is not that desirable, it seems a bit hard to explain, users would likely not employ the technique at first and then when they hit the wall would have to go back and fix a bunch of existing code. Could Module.apply could be some soft of factory that (optionally) recognized creating of previous modules and did this under the hood.