Experiment __init__() + setup() + configure() is a confusing mess

Summary

The interactions between this cluster of API methods is confusing, and demos don’t provide clarifying examples. The ideas here are from a brainstorming session between @alecpm and @jessesnyder.

Goals

1. Custom experiments should not need to override __init__(), except under extraordinary circumstances
2. Custom experiments should not need to call super() in standard cases, since this is always confusing and it can be hard to tell when it matters
3. Base class should handle the database transaction in standard cases
4. Demos should clarify semantics of the API via consistent… demonstration.

Tentative Proposal

1. Experiment base class’s __init__() method calls a private _setup() (name TBD) and finishes with a database commit
2. _setup() handles idempotence by checking if self.networks() is None and creates basic network as the current base class’s setup() method does
3. _setup() calls self.setup() at the end
4. setup() has an empty implementation in the base class, and is where the subclass adds its own special nodes to the now-initialized networks
5. statements like self.initial_recruitment_size = 2 belong in configure(), not __init__() or setup()

[pmcharrison]

@alecpm Makes sense. Maybe there is a way as part of this breaking change to make this dual functionality more explicit in the Experiment class.

[alecpm]

@pmcharrison Generally the experiment class is only instantiated when programmatically launching experiment runs using APIs like Experiment.run and Experiment.collect, and in tests. By default when an experiment is instantiated without a session, the setup and configure methods aren’t run. The Experiment should always be instantiated with a DB session when used by the experiment server.