Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Experiment __init__() + setup() + configure() is a confusing mess

See original GitHub issue


The interactions between this cluster of API methods is confusing, and demos don’t provide clarifying examples. The ideas here are from a brainstorming session between @alecpm and @jessesnyder.


  1. Custom experiments should not need to override __init__(), except under extraordinary circumstances
  2. Custom experiments should not need to call super() in standard cases, since this is always confusing and it can be hard to tell when it matters
  3. Base class should handle the database transaction in standard cases
  4. Demos should clarify semantics of the API via consistent… demonstration.

Tentative Proposal

  1. Experiment base class’s __init__() method calls a private _setup() (name TBD) and finishes with a database commit
  2. _setup() handles idempotence by checking if self.networks() is None and creates basic network as the current base class’s setup() method does
  3. _setup() calls self.setup() at the end
  4. setup() has an empty implementation in the base class, and is where the subclass adds its own special nodes to the now-initialized networks
  5. statements like self.initial_recruitment_size = 2 belong in configure(), not __init__() or setup()

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:7 (2 by maintainers)

github_iconTop GitHub Comments

pmcharrisoncommented, Jul 16, 2021

@alecpm Makes sense. Maybe there is a way as part of this breaking change to make this dual functionality more explicit in the Experiment class.

alecpmcommented, Jul 16, 2021

@pmcharrison Generally the experiment class is only instantiated when programmatically launching experiment runs using APIs like and Experiment.collect, and in tests. By default when an experiment is instantiated without a session, the setup and configure methods aren’t run. The Experiment should always be instantiated with a DB session when used by the experiment server.

Read more comments on GitHub >

github_iconTop Results From Across the Web

exp init experience · Issue #7130 · iterative/dvc - GitHub
One goal of dvc exp init was to introduce an easy way to setup typical project structures, such as deep learning with checkpoints...
Read more >
Enable loading and saving of question answers. · 7525ebcded ...
This lays the groundwork for interactive init, as well as being able to specify control and compute nodes. Added preliminary config lists for...
Read more >
What do you want to know about the process of converting an ...
Hey everyone! I'm going to be giving a talk at VimConf this year on how to transition from an init.vim setup to an...
Read more >
How to initialize test data for benchmark test in golang?
You can use *B.ResetTimer to remove the setup time from the overall benchmark ... ResetTimer() for i := 0; i < b.N; i++...
Read more >
How to Keep Track of Experiments in PyTorch -
Setting up Neptune experiment in Pytorch ... Just pass on these as parameters to the init() function, that's how easy it is:
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found