Best practices for `experiment` structures

Hello there.

At first, thank you for the sacred.

I’m trying to figure out the best way of using it in my workflow. Hence the question.

Roughly speaking, my workflow consists of two stages: training and prediction. If it is logical to imagine the first stage as an experiment, then what about the second one? In a sense, this is also an experiment - it also has parameters and some sort of selection. At the same time, I see it as 2 parts of a whole process. At first, I thought to design each part as a command. But in the end in the saсredboard everything looks like the launch of the same thing. And what I want is to see it as something separate, but still common.

Or should I just create 2 experiments, calling them somehow common, for example, “exp42.fit” and “exp42.predict”?

[LukasDrude]

My recommendation is, to have two independent experiments. The main reason is, that the day will come where you decode your model with several different settings.

To be more precise, I name both experiments differently and later analyse the results in a pandas dataframe. The first experiment than generates a model_id. To decode this models, I would then reference the model_id in the different decode runs.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.