guide: concerns with "Persisting" experiments

Follow-up to https://github.com/iterative/dvc.org/pull/2845

Term/Concept of “persisting”

_Per https://github.com/iterative/dvc.org/pull/2845#issuecomment-937151219_

First, “persisting” usually means to continue something despite it being difficult. We want it to mean to make something (that was ephemeral) persistent. So not sure it’s a good title already.

One option is to be explicit, avoid term “persist”, and say “commit experiments to Git”. This is a probably good first step regardless.

Then we should consider all the reasons why people want to do commit experiments and generalize them into a term. Currently we only mention sharing experiments, but then this whole content should be inside https://dvc.org/doc/user-guide/experiment-management/sharing-experiments instead (which I doubt is right but maybe?)

• Related: improve motivation in the intro of https://dvc.org/doc/user-guide/experiment-management/persisting-experiments. Probably reuse https://dvc.org/doc/user-guide/experiment-management#persistent-experiments (and remove that from the index or briefly mention somewhere and link). _From https://github.com/iterative/dvc.org/pull/2845#issuecomment-939426781_

Other

• Don’t call the version where experiments are run (in examples) experiments. It can be confusing/misleading _From https://github.com/iterative/dvc.org/pull/2845#discussion_r726818325_
• General copy edits if needed (e.g. https://github.com/iterative/dvc.org/pull/2845#pullrequestreview-773103226)

[iesahin]

I think persist (as @dberenbaum in an earlier discussion said) was a bit larger than what we mean by committing experiments to Git. Persist may also mean to send these to a common repository (dvc exp push), for example. What we’re trying to figure out is to convey the meaning for dvc exp apply operation, basically moving Git ref in .git/refs/exps/ABCD123... as a commit from the HEAD.

DVC experiments requires Git to run, so these better reflect that connection too. So, although I understand the concern behind commit, among the candidates I tried to produce above, it looked like the best.

[omesser]

@jorgeorpinel - Good point calling me out on mentioning “persist” + “track” as equals here. I see your point doubting my usage of “persist” for experiments. It’s not widely used indeed, it’s too narrow

When I wrote:

If I were to read/skim these docs and compare to other experiment tracking solutions I would look for (search / grep) the terms “persist” and “track” (=what) and not “commit” (=how) - all the more important when it comes to titles!

…“Persisting” was probably my backend instincts kicking in for understanding where something is saved(persisted, if you will) if I weren’t to find the term “track”. But, you’re right, the correct thing for me to have done would be to search for “track[ing]”. So while I definitely see “persist” used in the way I mentioned (to save to persistency, I stand behind this being familiar 😉 ), it’s indeed not used in the context of ML experiments. The conventional term (it’s in the name) is experiment tracking.

Let me modify/focus my above claim: what: track, how: by commit to git

“In my mind “committing” is not an implementation detail but actually the user’s goal (=what)”

Can this be a little iterative/dvc “biased” though? meaning - your “what” here already assumes git-centric approach which is “unique” to us (user knows they want to commit to git).

What I’m referring to is familiarity/discoverability of people that don’t know how this works. Most (all?) other tools for “tracking” experiments don’t revolve around git as persistency / tracking mechanism. So, I would say commit to git is our “how” (how we track experiments). The “layman” ds/ml eng wouldn’t necessarily know / look for a way to “commit” their experiment but to track their experiments. Obviously, tracking is wider than just persisting - it can include discoverability / comparability / monitoring / insights… Committing to git satisfies some of those to some extent, for sure, maybe not all. And that’s why I (currently) perceive it as “how” experiments are tracked