`exp show`: add output of `params diff` & `metrics diff` to `workspace` entry within `--show-json`

In the VS Code extension we recently added in a feature to our experiments table which highlights params and metrics which have changed with respect to the previous commit.

Example (example-dvc-experiments)

Given:

1. dvc params diff --show-json = {"params.yaml": {"train.epochs": {"old": 10, "new": 11, "diff": 1}}}
2. dvc metrics diff --show-json = {}

Then we will show:

The current cli implementation means that we now have to run 3 commands to get the required data for our table. These are:

1. dvc exp show --show-json
2. dvc params diff --show-json
3. dvc metrics diff --show-json

We are still running into repo locks when trying to run any of the above commands. Also, having to run all three is slower.

Ideally I’d like to include the output of 2 & 3 within the workspace dict of 1 but open to other suggestions.

Note: If we pushed this data into exp show then we would also be able to highlight the same records in the cli table.

Related issues:

• https://github.com/iterative/dvc/issues/5739
• https://github.com/iterative/dvc/issues/5966
• https://github.com/iterative/dvc/issues/6434
• https://github.com/iterative/dvc/issues/5515

[pmrowla]

Showing/highlighting deltas in the exp show table can already be implemented as UI on top of the existing available data in both DVC and the vscode extension (since both the original and new params/metrics values are already exposed in the existing exp show --show-json data).

I think @mattseddon is asking for the deltas to also be included in the json in addition to the existing data, which IMO is a separate discussion from any spinoff issue for displaying the deltas in the DVC CLI table (#6783)

---

On re-implementing DVC internals, handling the git ref mechanics of HEAD vs EXEC_BASELINE should be unnecessary on the vscode extension side (since exp show already accounts for it, and returns all the data needed in place of the original 3 commands that vscode was using)

[pmrowla]

You shouldn’t need the extra params diff and metrics diff calls. The original commit data should already be available in exp show in <orig_commit_sha>.baseline.data

❯ dvc exp show --show-json
{
  "workspace": {
    "baseline": {
      "data": {
        ...
      }
    }
  },
  "00071e8d7f137fc7265b386aaba06d39a1a30b61": {
    "baseline": {
      "data": {
        "timestamp": "2021-05-29T15:23:32",
        "params": {
          "params.yaml": {
            "data": {
              "prepare": {
                "split": 0.2,
                "seed": 20170428
              },
              "featurize": {
                "max_features": 3000,
                "ngrams": 2
              },
              "train": {
                "seed": 20170428,
                "n_est": 100,
                "min_split": 64
              }
            }
          }
        },
        "queued": false,
        "running": false,
        "executor": null,
        "metrics": {
          "scores.json": {
            "data": {
              "avg_prec": 0.6040544652105823,
              "roc_auc": 0.9608017142900953
            }
          }
        },
        "name": "master"
      }
    },
    "5d1639dc071d8f76c9655fcbaa1a97aef87392f1": {
      "data": {
        "timestamp": "2021-10-05T21:55:17",
        "params": {
          "params.yaml": {
            "data": {
              "prepare": {
                "split": 0.2,
                "seed": 20170428
              },
              "featurize": {
                "max_features": 2000,
                "ngrams": 2
              },
              "train": {
                "seed": 20170428,
                "n_est": 100,
                "min_split": 64
              }
            }
          }
        },
        "queued": false,
        "running": false,
        "executor": null,
        "metrics": {
          "scores.json": {
            "data": {
              "avg_prec": 0.5858888885424922,
              "roc_auc": 0.944996664954421
            }
          }
        },
        "name": "exp-8dd71"
      }
    },
...

in this example 00071e is the original commit SHA (master) and 5d1639 is the exp SHA (exp-8dd71) So diffing 00071e.baseline.data.params and 00071e.5d1639.data.params (using the full SHAs as keys) should give you the same information you get from params diff (and the same for data.metrics and metrics diff)