dvc exp run --set-param does not change parameter values from a Python dict

Bug Report

dvc exp run --set-param does not change parameter values from a Python dict

Description

When trying to do dvc exp run --set-param my_params.py:my_dict.my_key=3 the parameter value is not changed to 3, it remains the same as previously. The output is:

Stage 'my_stage' didn't change, skipping                                                             

Reproduce

1. git init
2. dvc init
3. dvc.yaml

stages:
  my_stage:
    cmd: cat a.txt > b.txt
    deps:
    - a.txt
    outs:
    - b.txt
    params:
    - my_params.py:
      - my_dict

1. my_params.py

my_dict = {
    'my_key': 2
}

1. echo dummyString > a.txt
2. dvc exp run --set-param my_params.py:my_dict.my_key=3

Expected

I would expect this output (which can be achieved by changing the value of my_key manually in my_params.py and then running dvc exp run):

Running stage 'my_stage':                                                                            
> cat a.txt > b.txt
Updating lock file 'dvc.lock'                                                                        

To track the changes with git, run:

	git add dvc.yaml a.txt dvc.lock my_params.py
                                                                      
Reproduced experiment(s): exp-e60e4
Experiment results have been applied to your workspace.

To promote an experiment to a Git branch run:

	dvc exp branch <exp> <branch>

Environment information

Output of dvc doctor:

$ dvc doctor
DVC version: 2.8.3 (pip)
---------------------------------
Platform: Python 3.8.10 on Linux-5.11.0-40-generic-x86_64-with-glibc2.29
Supports:
	webhdfs (fsspec = 2021.10.1),
	http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
	https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
	s3 (s3fs = 2021.8.1, boto3 = 1.17.106)
Cache types: reflink, hardlink, symlink
Cache directory: xfs on /dev/sdb
Caches: local
Remotes: None
Workspace directory: xfs on /dev/sdb
Repo: dvc, git

[dberenbaum]

Ultimately, it seems like other methods for logging parameters, like those discussed in https://github.com/iterative/dvc/discussions/6506, may be a more useful approach than parsing/replacing parameters from Python files.

[dberenbaum]

So dvc validates that my_params.py:my_dict.my_key exists but can’t replace it? In that case, I agree that we should probably raise an error to say something like Unable to parse {file} to set parameter {key}. Please update {file} instead of using --set-param.