Hydra composition writes scientific float without decimal point which is parsed by YAML as str
See original GitHub issueBug Report
Description
When using the Hydra Composition, float numbers in the input config files in scientific format, e.g., 1.0e-5
are written to the generated params.yaml
file as 1e-5
. The latter is parsed as a str
instead of a float
. The decimal point is required.
Reproduce
conf/config.yaml
number: 1.0e-5
test.py
from pathlib import Path
import yaml
text = Path("params.yaml").read_text()
config = yaml.load(text, Loader=yaml.SafeLoader)
assert isinstance(config["number"], float)
dvc.yaml
stages:
test:
cmd: python test.py
dvc exp run test
Generated params.yaml
:
number: 1e-5
Expected
Environment information
Output of dvc doctor
:
$ dvc doctor
DVC version: 2.30.0 (rpm)
---------------------------------
Platform: Python 3.8.3 on Linux-3.10.0-1160.15.2.el7.x86_64-x86_64-with-glibc2.14
Subprojects:
Supports:
azure (adlfs = None, knack = 0.10.0, azure-identity = None),
gdrive (pydrive2 = 1.14.0),
gs (gcsfs = None),
hdfs (fsspec = None, pyarrow = 9.0.0),
http (aiohttp = None, aiohttp-retry = 2.8.3),
https (aiohttp = None, aiohttp-retry = 2.8.3),
oss (ossfs = 2021.8.0),
s3 (s3fs = None, boto3 = None),
ssh (sshfs = 2022.6.0),
webdav (webdav4 = 0.9.7),
webdavs (webdav4 = 0.9.7),
webhdfs (fsspec = None)
Cache types: hardlink, symlink
Cache directory: xfs on /dev/md124
Caches: local, s3
Remotes: s3, s3
Workspace directory: xfs on /dev/md124
Repo: dvc (subdir), git
Additional Information (if any):
Issue Analytics
- State:
- Created a year ago
- Comments:6 (3 by maintainers)
Top Results From Across the Web
Numbers in scientific notation without dot are parsed as string
Hi, it appears that PyYAML requires a dot in the mantissa as well as a sign in the exponent in order to parse...
Read more >YAML loads 5e-6 as string and not a number - Stack Overflow
The matching of floats can be fixed so it will accept floating point values with an e / E but without decimal dot...
Read more >Config composition - Hydra
We use the same solution in configuration as in writing the underlying software: composition. To do this in Hydra, we first add a...
Read more >See raw diff - Hugging Face
droptol : float - tolerance for dropping near-zero values when writing compressed binary. - Default is None - chunk : int - chunk...
Read more >Floating point numbers (Float) - HitchDev
StrictYAML parses to a YAML object representing a decimal - e.g. ... require precision, but float is better for values for which precision...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
OK, I think I understand now. The conclusion is that the
params.yaml
you produce requires a YAML 1.2 compatible reader. And PyYAML is not, which is what I use in my code currently. Switching to ruaml.yaml will fix it. Got it 😃But when I install DVC from YUM repo, does it not use its own virtual environment / libraries in
/usr/lib/dvc
?