Could dictionaries returned by dask.config.get also be config objects
See original GitHub issueAssuming we have the following config.
# ~/.config/dask/foo.yaml
foo:
bar:
baz: "hello"
I can access the deepest nested object with dask.config.get
.
>>> import dask.config
>>> dask.config.get("foo.bar.baz")
'hello'
However if I do this in two steps it is not possible because the return value is a regular Python dictionary.
>>> import dask.config
>>> foo = dask.config.get("foo")
>>> foo.get("bar.baz")
None
Instead I must use dask.config.get
again and pass foo back in.
>>> dask.config.get("bar.baz", config=foo)
'hello'
It would be nice if when the result for dask.config.get
is a dictionary then instead it returns something which has the same behaviour.
Issue Analytics
- State:
- Created 3 years ago
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Configuration - Dask documentation
This function will recursively search through any nested dictionaries and/or lists. Parameters. configdict, iterable, or str. Input object to search for ...
Read more >Source code for dask.config
Source code for dask.config. from __future__ import annotations import ast import base64 import builtins # Explicitly use builtins.set as 'set' will be ...
Read more >configuration.rst.txt
These settings may also be cached for performance reasons. This is particularly true for ``dask.distributed`` objects such as Client, Scheduler, Worker, ...
Read more >Optimization - Dask documentation
Performance can be significantly improved in different contexts by making small optimizations on the Dask graph before calling the scheduler.
Read more >Create Dask Bags - Dask documentation
By default, Dask will try to partition your data into about 100 partitions. IMPORTANT: do not load your data into Python and then...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Also interesting side note, creating the
~/.config/dask/foo.yaml
file above for my small example causes the config tests to fail. Because one test doesassert "foo" not in config
. That lead to some unnecessary debugging 😂.@djhoese that does sound interesting.
My immediate use case for this issue has actually gone away now. Due to some refactoring, the config is no longer nested in the same way. Although this may still be useful in the future.