question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Could dictionaries returned by dask.config.get also be config objects

See original GitHub issue

Assuming we have the following config.

# ~/.config/dask/foo.yaml
foo:
  bar:
    baz: "hello"

I can access the deepest nested object with dask.config.get.

>>> import dask.config
>>> dask.config.get("foo.bar.baz")
'hello'

However if I do this in two steps it is not possible because the return value is a regular Python dictionary.

>>> import dask.config
>>> foo = dask.config.get("foo")
>>> foo.get("bar.baz")
None

Instead I must use dask.config.get again and pass foo back in.

>>> dask.config.get("bar.baz", config=foo)
'hello'

It would be nice if when the result for dask.config.get is a dictionary then instead it returns something which has the same behaviour.

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
jacobtomlinsoncommented, Nov 20, 2020

Also interesting side note, creating the ~/.config/dask/foo.yaml file above for my small example causes the config tests to fail. Because one test does assert "foo" not in config. That lead to some unnecessary debugging 😂.

0reactions
jacobtomlinsoncommented, Nov 20, 2020

@djhoese that does sound interesting.

My immediate use case for this issue has actually gone away now. Due to some refactoring, the config is no longer nested in the same way. Although this may still be useful in the future.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Configuration - Dask documentation
This function will recursively search through any nested dictionaries and/or lists. Parameters. configdict, iterable, or str. Input object to search for ...
Read more >
Source code for dask.config
Source code for dask.config. from __future__ import annotations import ast import base64 import builtins # Explicitly use builtins.set as 'set' will be ...
Read more >
configuration.rst.txt
These settings may also be cached for performance reasons. This is particularly true for ``dask.distributed`` objects such as Client, Scheduler, Worker, ...
Read more >
Optimization - Dask documentation
Performance can be significantly improved in different contexts by making small optimizations on the Dask graph before calling the scheduler.
Read more >
Create Dask Bags - Dask documentation
By default, Dask will try to partition your data into about 100 partitions. IMPORTANT: do not load your data into Python and then...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found