question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Bug] hydra.utils.to_absolute_path() does not work properly with the joblib launcher plugin

See original GitHub issue

🐛 Bug

.to_absolute_path() works but not inside a multirun with the joblib launcher plugin

To reproduce

If main.py contains something like hydra.utils.to_absolute_path('./some_file.txt')

This will work when running python main.py foo=bar

but not python main.py -m foo=bar

In the second case, the expanded version of some_file.txt is not in the same directory as main.py but is in the multirun output directory instead.

This problem only seems to occur when using the joblib launcher with

defaults:
  - hydra/launcher: joblib

System information

  • Hydra Version : hydra-core==1.0.0rc1 and hydra-joblib-launcher==1.0.2
  • Python version : 3.7.7
  • Virtual environment type and version : Miniconda
  • Operating system : Mac OS X

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
jbraggcommented, Apr 20, 2020

Hi @omry apologies for this. For some unknown reason I’m also currently unable to reproduce this bug. Thanks so much for looking into it and adding the test in any case.

1reaction
jbraggcommented, Apr 15, 2020

Thanks, it’s not urgent as it seems the bug only applies to relative paths.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Joblib Launcher plugin - Hydra
The Joblib Launcher plugin provides a launcher for parallel tasks based on ... By default, process-based parallelism using all available CPU cores is...
Read more >
hydra-joblib-launcher - PyPI
Joblib Launcher for Hydra apps. ... Released: May 17, 2022. Joblib Launcher for Hydra apps ... Parallel based Hydra Launcher supporting parallel execution....
Read more >
Complete tutorial on how to use Hydra in Machine Learning ...
The main point is use orig_cwd = hydra.utils.get_original_cwd() to get the original working directory path and then you do not have to worry ......
Read more >
Browse Python Technical Problem Clusters - Code Grepper
Get code examples instantly right from your google search results with the Grepper Chrome Extension.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found