question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Incorrect description for env_extra for HTCondorCluster

See original GitHub issue

Hi,

The description for env_extra in HTCondorCluster is not correct: the job that HTCondorCluster creates calls dask-worker directly instead of through a bash wrapper script, so you cannot put arbitrary shell commands into env_extra.

The interface supports environment variables as key=value pairs, which will be inserted into dask-worker’s environment (via the “Environment” attribute in the submit file). (For consistency, you can write export foo=bar but the word “export” will be ignored.)

This is also important to keep in mind with regards to #323; renaming env_extra to job_script_extra or similar would be even more inaccurate (for the HTCondor case anyway).

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:20 (20 by maintainers)

github_iconTop GitHub Comments

3reactions
jolangecommented, Jul 26, 2022

Ok, I made a proposal in #563. I’d suggest to start the renaming (#323) after we agreed on the implementation and merged it. (I can also do that if you like.)

2reactions
jolangecommented, Jul 26, 2022

Heh /tmp is there to be cluttered 😆

😆

So by now I would prefer the in-lining solution to an extra temporary script. But maybe someone else has a good idea how to realize this.

Read more comments on GitHub >

github_iconTop Results From Across the Web

job_extra, extra and env_extra naming could be improved #323
Incorrect description for env_extra for HTCondorCluster #393 ... Fix behaviour of env_extra for HTCondor #563.
Read more >
condor_submit — HTCondor Manual 10.1.1 documentation
Each submit description file describes one or more clusters of jobs to be placed in the HTCondor execution pool. All jobs in a...
Read more >
HTCondor User Tutorial - CERN Indico
HTCondor manages and runs work on your behalf. • Schedule tasks on a single computer to not overwhelm the computer.
Read more >
[HTCondor-users] PROBLEM Submitting CONDOR JOB ... - Re
Re: [HTCondor-users] PROBLEM Submitting CONDOR JOB WITH BASH SCRIPT · Date: Tue, 9 Jan 2018 09:11:57 -0600 · From: Jason Patton <jpatton@xxxxxxxxxxx> ...
Read more >
condor_submit Queue - jobs for execution under HTCondor
A cluster is a set of jobs specified in the submit description file between ... HTCondor does not create a log for this...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found