Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

enable default algo template caching?

See original GitHub issue

currently the default file is downloaded to a temporary folder and therefore skips any caching

https://github.com/Project-MONAI/MONAI/blob/cd2f4e15a050f2ff1abc66bf19d499ce3cd58a31/monai/apps/auto3dseg/bundle_gen.py#L317-L320

this causes an issue when running multiple tests:

[2022-10-18T08:02:43.002Z] ======================================================================
[2022-10-18T08:02:43.002Z] ERROR: test_autorunner_hpo (__main__.TestAutoRunner)
[2022-10-18T08:02:43.002Z] ----------------------------------------------------------------------
[2022-10-18T08:02:43.002Z] Traceback (most recent call last):
[2022-10-18T08:02:43.002Z]   File "tests/test_integration_autorunner.py", line 136, in test_autorunner_hpo
[2022-10-18T08:02:43.002Z]     runner.run()
[2022-10-18T08:02:43.002Z]   File "/home/jenkins/agent/workspace/Monai-pytorch-versions/monai/apps/auto3dseg/auto_runner.py", line 568, in run
[2022-10-18T08:02:43.002Z]     bundle_generator = BundleGen(
[2022-10-18T08:02:43.002Z]   File "/home/jenkins/agent/workspace/Monai-pytorch-versions/monai/apps/auto3dseg/bundle_gen.py", line 319, in __init__
[2022-10-18T08:02:43.002Z]     download_and_extract(default_algo_zip if algos is None else algos, algo_compressed_file, algo_path)
[2022-10-18T08:02:43.002Z]   File "/home/jenkins/agent/workspace/Monai-pytorch-versions/monai/apps/utils.py", line 312, in download_and_extract
[2022-10-18T08:02:43.002Z]     download_url(url=url, filepath=filename, hash_val=hash_val, hash_type=hash_type, progress=progress)
[2022-10-18T08:02:43.002Z]   File "/home/jenkins/agent/workspace/Monai-pytorch-versions/monai/apps/utils.py", line 203, in download_url
[2022-10-18T08:02:43.002Z]     _download_with_progress(url, tmp_name, progress=progress)
[2022-10-18T08:02:43.002Z]   File "/home/jenkins/agent/workspace/Monai-pytorch-versions/monai/apps/utils.py", line 114, in _download_with_progress
[2022-10-18T08:02:43.002Z]     raise e
[2022-10-18T08:02:43.002Z]   File "/home/jenkins/agent/workspace/Monai-pytorch-versions/monai/apps/utils.py", line 107, in _download_with_progress
[2022-10-18T08:02:43.002Z]     urlretrieve(url, filepath, reporthook=t.update_to)
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 247, in urlretrieve
[2022-10-18T08:02:43.002Z]     with contextlib.closing(urlopen(url, data)) as fp:
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
[2022-10-18T08:02:43.002Z]     return opener.open(url, data, timeout)
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 531, in open
[2022-10-18T08:02:43.002Z]     response = meth(req, response)
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
[2022-10-18T08:02:43.002Z]     response = self.parent.error(
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 563, in error
[2022-10-18T08:02:43.002Z]     result = self._call_chain(*args)
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
[2022-10-18T08:02:43.002Z]     result = func(*args)
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
[2022-10-18T08:02:43.002Z]     return self.parent.open(new, timeout=req.timeout)
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 531, in open
[2022-10-18T08:02:43.002Z]     response = meth(req, response)
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
[2022-10-18T08:02:43.002Z]     response = self.parent.error(
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 569, in error
[2022-10-18T08:02:43.002Z]     return self._call_chain(*args)
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
[2022-10-18T08:02:43.002Z]     result = func(*args)
[2022-10-18T08:02:43.002Z]   File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
[2022-10-18T08:02:43.002Z]     raise HTTPError(req.full_url, code, msg, hdrs, fp)
[2022-10-18T08:02:43.002Z] urllib.error.HTTPError: HTTP Error 503: Egress is over the account limit.
[2022-10-18T08:02:43.002Z] 
[2022-10-18T08:02:43.002Z] ----------------------------------------------------------------------

an option of caching the default algo could avoid the repeated donwloading cc @Nic-Ma @mingxin-zheng

Issue Analytics

State:
Created a year ago
Comments:5 (5 by maintainers)

Top GitHub Comments

1reaction

mingxin-zhengcommented, Dec 6, 2022

@wyli I think after #5623 BundleGen will make 3 attempts to download the templates to resolve the HTTP errors. Can we close this issue?

0reactions

wylicommented, Oct 21, 2022

I haven’t looked into a better idea, but previously in the mmar utilities we download the zip file to the torch hub caching folder, and unzip to a user-specified folder: https://github.com/Project-MONAI/MONAI/blob/8aafd882aedc2bc54e93fb14e02e4a000bca2869/monai/apps/mmars/mmars.py#L150-L151

the torch hub caching folder is by default ~/.cache/torch and it’s cached during CI: https://github.com/Project-MONAI/MONAI/blob/4bbf125eb5e6ceae1cd41a080f08feed66b4958d/.github/workflows/setupapp.yml#L38-L39

also, maybe file IO is not a critical issue because the downloading is to a temporary location first and then moved to the destination https://github.com/Project-MONAI/MONAI/blob/8aafd882aedc2bc54e93fb14e02e4a000bca2869/monai/apps/utils.py#L211

Top Results From Across the Web

29. Cache Abstraction - Spring

Default Key Generation Out of the box, the caching abstraction uses a simple KeyGenerator based on the following algorithm: If no params are...

Caching Pipeline Steps - Amazon SageMaker

Turn on step caching · Turn off step caching · Default cache key attributes by pipeline step type · Processing step · Training...

Optimize performance with caching on Databricks

Learn how Databricks disk caching accelerates data reads. ... To enable for caching, see Enable or disable the disk cache. The disk cache...

Cache Configuration - PolyScale.ai

When you create a new cache, the default behavior is automatic caching enabled. With this setting in place, PolyScale will adaptively write query...

mod_cache - Apache HTTP Server Version 2.4

In the default enabled configuration, the cache operates within the quick handler phase. This phase short circuits the majority of server processing, and ......