question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

What is your issue?

Importing the xarray package takes a significant amount of time. For instance:

❯ time python -c "import xarray"
python -c "import xarray"  1.44s user 0.52s system 132% cpu 1.476 total

compared to others

❯ time python -c "import pandas"
python -c "import pandas"  0.45s user 0.35s system 177% cpu 0.447 total

❯ time python -c "import scipy"
python -c "import scipy"  0.29s user 0.23s system 297% cpu 0.175 total

❯ time python -c "import numpy"
python -c "import numpy"  0.29s user 0.43s system 313% cpu 0.229 total

❯ time python -c "import datetime"
python -c "import datetime"  0.05s user 0.00s system 99% cpu 0.051 total

I am obviously not surprised that importing xarray takes longer than importing Pandas, Numpy or the datetime module, but 1.5 s is something you clearly notice when it is done e.g. by a command-line application.

I inquired about import performance and found out about a lazy module loader proposal by the Scientific Python community. AFAIK SciPy uses a similar system to populate its namespaces without import time penalty. Would it be possible for xarray to use delayed imports when relevant?

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:9 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
headtr1ckcommented, Oct 3, 2022

Nice. Does it work on python 3.8?

according to the docu it exists since 3.4.

1reaction
eendebakptcommented, Aug 23, 2022

Some other projects are considering lazy imports as well: https://scientific-python.org/specs/spec-0001/

Read more comments on GitHub >

github_iconTop Results From Across the Web

time — Time access and conversions — Python 3.11.1 ...
Convert a time expressed in seconds since the epoch to a string of a form: 'Sun Jun 20 23:21:05 1993' representing local time....
Read more >
Long import times for some modules in VS Code
pandas invokes numpy, and both of those are very large packages with many C DLLs. It takes many seconds for them to load....
Read more >
Slow Running Import Fixes And Workarounds - WP All Import
Are your imports running slower than you expected? This guide will walk you through common causes and how to resolve them.
Read more >
Troubleshooting Data Import Speed - Caspio Online Help
Just wait for it to finish. If the process is frozen, normally in one to two hours the automated system will terminate the...
Read more >
How long does an import take? - Knowledge Center
Guideline Time frames for Import: ; Less that 25GB. Upto 2 weeks ; 25GB to 50GB. 3 weeks ; 50GB to 100GB. 4...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found