Work out how to scale `.verify()` to lots of inits
See original GitHub issueRight now .verify()
does not scale well to many inits. Currently I am trying to verify an S2S system (that was setup like SubX) with ~1,000 inits and 45 leads. I am just working with a time series and it’s prohibitively slow to even set up the dask graph or to just do in-memory. I think this has to do with constructing a huge dictionary that has the right dates at every init to align.
I think we need to evaluate the shift in datetimes while evaluating to make dask happier. Although I think I went with this dictionary method since you’d only have to do the shifting once, and to figure out which inits to keep for the various alignment strategies.
Issue Analytics
- State:
- Created 3 years ago
- Comments:29 (2 by maintainers)
Top Results From Across the Web
What Is a Scale Factor? - Video & Lesson Transcript
To write the scale factor of any scaled copy to the original, find the corresponding sides and write a ratio of the original...
Read more >Scale Calculator
Scale calculator determines the size of scaled/real structures for a given scale or evaluates the scale between two objects.
Read more >How a PLC do the Scaling for a Sensor
PLC Scaling a Sensor : PLC do the analog input scaling for a transmitter and displays the Process Variable reading in Operator Workstation....
Read more >Lesson 19: Computing Actual Areas from a Scale Drawing
Calculate the area of the scale drawing, and then multiply by the square of the scale (or scale factor) to determine the actual...
Read more >How to Determine Returns to Scale: Table-Type Problem
A look at increasing returns to scale, constant returns to scale, ... Increasing returns to scale : If all inputs increase X%, out......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I think you need to do
%lprun -f he.verify he.verify(**kw).compute()
for lprun to work.isin() is the bottleneck here. Maybe dict helps, see https://twitter.com/jesperdramsch/status/1486235033153282049?s=21