question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Use IO-Subgraph + Blockwise throughout codebase

See original GitHub issue

~Blocked by #6715~ ~Blocked by #7281~

After #7281 goes in, most (maybe all) IO operations in the dataframe and array modules can be converted into Blockwise high-level graph (HLG) layers. This change should improve HLG optimizations (like blockwise fusion).

This issue is intended as a “living” checklist of action items related to Blockwise IO…

Dask-Dataframe:

Dask-Array:

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
jrbourbeaucommented, Jun 3, 2021

Is it worth creating a doc page explaining how people go about implementing an HLG operation themselves?

Cross referencing https://github.com/dask/dask/issues/7709 and https://github.com/dask/dask/issues/7755 for updating documentation around HLGs

1reaction
jakirkhamcommented, May 26, 2021

Is it worth creating a doc page explaining how people go about implementing an HLG operation themselves? Maybe that helps people feel more comfortable filling out gaps in the codebase. Not to mention writing their own in their code

Read more comments on GitHub >

github_iconTop Results From Across the Web

About - Codebase@Berkeley
Codebase is a UC Berkeley student organization that builds software projects for high growth tech companies.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found