question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

task mapping on generators?

See original GitHub issue

Is there a reason why we can’t pass a generator expression to task.map? loading the whole list in memory is potentially inefficient. If what is needed is just development time, I’d like to work on it.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:4
  • Comments:8 (1 by maintainers)

github_iconTop GitHub Comments

6reactions
cicdwcommented, Mar 13, 2021

Chiming in for josh here – there is currently no timeline for this although it is something we are increasingly interested in undertaking! The implementation will most likely be more general than mapping over generators, and allow for a “parent” task to asynchronously spawn new tasks that can immediately begin running (under the right executor). I will update here if and when this enters the planning phases.

3reactions
madkinszcommented, Nov 28, 2022

@sveinugu that commit is indeed unrelated but this was closed long ago by the opener of the issue. I believe mapping over generators is supported in Prefect v2.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Random-task functions (coNCePTuaL User's Guide)
The following functions are used to select tasks at random. coNCePTuaL currently uses the Mersenne Twister as its random-number generator.
Read more >
Introducing PDI Mapping Generators - Pacific Data Integrators
PDI has created tools that use existing APIs to automate redundant and repetitive tasks, which create significant timeline reductions and significant cost ...
Read more >
How to use IPython.parallel map() with generators as input to ...
This just generates a sequence of integers to use when testing if a number is prime. Now our trivial function that we will...
Read more >
Sequence Generator values - Informatica Documentation
The Sequences page in the mapping task wizard displays the name of the Sequence Generator transformation. Viewing and editing mapping task details ...
Read more >
MappingGenerator - Visual Studio Marketplace
Full Mapper generator. MappingGenerator provides code actions for generating a complete mapper class with desired mapping methods:.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found