question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Parallelism via function vectorization?

See original GitHub issue

Hi @yannikschaelte - I’m really enjoying using pyABC. Thanks for putting together such a powerful and easy to use tool!

A common scenario that I find myself in is that I have a very fast function that can be passed vector arguments. A simple example is some simply numpy function. Perhaps a more realistic case is when I have something like a Gaussian process regression surrogate trained. In either of these cases, passing several hundred points is almost as fast as just doing a single point call, since python function call overhead dominates.

Is there an easy way to use a sampler to make vectorized calls? I poked around at MappingSampler, but it seems like one would need to dig into the function being mapped, since the list being passed just controls the number of samples called (as far as I can tell).

I’d be happy to help implement this feature, but I want to make sure that it’s not already in place (or easy to hack). Thanks!

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:17 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
yannikschaeltecommented, Jan 13, 2021

Ah, no problem! Out of interest: Does the pymc3 MCMC implementation provide vectorization, or is it just parallelized?

1reaction
yannikschaeltecommented, Dec 16, 2020

But if this “guessing” is sufficiently sophisticated, this could work very well. Implementation-wise, this would mean a slight API extension of simulate_one to simulate_many (m=int), and then defining a new sampler pypesto.sampler.VectorizedSomethingSampler.

I hope this is consistent what I wrote earlier this year 😆

Read more comments on GitHub >

github_iconTop Results From Across the Web

Parallelization and Vectorization of ROOT Fitting classes
All these different tools for parallelism come together when paral- ... Vectorization will be applied for evaluating the function several.
Read more >
Vector Parallelism - an overview | ScienceDirect Topics
Vector intrinsics are a low-level, direct approach to explicit vectorization. Intrinsics are special data types and functions that map directly onto specific ...
Read more >
Parallel Vectorized Operations | R-bloggers
Vectorized Operations. How does the apply family speed up our functions? It converts a scalar (single item) process to a vector (array) ...
Read more >
Parallelism in Python*: Directing Vectorization with NumExpr
One interesting way of achieving Python parallelism is through NumExpr, in which a symbolic evaluator transforms numerical Python expressions into high- ...
Read more >
Vectorization vs Parallelization : Making you code run faster
The word parallel means that there are at least a couple of entities ... Using vectorization eliminates the requirement of loops. so now...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found