Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Less restrictive DataSource effect implicits

See original GitHub issue

Currently the methods that fetch data in DataSource require an effect type F with Par[F] and ConcurrentEffect[F].

Par is required due to the DataSource#batch implementation defaulting to running individual fetches in parallel. If we move away from it, Data sources that don’t implement batching will run their fetches sequentially. Par[F] will still be required when creating or running a fetch to F, since is used when running multiple batches in parallel.

ConcurrentEffect is perhaps too restrictive, since we may not need the cancellation semantics that Concurrent provides. We could use Effect here, thoughts?


Issue Analytics

  • State:open
  • Created 5 years ago
  • Reactions:1
  • Comments:8 (7 by maintainers)

github_iconTop GitHub Comments

kubukozcommented, Mar 13, 2019

Would you feel any different about Par if cats-par was officially endorsed by typelevel?

Here are all the reasons I found not to require Concurrent:

  • it’s only used for a reimplementation of parTraverse for which Parallel/Par is enough
  • it’s one of the most powerful type classes in the whole of cats/cats-effect: it provides ways to embed arbitrary side effects using FFI-like methods like delay, async, cancelable and friends; which is the main reason why it’s usually a good idea to use it sparingly and hide the details like calls to race behind algebras that’ll be implemented with Concurrent
  • it requires Throwable errors, whereas the only place where Fetch catches and handles error is the run part (in which MonadError[F, Throwable] would probably suffice)
  • very minor: you can have a Parallel for any monad, which gives the ability to get a class under test with a simple type like Id or Either and not a full blown IO that may or may not consist of asynchronous blocks

FYI ContextShift is completely unused in the code for fetch, too, so the only thing that remains is Clock, usage of which could be superseded by a custom algebra for measuring time (allowing custom implementations that’d simply return a dumb value for tests).

peterneyenscommented, Mar 13, 2019

I think Par is worth having to avoid having to duplicate its implementation for types that have Concurrent.

I think I disagree know with what I said almost 6 months ago as well, and I agree that we should look if we can’t get by with just Concurrent (instead of ConcurrentEffect).

Read more comments on GitHub >

github_iconTop Results From Across the Web

Less restrictive DataSource effect implicits · Issue #163
Par is required due to the DataSource#batch implementation defaulting to running individual fetches in parallel. If we move away from it, Data ......
Read more >
Indexer troubleshooting guidance - Azure Cognitive Search
Currently, indexers can access restricted data sources behind an IP ... Azure Cognitive Search has an implicit dependency on Azure Cosmos DB ...
Read more >
IBM Data Server Driver for JDBC and SQLJ
The effect of fullyMaterializeLobData depends on whether the data source supports progressive streaming, which is also known as dynamic data format: If the...
Read more >
4. Risks and challenges of data access and sharing
Digital security risks and confidentiality breaches in particular · Digital security risks of more data openness · Increasing impact of (personal) data breaches....
Read more >
Continuous Availability - MAA Checklist for Applications for ...
effects or callbacks, or have an application that uses state such as temp tables ... service as this cannot be disabled, relocated, or...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found