question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Batch fetch operations

See original GitHub issue

I see following three unified API operations which would ideally have their batch equivalent:

  • fetchOrderBook (symbol): Fetch an order book for a particular product trading symbol.
  • fetchTrades (symbol): Fetch recent trades for a particular trading symbol.
  • fetchTicker (symbol): Fetch latest ticker data by trading symbol.

I’m using python syntax

Thus for example istead of:

ex = some_exchange
ex.fetchTicker("BTC/ETH"), ex.fetchTicker("DSH/BTC")

I could write (executor would be instace of https://docs.python.org/3/library/concurrent.futures.html):

ex.batchFetchTicker(["BTC/ETH", "DSH/BTC"], executor=exec)

Or by using asyncio and python3.5+ syntax:

await ex.batchFetchTicker(["BTC/ETH", "DSH/BTC"])

Some markets implement batch REST API, for example kraken uses comma separated symbol lists. Others like GDAX have no such option, and this method could be implemented in fallback mode using fetchX functions (with taking care of concurrent HTTP request, and not executing them sequentially).

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:24 (23 by maintainers)

github_iconTop GitHub Comments

2reactions
Nilotavianocommented, Aug 8, 2017

First of all, sorry if I’m posting this in the wrong place. I’ve seen you implemented fetchTickers for some exchanges, I’m just wondering if you are planning to do the same for bittrex and poloniex. I may be able to do it at night, if you take pull requests.

2reactions
nmiculiniccommented, Aug 2, 2017

Regading asycio, I have a suggestion. Reading your code, the only function doing HTTP IO is fetch. This one could be made async using python’s aiohttp; examples of which are many online. Consequently, every other function depending on it becomes a coroutine from load_products to everything else. Thus instead of having

a = fetch(....)

You’d have

a = await fetch(..)

which doesn’t seem to hard to integrate into the transpiler. Maybe you could create new module aioccxt for users who’d like asyncio capabilities with python3.5+ API (async/await)

Read more comments on GitHub >

github_iconTop Results From Across the Web

Amazon S3 Batch Operations - AWS
S3 Batch Operations is a managed solution for performing storage actions like copying and tagging objects at scale, whether for one-time tasks or...
Read more >
Execute batch operations using the Web API - Microsoft Learn
Batch operation lets you group multiple operations in a single HTTP request. Read how to execute batch operations using the Web API.
Read more >
Batch Fetching API - Texture
Texture's Batch Fetching API makes it easy to add fetching chunks of new data. Usually this would be done in a -scrollViewDidScroll: method, ......
Read more >
Batch HTTP Link - Apollo GraphQL Docs
The BatchHttpLink is a terminating link that batches an array of individual GraphQL operations into a single HTTP request that's sent to a...
Read more >
How does the batch execution fetch records to do any ...
Okay, I have huge set of records in a Custom object and i'm fetching these to do some kind of DML operations. During...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found