Is it possible to speed up exchange.fetchTickers()

• OS: Windows 10
• Programming Language version: Node.js
• CCXT version: #465
• Exchange: Bitfinex, Binance etc
• Node.js 11.9.0 installed
• Method: exchange.fetchTickers()

Hello!

I just have a general question for: exchange.fetchTickers() I am doing some tests and exchange.fetchTickers() for 60 exchanges. This can take about 10 minutes before all code has run. I beleive this should depend on that HTTP requests are “slow”?

My question is if there is any solution to speed up this code in Node.js? I have tried to create 60 threads in parallell but that does not make it faster though.

The approach I use is with the below code. I know that I create files which could take some time (I/O) but apart from that if there is a better approach to this?

I run the program with the command: node workerTickersCreator.js

Below .js file starts 2 threads where each thread has its own .js file seen after this code:

workerTickersCreator.js

var cluster = require('cluster');
var threads = 2;

for (var i = 0; i < threads; i++)
{
       //There are 133 exchanges in this loop normally
       if(i == 0) { cluster.setupMaster({ exec: 'workerTickersbinance.js', }); cluster.fork(); }
       if(i == 1) { cluster.setupMaster({ exec: 'workerTickersbitfinex.js', }); cluster.fork(); }
}

Above code calls the below 2 other .js files which: “exchange.fetchTickers()”

workerTickersbinance.js

'use strict';
const ccxt = require ('ccxt');

(async () => {
            try{
       const exchange = new ccxt.binance({ enableRateLimit: true })
       const tickers = await exchange.fetchTickers()
       const obj = { tickers }
       const fs = require('fs');
       fs.writeFile("/myproject/markets/binanceTickers.txt", JSON.stringify(obj), function(err) { });
       }catch{}
}) ()
process.exitCode = 1;

workerTickersbitfinex.js

'use strict';
const ccxt = require ('ccxt');

(async () => {
            try{
       const exchange = new ccxt.bitfinex({ enableRateLimit: true })
       const tickers = await exchange.fetchTickers()
       const obj = { tickers }
       const fs = require('fs');
       fs.writeFile("/myproject/markets/bitfinexTickers.txt", JSON.stringify(obj), function(err) { });
       }catch{}
}) ()
process.exitCode = 1;

Thank you!

[kroitor]

Are we talking about doing this code approach splitting the work up on different cores using for example “cluster” ?

Nope, not really… first of, you don’t want to add the overhead that you don’t completely understand. The mere fact that you’re asking this question kinda tells that you don’t. In that case, I suggest that you first obtain the necessary skills with modern ES6 JavaScript (find out what is ES6 – that is a big self-educational task already), learn about Promises and how to use async/await in pure JavaScript, without additional “clusters” or whatever.

You can shrink the execution time down to 3 seconds with proper management of async promises and with a bit smarter caching, so this is possible, if you:

• understand the language very well
• understand how exchanges work economically
• understand how exchanges’ APIs work technically
• understand the Manual for this lib
• understand the examples for this lib

I can do that script for you, but if I do – you won’t learn as much as you want, really. And tbh, I don’t have the time to teach everyone personally, that’s physically impossible, hope for your understanding. All I can give is just a clue on where to dig deeper 😉

[jackpotcityco]

I don’t have the time to teach everyone personally

Yes I completely understand that I have the same problem with C# to learn everyone. There is no time for that 😉

I am really greatful for where I should start looking in order to learn more about the javascript language and where to dig deeper. It helps to know on what area to focus. Thank you! 😃