Question: how can I consume this array of Streams?

Hello, nice folks!

I have a function that makes a request and returns a stream.

const getUser = (userId) => request(`https://www.myapi.com/users?id=${userId}`)

I have an array of users’ IDs and I want to call getUser on each one of them:

const users = [
  159753,
  753159,
  654789,
  159753,
  753368
]

*Most of the time I don’t know how many users I will have in my array.

Now I want to create an array of streams:

const requests = users.reduce((arr, user) => [...arr, getUser(user)], [])

What I am trying to do is iterate over requests and pipe its response to http.ServerResponse:

app.get('/', (req, res) => {
  _(requests).pipe(res)
})

When I do this I get an error: Stream already being consumed, you must either fork() or observe()

What I tried is something like _(requests.fork()).pipe(res) but it obviously didn’t work.

It might be something silly that I’m missing but if anyone could point me to what am I doing wrong, or have any hint, I’d really appreciate.

Thank you!

[vqvu]

The reason why I said you didn’t need toBuffer was because you were making the secondary call to myapi.com, which already returned buffers. Now that you’re not doing that anymore, you need to add toBuffer (or its equivalent) back in.

Change filter to

const filter = (stream) => stream.map(parseJSON).map(filterSubmissions).map(JSON.stringify);

The server outputs when fetching the test jl user.

[14237416,11871616,11483492]

[viniciusCamargo]

I had to pass that data to a new Buffer, otherwise, I’d get a TypeError: First argument must be a string or Buffer. I refactored the code and changed some functions names to help, but it still doesn’t work.

const express = require('express')
const rp = require('request-promise')
const _ = require('highland')

const app = express()

const parseJSON = (buffer) => JSON.parse(buffer.toString())

const filterSubmissions = (data) => {
  const { submitted } = data

  return submitted.filter((e, i) => i < 3) // returns the first 3 entries
}

const filter = (stream) => stream.map(parseJSON).map(filterSubmissions)

const getUser = (id) => `https://hacker-news.firebaseio.com/v0/user/${id}.json?print=pretty`

app.get('/', (req, res) => {
  _(rp(getUser('user'))).through(filter).pipe(res)
})

app.listen(3000, () => console.log('http://localhost:3000'))