How to parse file and create a readable stream to send in a http request.

I want to parse a file and create a readable stream that can be sent to another service in a http request.

Uploadmiddleware

export const UploadMiddleware = (req: Request, res: Response, next: NextFunction) => {

    const busboy = new Busboy({ headers: req.headers });
    busboy.on("file", (fieldname, file, filename, encoding, mimetype) => {
        req.body.image = file;
    });
    busboy.on("field", (fieldname, val, fieldnameTruncated, valTruncated, encoding, mimetype) => {
        req.body[fieldname] = val;
    });
    busboy.on("finish", () => {
        next();
    });
    req.pipe(busboy);
}

This upload middleware is adding the file to the req.body.image field. This field now contains this stream.

File Stream

  image: FileStream {
    _readableState: ReadableState {
      objectMode: false,
      highWaterMark: 16384,
      buffer: BufferList { head: null, tail: null, length: 0 },
      length: 0,
      pipes: null,
      pipesCount: 0,
      flowing: true,
      ended: true,
      endEmitted: true,
      reading: false,
      sync: false,
      needReadable: false,
      emittedReadable: false,
      readableListening: false,
      resumeScheduled: false,
      paused: false,
      emitClose: true,
      autoDestroy: false,
      destroyed: false,
      defaultEncoding: 'utf8',
      awaitDrain: 0,
      readingMore: false,
      decoder: null,
      encoding: null
    },
    readable: false,
    _events: [Object: null prototype] { end: [Array], data: [Function] },
    _eventsCount: 2,
    _maxListeners: undefined,
    truncated: false,
    _read: [Function]
  },

However when i try to post this stream to another service using axios. It does not like it and the other service times out. I believe this stream is not something the other service can read.

Http request to the other service

      const formData = new FormData();
      formData.append("image", request.image); //req.image is the above stream

      const options: any = {
            url,
            method: "post",
            headers: formData.getHeaders(),
            data: formData,
       };

        await axios(options)

Side Note:

The stream obtained from fs.createReadStream is something that the other service can read when sent in the same http post request using axios. That stream looks like the following.

  image: ReadStream {
    _readableState: ReadableState {
      objectMode: false,
      highWaterMark: 65536,
      buffer: BufferList { head: null, tail: null, length: 0 },
      length: 0,
      pipes: null,
      pipesCount: 0,
      flowing: null,
      ended: false,
      endEmitted: false,
      reading: false,
      sync: true,
      needReadable: false,
      emittedReadable: false,
      readableListening: false,
      resumeScheduled: false,
      paused: true,
      emitClose: false,
      autoDestroy: false,
      destroyed: false,
      defaultEncoding: 'utf8',
      awaitDrain: 0,
      readingMore: false,
      decoder: null,
      encoding: null
    },
    readable: true,
    _events: [Object: null prototype] { end: [Function] },
    _eventsCount: 1,
    _maxListeners: undefined,
    path: 'front.jpg',
    fd: null,
    flags: 'r',
    mode: 438,
    start: undefined,
    end: Infinity,
    autoClose: true,
    pos: undefined,
    bytesRead: 0,
    closed: false
  },

---

Can i convert this Filestream obtained from busboy into a stream that can be sent in a http post request to be read.

[ghost]

Okay so if anyone else is interested this is how i solved the problem. In the end i stored the bytes in an array and then created a new Readable Stream because i couldint send original file stream via axios(http) to another service.

      const f = [];
      busboy.on("file", (fieldname, file, filename, encoding, mimetype) => {
            file.on("data", (data) => {
                f.push(data);
            });
            file.on("end", () => {
                const fileStream = createReadableStream(f);
                req.body[fieldname] = fileStream;
            });
        });

const createReadableStream = (buffer) => {
    const readableInstanceStream = new Readable({
        read() {
            for (const bytes of buffer) {
                this.push(bytes);
            }
            this.push(null);
        },
    });
    return readableInstanceStream;
};

[Asheboy]

@mscdex Should this be closed? It’s definitely still an issue and I’m not sure there’s an explanation as to why this happens.