bulk insert not working - newline separated JSON body not working

🐛 Bug Report

Cannot insert newline separate JSON

To Reproduce

  const v = lst.map(v => JSON.stringify({doc: v.value})).join('\n') + '\n';

  client.bulk({
    index: 'foo',
    body: v
  })
   .catch(e => {
     log.error(e);
   });

I get this error:

      ResponseError: illegal_argument_exception
      at IncomingMessage.<anonymous> (/Users/alex/codes/interos/@interos/elastic-search/node_modules/@elastic/elasticsearch/lib/Transport.js:287:25)
      at IncomingMessage.emit (events.js:208:15)
      at endReadableNT (_stream_readable.js:1154:12)
      at processTicksAndRejections (internal/process/task_queues.js:77:11)

Why doesn’t the stack trace give me more to work with btw? ResponseError: illegal_argument_exception with no further information does not give me much to work with 😦

Your Environment

• node version: 12
• “@elastic/elasticsearch”: “^7.3.0”,
• os: Mac

This however does work fine:

  client.index({
      index: 'foo',
      body: {
        doc: v.value
      }
    })
     .catch(e => {
       log.error(e);
     });

[delvedor]

What is log in your code?

Take this example to reproduce the error:

'use strict'

const { Client } = require('@elastic/elasticsearch')
const client = new Client({ node: 'http://localhost:9200' })

client.indices.create({
  index: 'test',
  body: {
    mappings: {
      properties: {
        foo: { type: 'integer' }
      }
    }
  }
}, { ignore: [400] }, (err, result) => {
  if (err) return console.log(err)
  client.index({
    index: 'test',
    body: { foo: 'bar' }
  }, (err, result) => {
    if (err) return console.log(err)
  })
})

The code above produces the following log on Node.js v12:

ResponseError: mapper_parsing_exception
    at IncomingMessage.<anonymous> (.../elasticsearch-js/lib/Transport.js:287:25)
    at IncomingMessage.emit (events.js:201:15)
    at endReadableNT (_stream_readable.js:1130:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:9) {
  name: 'ResponseError',
  meta: {
    body: { error: [Object], status: 400 },
    statusCode: 400,
    headers: {
      'content-type': 'application/json; charset=UTF-8',
      'content-length': '459'
    },
    warnings: null,
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    }
  }
}

As you can see, the full error object is logged and the properties are there. I think you are using a custom logger that does not know how to serialize the Error objects, and by default, it logs only the error message and stacktrace.

[delvedor]

Hello! If you inspect the error object you should be able to see many other fields. Unfortunately, they are not well documented in the docs, I’ll take care of this. In the meantime, you can find the error classes here: https://github.com/elastic/elasticsearch-js/blob/f7be49f2bafa3b9ddc90b0caac5b70932e6db35b/lib/errors.js#L71-L94

Regarding the bulk body, I think you are not creating the body correctly. I would suggest you take a look at this example 😃