The producer attempted a transactional operation in an invalid state

So, using the example given here, I tried to set up a transactional producer.

client:

const client = new Kafka({
  clientId: 'my-client',
  brokers: ['my-broker'],
})
const producer = client.producer({ 
  maxInFlightRequests: 1, 
  idempotent: true,
  transactionalId: 'my-transaction'
})

transaction:

const  transaction = await producer.transaction()

try {
  await transaction.send({ topic, messages })

  await transaction.commit()
} catch (e) {
  await transaction.abort()
}

As long as I send and commit without issue, it works. However, if any error is thrown, and abort is called, I get the following error:

[Connection] Response EndTxn(key: 26, version: 0)
    broker: "my-broker"
    clientId: "my-client"
    error: "The producer attempted a transactional operation in an invalid state"
    correlationId: 1
    size: 10

    {
      "type": "KafkaJSProtocolError",
      "message": "The producer attempted a transactional operation in an invalid state",
      "stack":
          KafkaJSProtocolError: The producer attempted a transactional operation in an invalid state
              at createErrorFromCode (/Users/eliw00d/Documents/GitHub/my-project/node_modules/kafkajs/src/protocol/error.js:536:10)
              at Object.parse (/Users/eliw00d/Documents/GitHub/my-project/node_modules/kafkajs/src/protocol/requests/endTxn/v0/response.js:24:11)
              at Connection.send (/Users/eliw00d/Documents/GitHub/my-project/node_modules/kafkajs/src/network/connection.js:306:35)
              at process._tickCallback (internal/process/next_tick.js:68:7)
      "name": "KafkaJSProtocolError",
      "retriable": false,
      "code": 48
    }

To better illustrate the issue, I even tried:

const transaction = await producer.transaction()
await transaction.abort()

with the same result.

Is this a bug or am I missing something?

[konapun]

(Disclaimer: I’m not a kafkajs maintainer) I’ve looked into this a bit and it looks like the error is caused by the kafka backend not transitioning to a transacting state until producer.send is called so if the kafkajs client attempts to abort a transaction before the send is attempted you’ll receive that error.

To get around this I’m wrapping the abort in a try/catch and swallowing the error if its code is 48:

try {
    transaction.abort()
} catch (err) {
    if (err.code !== 48) {
        throw err
    }
}

This works for now but I would like to see the actual bug get fixed in kafkajs.

[tulios]

@eliw00d I like your proposal of a transaction.isAbortable(), I will check how the Java client work with this use-case and I can start a PR.