Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

KTB are not providing correct ACLs for streams apps

See original GitHub issue

KTB fails to provide correct ACLs for streams apps. According to the documentation the following ACLs are needed:

  • Topic resource (for internal topics): READ, DELETE, WRITE, CREATE (prefixed)
    • Or the ALL operation as KTB uses today
  • Consumer Group resource: READ, DESCRIBE
  • Topic resource (for input topic): READ
  • Topic resource (for output topic): WRITE

KTB does not provide ACLs for the consumer group. Also for the internal topics the ALL operation is granted, instead more restricted operations should be granted.

Another problem is related to the feature for custom topic naming in KTB. This allows for topic naming with just the name given for the topic in yaml, then no topic prefix exists. So KTB must support some other way to give the prefix in these cases. Ref. the documentation linked above then the option is to provide the applicationId for the streams app.

Currently the following ACL is added in KTB to support streams app internal topics:

'TOPIC', <topicPrefix>, '*', 'ALL', 'User:StreamsUser', 'PREFIX'

When an applicationId is given for the streams app in the topology this should changed to:

'TOPIC', <streamApplicationId>, '*', 'ALL', 'User:StreamsUser', 'PREFIX'

The topology might then look like this: the streams section has a new key applicationId. E.g.:

  - name: "projectA"
      - principal: "User:StreamsUser"
        applicationId: "aStreamsApp"
            - "topicA"
            - "topicB"
            - "topicC"
            - "topicD"

Internally this field will then be used to create prefix-based ACLs for both consumer group and internal topics.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (5 by maintainers)

github_iconTop GitHub Comments

akselhcommented, Nov 26, 2020

@purbon I have changed this issue after new findings. This should be fixed pretty soon!

I can help out, but not before next week. If I do so I would only ask for timely review of the PR. 😉

akselhcommented, Nov 27, 2020

I suggest removing the ALL to READ, DELETE, WRITE, CREATE acl splitting for internal topics to another issue. If we want to change this at all. When using applicationId the permissions will be restricted to only streams internal topics anyway, and a huge amount of acls can possibly affect performance of Kafka at runtime.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Controlling the access - Kafka Topology Builder documentation
ACLs ¶. In the topology descriptor files users can create permissions for different types of applications, Consumers, Producers, Kafka streams apps or Kafka ......
Read more >
AAPL Stock Price — Apple Chart - TradingView
View live Apple Inc chart to track its stock's price action. Find market predictions, AAPL financials and market news.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found