KTB are not providing correct ACLs for streams apps

KTB fails to provide correct ACLs for streams apps. According to the documentation the following ACLs are needed:

• Topic resource (for internal topics): READ, DELETE, WRITE, CREATE (prefixed)
  • Or the ALL operation as KTB uses today
• Consumer Group resource: READ, DESCRIBE
• Topic resource (for input topic): READ
• Topic resource (for output topic): WRITE

KTB does not provide ACLs for the consumer group. Also for the internal topics the ALL operation is granted, instead more restricted operations should be granted.

Another problem is related to the feature for custom topic naming in KTB. This allows for topic naming with just the name given for the topic in yaml, then no topic prefix exists. So KTB must support some other way to give the prefix in these cases. Ref. the documentation linked above then the option is to provide the applicationId for the streams app.

Currently the following ACL is added in KTB to support streams app internal topics:

'TOPIC', <topicPrefix>, '*', 'ALL', 'User:StreamsUser', 'PREFIX'

When an applicationId is given for the streams app in the topology this should changed to:

'TOPIC', <streamApplicationId>, '*', 'ALL', 'User:StreamsUser', 'PREFIX'

The topology might then look like this: the streams section has a new key applicationId. E.g.:

projects:
  - name: "projectA"
    streams:
      - principal: "User:StreamsUser"
        applicationId: "aStreamsApp"
        topics:
          read:
            - "topicA"
            - "topicB"
          write:
            - "topicC"
            - "topicD"

Internally this field will then be used to create prefix-based ACLs for both consumer group and internal topics.

[akselh]

@purbon I have changed this issue after new findings. This should be fixed pretty soon!

I can help out, but not before next week. If I do so I would only ask for timely review of the PR. 😉

[akselh]

I suggest removing the ALL to READ, DELETE, WRITE, CREATE acl splitting for internal topics to another issue. If we want to change this at all. When using applicationId the permissions will be restricted to only streams internal topics anyway, and a huge amount of acls can possibly affect performance of Kafka at runtime.