[Enabling one way TLS authentication in Kafka] ...

Hi Team,

I have a question regarding the enabling of the one way TLS authentication, How do we achieve this and what change do we need in the Kafka CR.

Let’s say if I want to test producing messages on a particular topic on internal bootstrap with 9093, Do I need to disable the following configuration and then call the producer.sh by using only the Cluster CA crt, Is this called as 1-way TLS authentication?

tls:
        authentication:
          type: tls

./bin/kafka-console-producer.sh --broker-list mm-backup-cluster-kafka-bootstrap:9093 --topic mm-src-cluster.mm2-topic \
--producer-property security.protocol=SSL \
--producer-property ssl.truststore.type=PKCS12 \
--producer-property ssl.truststore.password=123456 \
--producer-property ssl.truststore.location=/tmp/certs/cluster.truststore.p12 

need your inputs and help here.

[vperi1730]

I have another question here, after enabling the tls: {}, Now i am able to hit the request with ANONYMOUS user, However, one of the issues I see is, ANONYMOUS doesn’t have access to a particular topic.

Question is, Do we need to manually add the ACLs for the ANONYMOUS user, or are they enable by default?

 ./bin/kafka-console-producer.sh --broker-list mm-backup-cluster-kafka-bootstrap:9093 --topic mm-src-cluster.mm2-topic \
> --producer-property security.protocol=SSL \
> --producer-property ssl.truststore.type=PKCS12 \
> --producer-property ssl.truststore.password=123456 \
> --producer-property ssl.truststore.location=/tmp/certs/cluster.truststore.p12
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
>ok
[2020-07-03 06:39:11,222] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {mm-src-cluster.mm2-topic=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2020-07-03 06:39:11,223] ERROR [Producer clientId=console-producer] Topic authorization failed for topics [mm-src-cluster.mm2-topic] (org.apache.kafka.clients.Metadata)
[2020-07-03 06:39:11,224] ERROR Error when sending message to topic mm-src-cluster.mm2-topic with key: null, value: 2 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [mm-src-cluster.mm2-topic]

here is my ACL’s list which doesn’t show any ANONYMOUS list.

./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=127.0.0.1:2181 --list
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Current ACLs for resource `Topic:LITERAL:*`:
        User:CN=client-producer-user has Allow permission for operations: Read from hosts: *
        User:CN=client-producer-user has Allow permission for operations: Write from hosts: *
        User:CN=client-producer-user has Allow permission for operations: Create from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Write from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Describe from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Create from hosts: *
        User:CN=client-consumer-user has Allow permission for operations: Write from hosts: *
        User:CN=client-producer-user has Allow permission for operations: Describe from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Read from hosts: *
        User:CN=client-consumer-user has Allow permission for operations: Read from hosts: *
        User:CN=client-consumer-user has Allow permission for operations: Describe from hosts: *
        User:CN=client-consumer-user has Allow permission for operations: Create from hosts: *

Current ACLs for resource `Group:LITERAL:connect-cluster`:
        User:CN=client-consumer-user has Allow permission for operations: All from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Describe from hosts: *

1. For disabling tls in external does the following configuration looks correct??

external:
        tls: false
        type: loadbalancer

[alokhom]

Kafka spec

apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  creationTimestamp: '2020-08-25T14:38:29Z'
  generation: 11
  name: my-cluster
  namespace: kafka
  resourceVersion: '2334765'
  selfLink: /apis/kafka.strimzi.io/v1beta1/namespaces/kafka/kafkas/my-cluster
  uid: 984954e3-0c68-4ea0-a9e0-0d9e8f55f4a6
spec:
  entityOperator:
    topicOperator:
      reconciliationIntervalSeconds: 90
    userOperator:
      reconciliationIntervalSeconds: 120
  kafka:
    authorization:
      type: simple
    config:
      log.message.format.version: '2.5'
      offsets.topic.replication.factor: 3
      transaction.state.log.min.isr: 2
      transaction.state.log.replication.factor: 3
    listeners:
      external:
        authentication:
          type: tls
        overrides:
          bootstrap:
            host: bootstrap.apps.kafka.cluster42.openshift-enfo.se
          brokers:
            - broker: 0
              host: broker-0.apps.kafka.cluster42.openshift-enfo.se
            - broker: 1
              host: broker-1.apps.kafka.cluster42.openshift-enfo.se
            - broker: 2
              host: broker-2.apps.kafka.cluster42.openshift-enfo.se
        type: route
      plain:
        authentiation:
          type: scram-sha-512
      tls:
        authentiation:
          type: tls
    replicas: 3
    storage:
      class: rook-ceph-block
      size: 20Gi
      type: persistent-claim
    version: 2.5.0
  zookeeper:
    replicas: 3
    storage:
      type: ephemeral

Kafka Topic spec -

apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaTopic
metadata:
  creationTimestamp: '2020-08-25T14:41:13Z'
  generation: 1
  labels:
    strimzi.io/cluster: my-cluster
  name: my-topic
  namespace: kafka
  resourceVersion: '1468960'
  selfLink: /apis/kafka.strimzi.io/v1beta1/namespaces/kafka/kafkatopics/my-topic
  uid: 52d65111-ba4f-49ad-bb56-1db93b35d60b
spec:
  config:
    retention.ms: 604800000
    segment.bytes: 1073741824
  partitions: 10
  replicas: 3

kafkaUser spec-

apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaUser
metadata:
  creationTimestamp: '2020-08-26T20:03:09Z'
  generation: 3
  labels:
    strimzi.io/cluster: my-cluster
  name: my-user
  namespace: kafka
  resourceVersion: '2336125'
  selfLink: /apis/kafka.strimzi.io/v1beta1/namespaces/kafka/kafkausers/my-user
  uid: ecab2527-000a-47cb-bae8-e24f31bd2406
spec:
  authentication:
    type: tls
  authorization:
    acls:
      - host: '*'
        operation: Read
        resource:
          name: my-topic
          patternType: literal
          type: topic
      - host: '*'
        operation: Describe
        resource:
          name: my-topic
          patternType: literal
          type: topic
      - host: '*'
        operation: Read
        resource:
          name: my-group
          patternType: literal
          type: group
      - host: '*'
        operation: Write
        resource:
          name: my-topic
          patternType: literal
          type: topic
      - host: '*'
        operation: Create
        resource:
          name: my-topic
          patternType: literal
          type: topic
      - host: '*'
        operation: Describe
        resource:
          name: my-topic
          patternType: literal
          type: topic
    type: simple

I applied TLS on external and internal listeners I extracted the CA.cert to make a truststore. I extracted the user.p12 and user.password to make the keystore i used groupID my-group for Kafka Consumer to read data from topic my-user it works now to use an external Camel route application to connect to a Openshift Strimzi !!!

Request: It would be great if you provided a portal for the strimzi from the operator which:

1. could show us data in the topics
2. help configure/select the CA/User files from the namespace
3. check the ZK
4. provide a console that would help run the Kafka CLI commands like listing ACLs / etc. Other commands of Kafka.
5. give us realtime/basic/5 min old analytics
6. add/remove brokers make it a more easy for non Kafka technicians to do some day-2 operations. etc.