Failing to connect to Confluent Cloud Kafka

Sorry for the typo in https://github.com/provectus/kafka-ui/issues/920. After I fix this and use these settings

• KAFKA_CLUSTERS_0_NAME=azureDev
• KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=<server-name>.azure.confluent.cloud:9092
• KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL=SASL_SSL
• KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM=PLAIN
• KAFKA_CLUSTERS_0_PROPERTIES_CLIENT_DNS_LOOKUP=use_all_dns_ips
• KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG='org.apache.kafka.common.security.plain.PlainLoginModule required username=“*******” password=“***********”;’
• KAFKA_CLUSTERS_0_DISABLELOGDIRSCOLLECTION=true

I get org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient

In detail:

kafka-ui  | 16:04:57.390 [main] INFO  org.springframework.core.KotlinDetector - Kotlin reflection implementation not found at runtime, related features won't be available.
kafka-ui  |
kafka-ui  |   .   ____          _            __ _ _
kafka-ui  |  /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
kafka-ui  | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
kafka-ui  |  \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
kafka-ui  |   '  |____| .__|_| |_|_| |_\__, | / / / /
kafka-ui  |  =========|_|==============|___/=/_/_/_/
kafka-ui  |  :: Spring Boot ::        (v2.2.4.RELEASE)
kafka-ui  |
kafka-ui  | 16:04:57.981 [main] INFO  com.provectus.kafka.ui.KafkaUiApplication - Starting KafkaUiApplication on b64d97c898ac with PID 1 (/kafka-ui-api.jar started by root in /)
kafka-ui  | 16:04:57.982 [main] DEBUG com.provectus.kafka.ui.KafkaUiApplication - Running with Spring Boot v2.2.4.RELEASE, Spring v5.2.3.RELEASE
kafka-ui  | 16:04:57.982 [main] INFO  com.provectus.kafka.ui.KafkaUiApplication - No active profile set, falling back to default profiles: default
kafka-ui  | 16:04:58.253 [background-preinit] WARN  org.springframework.http.converter.json.Jackson2ObjectMapperBuilder - For Jackson Kotlin classes support please add "com.fasterxml.jackson.module:jackson-module-kotlin" to the classpath
kafka-ui  | SLF4J: Class path contains multiple SLF4J bindings.
kafka-ui  | SLF4J: Found binding in [jar:file:/kafka-ui-api.jar!/BOOT-INF/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
kafka-ui  | SLF4J: Found binding in [jar:file:/kafka-ui-api.jar!/BOOT-INF/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
kafka-ui  | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
kafka-ui  | SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
kafka-ui  | log4j:WARN No appenders could be found for logger (reactor.util.Loggers$LoggerFactory).
kafka-ui  | log4j:WARN Please initialize the log4j system properly.
kafka-ui  | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
kafka-ui  | 16:04:59.357 [main] WARN  org.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration$JodaDateTimeJacksonConfiguration - Auto-configuration of Jackson's Joda-Time integration is deprecated in favor of using java.time (JSR-310).
kafka-ui  | 16:04:59.388 [main] INFO  com.provectus.kafka.ui.serde.DeserializationService - Using SchemaRegistryAwareRecordSerDe for cluster 'azureDev'
kafka-ui  | 16:04:59.820 [main] INFO  org.springframework.boot.autoconfigure.security.reactive.ReactiveUserDetailsServiceAutoConfiguration -
kafka-ui  |
kafka-ui  | Using generated security password: *************
kafka-ui  |
kafka-ui  | 16:05:00.341 [main] INFO  org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler - Initializing ExecutorService 'taskScheduler'
kafka-ui  | 16:05:00.447 [parallel-1] DEBUG com.provectus.kafka.ui.service.MetricsUpdateService - Start getting metrics for kafkaCluster: azureDev
kafka-ui  | 16:05:00.620 [main] INFO  org.springframework.boot.web.embedded.netty.NettyWebServer - Netty started on port(s): 8080
kafka-ui  | 16:05:00.624 [main] INFO  com.provectus.kafka.ui.KafkaUiApplication - Started KafkaUiApplication in 3.173 seconds (JVM running for 4.416)
kafka-ui  | 16:05:00.810 [parallel-1] ERROR com.provectus.kafka.ui.service.KafkaService - Failed to collect cluster azureDev info
kafka-ui  | org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient
kafka-ui  | 	at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:540) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at org.apache.kafka.clients.admin.Admin.create(Admin.java:134) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:39) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at com.provectus.kafka.ui.service.AdminClientServiceImpl.lambda$createAdminClient$2(AdminClientServiceImpl.java:41) ~[classes!/:?]
kafka-ui  | 	at reactor.core.publisher.MonoSupplier.call(MonoSupplier.java:85) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:126) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.Mono.subscribe(Mono.java:4090) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:75) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.Operators.complete(Operators.java:132) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.MonoEmpty.subscribe(MonoEmpty.java:45) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.Mono.subscribe(Mono.java:4105) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:418) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:398) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:484) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
kafka-ui  | 	at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
kafka-ui  | 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [?:?]
kafka-ui  | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
kafka-ui  | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
kafka-ui  | 	at java.lang.Thread.run(Unknown Source) [?:?]
kafka-ui  | Caused by: java.lang.IllegalArgumentException: Login module control flag not specified in JAAS config
kafka-ui  | 	at org.apache.kafka.common.security.JaasConfig.parseAppConfigurationEntry(JaasConfig.java:110) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:63) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:88) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:82) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:167) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:513) ~[kafka-clients-2.8.0.jar!/:?]
kafka-ui  | 	... 22 more

[dmcarba]

Hello,

I’ve found this post because I was getting the same error: org.apache.kafka.common.errors.ClusterAuthorizationException. For me the problem was the ACLs of the user, you need to enable both describe and describe config for the Cluster in order to remove this error.

That’s a problem because there are different tenants accessing the cluster with segregated ACLs permission on topics and group ids based on different prefix strings per each tenant. Allowing this cluster visibility is a potential risk, since each tenant can then see the other topic names and ACLs.

I wonder if there is the possibility of making the access to the extra cluster information optional so we can still use kafka-ui without allowing these permissions.

Thanks

[Haarolean]

Don’t give up yet. Since it works for me, I’ll provide you a test build with extra debugging to check it out, deal?

On 11 Oct 2021, at 21:32, Ron Harlev @.***> wrote:

Thanks for testing it!!!

I copied your configuration as is, only replacing the bootstrap server name, username and password. I still get org.apache.kafka.common.errors.ClusterAuthorizationException: Cluster authorization failed.

I changed the username to a wrong one, just to see the difference. I get, as expected org.apache.kafka.common.errors.SaslAuthenticationException: Authentication failed

May be something specific to our cluster. Willing to give up at this point 😦

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.