Unable to have it in Running within Azure Blob Storage

Hey all,

In my File Pulse Connector I have the following configurations. I am using 2.3.0. We use Confluent Cloud and have our Kafka Connect running on Confluent for Kubernetes (CFK). CFK and our Kafka Connect authenticates onto Confluent Cloud.

“tasks.reader.class”: “io.streamthoughts.kafka.connect.filepulse.fs.AzureBlobStorageConfig”, “fs.listing.class”: “io.streamthoughts.kafka.connect.filepulse.fs.AzureBlobStorageFileSystemListing”,

When putting it in the rest API, I am getting the following error

"message": "Connector configuration is invalid and contains the following 2 error(s):\nInvalid value io.streamthoughts.kafka.connect.filepulse.fs.AzureBlobStorageFileSystemListing for configuration fs.listing.class: Class io.streamthoughts.kafka.connect.filepulse.fs.AzureBlobStorageFileSystemListing could not be found.\nInvalid value io.streamthoughts.kafka.connect.filepulse.fs.AzureBlobStorageConfig for configuration tasks.reader.class: Class io.streamthoughts.kafka.connect.filepulse.fs.AzureBlobStorageConfig could not be found.\nYou can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`"

I ended up putting an on premise Kafka on my local machine to test things out.

“tasks.reader.class”: “io.streamthoughts.kafka.connect.filepulse.fs.AzureBlobStorageConfig”, “fs.listing.class”: “io.streamthoughts.kafka.connect.filepulse.fs.AzureBlobStorageFileSystemListing”,

I was able to create the connector.

Compared to our local, on Confluent Cloud, we do have a bunch of other connectors. Meanwhile, on our Confluent Cloud that leverages CFK, we installed it via the confluent-hub CLI whereas on my on premise machine, I simply had the connector built when running the docker compose.

confluent-hub install streamthoughts/kafka-connect-file-pulse:2.3.0

Has anyone had this issue before? I followed the documentation thoroughly and it seems to be a bit odd.

[fhussonnois]

Hi @kjang1750, it seems that connect file 2.3 available on confluent-hub does not contain the extension for Azure and other could providers, see : https://github.com/streamthoughts/kafka-connect-file-pulse/issues/182

This will be fixed in next version (2.4) that should be released very soon.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.