I have created a MSK cluster with 3 brokers and enabled encryption at rest and in transit but
disabled TLS authentication.
I'm trying to send data from Filebeat (7.10.0) --> AWS MSK over TLS endpoint (9094).
Filebeat in installed in one of the AWS EC2 instance within same VPC of AWS MSK cluster.
Also included the below value in filebeat.yml o/p section:
We have generated the kafka.client.truststore.jks by coping from Java . Refer: [+] Step 6: Produce and Consume Data - Step 6: Produce and Consume Data - Amazon Managed Streaming for Apache Kafka
From Filebeat Log I'm getting the below msg:
2021-02-12T14:22:33.888Z DEBUG https://forums.aws.amazon.com/ kafka/client.go:291 Kafka publish failed with: circuit breaker is open
I also enabled cloud watch logs for the broker and on the broker side I see the following error message "SocketServer brokerId=1 Failed authentication with /INTERNAL_IP (SSL handshake failed) (org.apache.kafka.common.network.Selector)"
Could you please help out here, many thanks?