Trying to set up ssl kafka output in filebeat. reading a log file and sending to kafka.
non-ssl works fine. for ssl i am trying the below
#---- kafka output
output.kafka:
# initial brokers for reading cluster metadata
hosts: ["host1:9093",
"host2:9093", "host3:9093"
ssl.certificate: /test/ssl/monitoring.crt
ssl.certificate_key: /test/ssl/monitoring.key
ssl.certificate_authorities: ["/test/ssl/digicertroot.cer"]
# message topic selection + partitioning
topic: 'filebeattest'
partition.round_robin:
reachable_only: false
required_acks: 1
compression: gzip
max_message_bytes: 1000000
I get the below error
ERROR instance/beat.go:958 Exiting: error initializing publisher: key file not
configured accessing 'output.kafka.ssl' (source:'filebeat.yml')
Exiting: error initializing publisher: key file not configured accessing
'output.kafka.ssl' (source:'filebeat.yml')
stephenb
(Stephen Brown)
April 9, 2021, 3:48am
2
Hi @madhan_dhanikachalam Welcome to the community.
First please make sure you format your yaml correctly.
You have several syntax issues.
hosts: ["host1:9093",
"host2:9093", "host3:9093" <!---- Missing Closing Bracket and should be on the same line
Please review the SSL configuration Docs Here
Incorrect
ssl.certificate_key: /test/ssl/monitoring.key
ssl.certificate_authorities: ["/test/ssl/digicertroot.cer"]
should be
ssl.certificate.key: /test/ssl/monitoring.key
ssl.certificate.authorities: ["/test/ssl/digicertroot.cer"]
Fix those see what happens...
indent preformatted text by 4 spaces
output.kafka:
hosts: ["host1:9093","host2:9093", "host3:9093"]
ssl.certificate: /test/ssl/monitoring.crt
ssl.certificate.key: /test/ssl/monitoring.key
ssl.certificate.authorities: ["/test/ssl/digicertroot.cer"]
topic: 'filebeattest'
partition.round_robin:
reachable_only: false
required_acks: 1
compression: gzip
max_message_bytes: 1000000
formatting was good, maybe it was just me pasting on the forum. Changed the config field and now i am getting the below error
Exiting: error loading config file: required 'object', but found 'string' in field
'ssl.certificate' (source:'filebeat.yml')
turns out
ssl.certificate.key:
ssl.certificate.authorities:
should be
ssl.key:
ssl.authorities:
worked with this change.
1 Like
stephenb
(Stephen Brown)
April 9, 2021, 5:12pm
5
Yep apologies cut n pasta error my side
system
(system)
Closed
May 7, 2021, 7:13pm
6
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.