Hi!
I configured Filebeat with module kafka.
/opt/filebeat/config/filebeat.ym
filebeat.inputs:
- type: log
enabled: false
paths:
- /var/log/*.log
- type: filestream
enabled: false
paths:
- /var/log/*.log
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
setup.kibana:
host: "https://el2:5601"
ssl.enabled: true
ssl.certificate_authorities: ["/etc/certificates/salt-ca.crt"]
ssl.certificate: "/etc/certificates/kafka-1.crt"
ssl.key: "/etc/certificates/kafka-1.key"
setup.template.settings:
index.number_of_shards: 1
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
output.elasticsearch:
hosts: ["https://el1:9200", "https://el2:9200", "https://el3:9200", "https://el4:9200"]
ssl.enabled: true
ssl.verification_mode: "full"
ssl.certificate_authorities: ["/etc/certificates/salt-ca.crt"]
ssl.certificate: "/etc/certificates/kafka-1.crt"
ssl.key: "/etc/certificates/kafka-1.key"
username: "elastic"
password: "Passwd1234"
/opt/filebeat/config/modules.d/kafka.yml
- module: kafka
# All logs
log:
enabled: true
var.paths:
- "/opt/kafka/logs/controller.log*"
- "/opt/kafka/logs/server.log*"
- "/opt/kafka/logs/state-change.log*"
- "/opt/kafka/logs/kafka-*.log*"
var.kafka_home: /opt/kafka
As I understood from Kafka module | Filebeat Reference [7.7] | Elastic and Apache Kafka Monitoring with Elasticsearch, Beats, and Kibana | by Imteyaz Ahmad | Medium, Dashboard [Filebeat Kafka] should be appeared in Kibana, but it did not.
In journalctl I did not see any errors and warning about connection to Kibana and Elastic.
Which settings are incorrect?
Thank you.