Logs are not ingesting to elasticsearch

Elastic Stack Elasticsearch
1

This was my set up earlier.
filebeat => logstash (SQS-PUSH) => logstash (SQS-PULL) => elasticsearch.
The above approach seems very costlier to us because of (SQS API) calls. Hence we replaced kafka (standalone).
filebeat => kafka => logstash => elasticsearch

How ever with this new approach i am not sure what was causing the issue logs are not getting ingested, i can see messages in kafka and logstash outout but they are not getting ingested to elasticsearch. Not seeing any errors. Any suggestions where i have to take a look.?

I am using 100 partitions for the topic and below is my config.

input {
  kafka {
    bootstrap_servers => "IP:9092"
    topics => ["kafka"]
    codec  => json
    consumer_threads => 100
}
}
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.