Hello,
We are using Logstash5.5 send event logs to Kafka newer version(2.4). We are seeing some data discrepancy. Issue seems to be resolving if we restart logstash service, however it is reoccurring nearly in 48 hours every time and we ended up restarting as we dont have root cause of this issue.
We have captured node api stats and created dashboards in grafana.
We couldn't see any metrics which is alarming.
- wait for worker % < 1% ==> ((queue_push_duration_in_millis/duration_in_millis)*100)
- Queue(persistent)% used < 3%
- Heap % used < 30%
Also, enabled DEBUG logs for logstash and couldn't find any specific errors or failures. Kafka cluster is healthy and all other pipelines which is not using logstash5 is good.
We understand logstash5.5 used Kafka client version 0.10, our understanding it is compatible with higher version of kafka also.
- Does it cause of any data discripency?
- Any performance issue or compatibility issue for using logstash5.5 --> Kafka 2.4?
- Our legacy pipeline was using logstash1.5 --> Kafka 0.8, which is still working fine.
Appreciate any inputs on this issue.