Following is a screenshot
This show the logs being lagged at Kafka (the LAG column) when I query at kibana. After kafka, there is a logstash which applies gork filter and sends the data to elasticsearch cluster.
When I stop data from source and wait for few minutes, the LAG gets to zero. If I continue the logs from source, it is shown at kibana but with delay as the logs are queued at kafka. The LAG increases with time (even if I stop query at kibana).
Moreover, one thing I noted is that, LAG is zero when I start logs all over again and apply no query at kibana. And once I apply my first query at kibana, the LAG increases.
How can I reduce this lag?