Filebeat published event but ElasticSearch doesn't show all those

I have multi line file beat (5.2) that sends entire xml as 1 event, this works as i can see the number of events published from filebeat are same as the number of files in the file beat path.

I am pushing these events to logstash using xml plugin at the logstash, which consumes file beat events, and stores them in elastic db after some data crunching

At this point i have following questions

Doc count in the index doesn't match event count from file beat - Where are my events being dropped ? and what can i do to prevent that ? The number of documents in index for same number of files being sent from file beat is kind a random, but its almost always not same as total events being sent.

Any pointers

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.