Hi,
I have multi line file beat (5.2) that sends entire xml as 1 event, this works as i can see the number of events published from filebeat are same as the number of files in the file beat path.
I am pushing these events to logstash using xml plugin at the logstash, which consumes file beat events, and stores them in elastic db after some data crunching
At this point i have following questions
Doc count in the index doesn't match event count from file beat - Where are my events being dropped ? and what can i do to prevent that ? The number of documents in index for same number of files being sent from file beat is kind a random, but its almost always not same as total events being sent.
Any pointers