Multiple split filters are used in the logstash for processing xml file of size 8MB is taking more than 30 minutes

Elastic Stack Logstash
1

JVM size : 4GB, 4 core CPU

The xml file size is 8MB and in the beat.conf of logstash we used 4-5 split, xml filters. It is taking more than 30 minutes.

Please can u help on this or suggest a way whether any configuration tunning can help this (ex: pipeline)

2

You will have a better chance of someone being able to help you if you show your config and what your data look like.

3

As you suggested in the above example for 2 child elements , this is extension of this as i had more child elements, so i have kept 5 filters and again comparing them based on common element/field.

4

@Badger.. Hope you remebered this

5

How many documents will that file result in after all the splits?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.