What is the file size limit of the logstash config file ? Actually I have many grok patterns to fill in it and it might big and cumbersome to handle. Please guide me how to overcome this. Will it slow down the pipeline performance of logstash? I am using ELK for centralized logging of logs.
Please Help
@magnusbaeck Can you help me on this one?
In recent versions you can split out processing of different data types into multiple pipelines. Once you have identified the data type you can send the data to the appropriate pipeline. This generally makes it easier to manage larger configs.
Lots of unnecessary grok processing can be very inefficient and slow down the pipeline, so I would also recommend reading this blog post about grok optimisation. If your data is delimited, it may be faster and more efficient to use the dissect filter where applicable.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.