Handle Large Messages in Logstash

Elastic Stack Logstash
1

Hello. i am using multiline patterns in my logstash input, it works well. sometimes i have very large logs say 20MB, logstash is crashing when this logs comes in. is there a way to handle it?
i am also using grok patterns on the message.

Thanks
Amith

2

Why do you have such large messages? What does it contain?

Even if you manage to pass this through Logstash, what effect will it have on Elasticsearch when you try to retrieve this through Kibana?

3

For now, i am dropping logs greater than 2 mb when sending logs by using max_request_size option. is there a way for preventing the logs to go through the grok pattern filter based on the size ?

Thanks

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.