Logstash multiline issue - limit of size in single event

Elastic Stack Logstash
1

Logstash file input configured with multiline pattern as

    codec => multiline {
        pattern => '^  {'
        negate => true
        what => previous
    }
  }

Input file consist of multiple json objects. This multiline pattern w=works as expected for most of events, but strange it has created two event/message for single json object. Where logstash created first message starting with expected pattern of two white spaces and '{' but the second line is created at point where there are four spaces at start of line like ^ {, which is not matching logstash pattern ^ { having two spaces at start of line.
Ideally the second line created by logstash should have been appended to first line to form complete json object

2

I suspect if there is limit of lines or characters for multiline filter or logstash input filter, but I don't find any warning in log messages.

The size of JSON object which was divided into two lines instead of one is 4.6K & 94 lines

& the event/message created by logstash is merged with next JSON object creating message on elasticsearch of 500 lines and 21K size

I guess this gives hint to the size limit?

3

The max_lines and max_bytes options for the multiline codec are documented.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.