Filebeat not publishing multi-line event after timeout

[dev.net (John)]

Hi,

I have configured filebeat with a multi line pattern to watch log files that will be written to once and contain a single root XML element with no XML declaration. Sometimes the root XML element can span multiple lines and sometimes it can span a single line.

close_eof: true
ignore_older: 5m
clean_inactive: 10m
multiline.pattern: ^<error.*
multiline.negate: true
multiline.match: after
multiline.flush: </error>
multiline.timeout: 5s

If the XML element is all on a single line then the multiline flush pattern will not be present and I have configured a timeout to flush the event anyway. The config works when the xml file has multiple lines but doesn't work when there is only a single line.

I can see from the debug logs that the event is being flushed due to the timeout but nothing is actually published. Unfortunately the XML file doesn't end every line with CRLF.

Can I make this work for a file with a single XML element on a single line that has no CRLF?

Thanks

[jsoriano (Jaime Soriano)]

Hi @dev.net and welcome

I think this can be a case of this issue https://github.com/elastic/beats/issues/1324.
As a workaround, would there be any chance you can make your application to always write a new line after the XML?

[jsoriano (Jaime Soriano)]

Filebeat needs lines to be terminated by \n.