Filebeat multiline ignores line if the newline doesn't exist

Elastic Stack Beats
1

Using filebeat 1.1.2, i have a log event that have 3 lines and starts with the date, but the last line have no newline character. After 5 seconds, filebeat sends the 2 first lines of the event... after 10 minutes i get the same event and filebeat sends the last line of the previous event, then detects the date and start a new multiline. example:

2016-04-01 11:11:11 blabla\n _ sent after 5 s
bla bla bla\n /
bla bla no newline - sent after 10 minutes
(repeat the above lines each 10 minutes)

So looks like that multiline check will not look to the line is it doesn't end with a newline.
Only after a new event is trigger it will see it (probably the "date" is really "\ndate", but is not easy to check)

Of course the log is broke, but i can not change the app... so filebeat needs one option to "ignore missing newline on timeout" so it can catch the last line without a newline on it. Or is there any workaround for this?

Thanks in advance for the help

Daniel

2

processing in filebeat is more like:

line reader -> multiline -> publish. The multiline handler by default has some timeout configured, to print the currently buffered event if no full line has been received for some time.

Look for timeout option in multiline settings. Default is 5s. You can set this value to some very large value or set to '-1s' to disable multiline timeout.
Drawback: event will be only generated if events newline is finally written to file. This is a common problem with loggers writing the newline character only if new log is to be appended. Problem with 'partial' lines is: it clashes with loggers buffering output without flushing-timer.

3

Thanks for the reply

yes, i understand, but i keeping pushing the timeout and if there is no new event, the last even will be hidden for a unknown time

i would suggest adding a optional preference for line_reader_timeout (disable by default), so after that time the line reader would push to the multiline the partial line it would already have. This would not be used for most people, but would allow one to fix this kind of problems on logs with small number of events

Another alternative would be adding a option flush_on_text, where when some defined text/regexp would force the line reader to flush even lines without newline. Or even replace the newline with other text as line ending flag, again with a new option line_ending

this seems like the only way to fix this, give the admin some control to line reader, either by timeout or by matching characters or newline replacement.

fixing the log format would be better, but not everyone can do it

1 Like
4

What's you application based off? I remember having seen similar issues with PHP based logging framework.

I agree, the reader should be somehow configurable to adapt. Sometimes we can not change/adapt the logger.

Feature you're asking for is partial line support. I'd recommend to add a ticket with partial line support based on timeouts.

Not sure about regex based matching. You've got a good use-case/sample?