So I'm using the multiline codec, and it seems to be working great. However, I'm doing filtering on the FileBeats side so that I'm only sending the multiline values that I expect to parse on the other end.
(I can't use the multiline parsing on FileBeat, because the include_filter is done after the multiline is processed, which is too bad, or I'd do all my processing on the client side.)
What I'm running into, is that the final set of events isn't processed (flushed? I'm new to Logstash terminology), as no additional event is sent over.
Is there a way to tell Logstash via Beats that all events have been sent, and to flush the buffer? Or is the proper way to solve this via the auto_flush_interval setting in the multiline codec configuration?
Edit: Seems the answer is, as I somewhat figured, was to add an auto_flush option: https://github.com/elastic/logstash/issues/4567