Multiline Config from LS1.5.4 not working in LS2.0.0

Elastic Stack Logstash
1

Hi LS community,

I am trying to migrate my LS setup from 1.5.4 (which doesnt seem to be happy with ES2.0.0) over to LS 2.0.0. Everything is the same except the change from 'host' --> 'hosts'.

However, I am not able to start LS because of this:

Error: Cannot use more than 1 filter worker because the following plugins don't work with more than one worker: multiline, multiline

The same config works fine for LS1.5.4. I have several conditional 'multilines' in the filter section.

Thoughts?

2

Using multiple workers with multiline can be dangerous as you can end up splitting messages.

It does mention it here https://www.elastic.co/guide/en/logstash/current/plugins-filters-multiline.html and also https://www.elastic.co/guide/en/logstash/1.5/plugins-filters-multiline.html.
Are you sure this worked in 1.5?

3

Thanks for replying @warkolm

I was not using multiple workers in 1.5.4. I did try passing the parameter, but it had refused it. So there were no multiple workers for multiline in 1.5.4 and that was working with multiple multilines.

In 2.0.0, I am not passing any multiple worker flag either. Maybe there is a new default in 2.0 that gets passed for multiple workers?

Thanks.

4

FYI, I found this:

5

Thanks @jcollie.

So far, I am not able to see whether I will have an issue with multiline and LS2.0. Thats because I cant get LS2.0 to start up. :smile:
The issue is, (i think) LS2.0 is trying to startup with multiple

  • workers

(by default i guess), but because I have multilines, startup is failing. This behavior is known of course as mentioned by @warkolm.

The questions i guess are:

  1. Does LS2.0 startup with multiple workers by default?
  2. If not #1, then why does LS2.0 think I am trying to startup with multiple workers? (because I am not)

Thanks!

6

Despite what the documentation claims, I'm pretty sure that Logstash 2.x starts up with multiple filter workers. Try adding "-w 1" to your command line and see if that helps.

7

Logstash 2.0 does, indeed, start with what is effectively -w #_of_cores/2

We've found that this causes issues with some plugins (multiline filter, specifically), so this is being addressed for the next release.

Feel free to launch with -w 1 in the meanwhile.

8

Thanks @jcollie and @theuntergeek.

Common sense is the most uncommon. I should have just tried with -w 1.

This worked indeed!

Thanks.