Multiline Config from LS1.5.4 not working in LS2.0.0

(Sukrit Dasgupta) #1

Hi LS community,

I am trying to migrate my LS setup from 1.5.4 (which doesnt seem to be happy with ES2.0.0) over to LS 2.0.0. Everything is the same except the change from 'host' --> 'hosts'.

However, I am not able to start LS because of this:

Error: Cannot use more than 1 filter worker because the following plugins don't work with more than one worker: multiline, multiline

The same config works fine for LS1.5.4. I have several conditional 'multilines' in the filter section.


(Mark Walkom) #2

Using multiple workers with multiline can be dangerous as you can end up splitting messages.

It does mention it here and also
Are you sure this worked in 1.5?

(Sukrit Dasgupta) #3

Thanks for replying @warkolm

I was not using multiple workers in 1.5.4. I did try passing the parameter, but it had refused it. So there were no multiple workers for multiline in 1.5.4 and that was working with multiple multilines.

In 2.0.0, I am not passing any multiple worker flag either. Maybe there is a new default in 2.0 that gets passed for multiple workers?


(Jeffrey Ollie) #4

FYI, I found this:

(Sukrit Dasgupta) #5

Thanks @jcollie.

So far, I am not able to see whether I will have an issue with multiline and LS2.0. Thats because I cant get LS2.0 to start up. :smile:
The issue is, (i think) LS2.0 is trying to startup with multiple

  • workers

(by default i guess), but because I have multilines, startup is failing. This behavior is known of course as mentioned by @warkolm.

The questions i guess are:

  1. Does LS2.0 startup with multiple workers by default?
  2. If not #1, then why does LS2.0 think I am trying to startup with multiple workers? (because I am not)


(Jeffrey Ollie) #6

Despite what the documentation claims, I'm pretty sure that Logstash 2.x starts up with multiple filter workers. Try adding "-w 1" to your command line and see if that helps.

(Aaron Mildenstein) #7

Logstash 2.0 does, indeed, start with what is effectively -w #_of_cores/2

We've found that this causes issues with some plugins (multiline filter, specifically), so this is being addressed for the next release.

Feel free to launch with -w 1 in the meanwhile.

(Sukrit Dasgupta) #8

Thanks @jcollie and @theuntergeek.

Common sense is the most uncommon. I should have just tried with -w 1.

This worked indeed!


(system) #9