Hi LS community,
I am trying to migrate my LS setup from 1.5.4 (which doesnt seem to be happy with ES2.0.0) over to LS 2.0.0. Everything is the same except the change from 'host' --> 'hosts'.
However, I am not able to start LS because of this:
Error: Cannot use more than 1 filter worker because the following plugins don't work with more than one worker: multiline, multiline
The same config works fine for LS1.5.4. I have several conditional 'multilines' in the filter section.
Using multiple workers with multiline can be dangerous as you can end up splitting messages.
It does mention it here https://www.elastic.co/guide/en/logstash/current/plugins-filters-multiline.html and also https://www.elastic.co/guide/en/logstash/1.5/plugins-filters-multiline.html.
Are you sure this worked in 1.5?
Thanks for replying @warkolm
I was not using multiple workers in 1.5.4. I did try passing the parameter, but it had refused it. So there were no multiple workers for multiline in 1.5.4 and that was working with multiple multilines.
In 2.0.0, I am not passing any multiple worker flag either. Maybe there is a new default in 2.0 that gets passed for multiple workers?
So far, I am not able to see whether I will have an issue with multiline and LS2.0. Thats because I cant get LS2.0 to start up.
The issue is, (i think) LS2.0 is trying to startup with multiple
(by default i guess), but because I have multilines, startup is failing. This behavior is known of course as mentioned by @warkolm.
The questions i guess are:
- Does LS2.0 startup with multiple workers by default?
- If not #1, then why does LS2.0 think I am trying to startup with multiple workers? (because I am not)
Despite what the documentation claims, I'm pretty sure that Logstash 2.x starts up with multiple filter workers. Try adding "-w 1" to your command line and see if that helps.
Logstash 2.0 does, indeed, start with what is effectively
We've found that this causes issues with some plugins (multiline filter, specifically), so this is being addressed for the next release.
Feel free to launch with
-w 1 in the meanwhile.
Thanks @jcollie and @theuntergeek.
Common sense is the most uncommon. I should have just tried with -w 1.
This worked indeed!