How to debug multiline in filebeat?

Elastic Stack Beats
1

Here is my filebeat.yml:

filebeat.inputs:

- type: log

  enabled: true

  paths:

    - /home/xyz/nohup.txt

  multiline.pattern: '^\[[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after

Here is the command I used to append the events to the log file:

echo [2019-01-02 1234567 >> nohup.txt
echo [2019-01-02 1234567 >> nohup.txt
echo c>> nohup.txt
echo [2019-01-02 1234567 >> nohup.txt

And eventually 'c' was recognized as a new event, so multiline was not working, why?

Thanks for your help.

BTW: How do I debug this?

2

I think your test was a false negative -- I just tried this configuration and it worked fine on those lines. I think what you're seeing is that when you append the lines with echo, Filebeat reads them as soon as they're added, which means by the time you run echo c, the previous line has already been ingested, and it has to start a new one. (I replicated this on my system by delaying slightly between the two echo commands.) If you instead use cat to append the multiline entries, or start Filebeat when the entry is already complete, then it seems to work fine.

3

Thanks Fae, you are right, it is working by using 'cat'.

BTW: I thought FileBeat won't send the event until it matches the next pattern, and also it has the timeout property to define how long time it will wait, is it right?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.