Filebeat multiline with line breaks


(Amrutha P Bhat) #1

I have a file in the below format
Ex:

This is the first line.

This is the second line.
I am using no pattern since I want all the lines in the text to send to Logstash which is around 2000 lines.

If the file is in the below format it works.
Ex:

This is the first line.
This is the second line.
If the file is in the above format multiline works.

Could you please help me out with this.


(Carlos Pérez Aradros) #2

Hi @amruthapbhat,

Could you please share your filebeat.yml settings? It should help diagnosing your issue


(Amrutha P Bhat) #3

filebeat:
prospectors:

-
  paths:
    - /home/ubuntu/containers.d/*/*.log

  input_type: log

  document_type: syslog

  multiline:
    match: after
    max_lines: 2000

(Amrutha P Bhat) #4

Hi @exekias,

Please find the above prospectors


(Carlos Pérez Aradros) #5

uhm, now I see this I'm wondering, what are you looking for?

Are you trying to send all lines in the same event? You don't need to set a multiline pattern to send all file lines, they will be sent one by one.

Could you please clarify what do you want to achieve?


(Amrutha P Bhat) #6

Hi @exekias,

I have a log file which could have around 1000 lines of content along with line breaks. i want to display the entire file as one event


(Amrutha P Bhat) #7

Hi @exekias,
The log files works correctly if there are no line breaks with the above configs. It displays the entire 1000 lines of content as a single log. The issue is when i have line breaks


(Carlos Pérez Aradros) #8

I think you need to set a multiline pattern like this:

multiline.pattern: '.'


Logs with lime breaks not working with multiline
(Amrutha P Bhat) #9

Hi @exekias,

i have given a link below where i have placed a sample log.

Link to the Dockerlog:

Please let me know if there is any pattern to combine all the lines into one event


(Steffen Siering) #10

Have you tried the multiline tester link from our docs? e.g. setting the regex pattern to ^.|^$ does help: https://play.golang.org/p/fsNzzM8bHA


(system) #11

This topic was automatically closed after 21 days. New replies are no longer allowed.