Filebeat for shipping the entire logfile as it is

Elastic Stack Beats
1

I need to get the entire log file as it is in the elk stack.
Consider I have this log: <line1> <line2> <line3>|
If my pointer is placed on the line 3 i will get the entire log file except <line3>
. If the cursor is in new line then i will be able to get the file as expected.

This is my filebeat: multiline.pattern: '.*' multiline.negate: false multiline.match: after

Kindly help me out with the settings.attach1

2

What is written in line 3? One needs a regular expression to detect multiline patterns. The pattern .* just captures everything.

3

I actually want the entire data from the log file. There are different type of logs which iam trying to import so in my case there is no fixed pattern as such.

4

Iam waiting for it. Please respond.

5

Try using multiline.pattern: '.' to capture the entire log file as a single event.

6

This pattern sends each line as a separate event.

7

Can you attach a sample log file with a few lines?

8

line1
line1.png1283×139 16.1 KB

Iam able to get the entire file when the pointer is pointing in new line as shown above but if the pointer is placed at the end of <EndLine>| , then I will get all the data till <SecondLine>ABC skipping the end line.

9

Filebeat first splits the file into lines. The multiline processing combines them again. One can use multiline.timeout to trigger a flush of the multiline buffer, but EndLine might not be included in this case.

10

That's exactly my issue, kindly tell me if it is possible to get the full file including the <EndLine>. I'm stuck here for a long time, I have also attached my filebeat config file above in my question description for further details.

1 Like
11

Without the final new line control character it's currently not possible.
There is an open enhancement request for sending complete files: https://github.com/elastic/beats/issues/4982

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.