Shipping whole file as message?

Elastic Stack Beats
1

Hello everyone,

I am kindly new to ELK and now I want to ship the content of a whole file as message into logstash and elastic.
So as I have different files, each file should be a single event with all its content as message. I know that I have to use multiline in the filebeat prospector. But I don't know how to set multiline.pattern, mutliline.negate, multiline.match. The message should be in the correct order, meaning first line of the file should be also first line in the message. Can somebody please give me a hint?

Thanks in advance!

1 Like
2

Try using the file input plugin in logstash file input plugin

3

You can do it with filebeat and multiline if you can easily identify the last line of you file.

Try to configure the multiline pattern to detect the last line.

You can test the pattern and negate behavior with the Go Playground described here: https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html#_testing_your_regexp_pattern_for_multiline

4

I guess I have a solution.
I don't have to detect the last line. Seems for me, that filebeat detects it itself. Only have to be sure to start at first line and put everything on it.
filebeat.yml

filebeat.prospectors: 
- input_type: log
      paths:
        - /X/Y/*/*
      multiline.pattern: ' \A.*'
      multiline.negate: true
      multiline.match: after
      tags: ["my-tag"]
1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.