Parsing typical logback log lines

Elastic Stack Beats
1

Hi,

I am trying to send the following format log line from filebeat 6 to es6:
%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n

In my prospectors I specify:

  • type: log

Looking at fields.xml the default log fields do not contain fields described in the above log line, from forum posts from '16 it looks like input-type: log used to be able to parse the typical logback log lines.

In the latest filebeat though I see a lot of fields I do not need such as offset, beat.name, beat.version etc.

Can I modify fields.xml to extact the fields like thread, logger and level or is there a default module that let's me do that out of the box?

Thank you very much in advance!

2

Filebeat is not doing any parsing for you. Do you use logstash or Elasticsearch ingest node for parsing?

Some fields (like offset) can be removed using the drop_fields or include_fields processors.

The fields.yml file only specifies the mapping of fields (essentially defines schema of beats events), but does not configure the fields to be exported.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.