Parsing typical logback log lines

Hi,

I am trying to send the following format log line from filebeat 6 to es6:
%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n

In my prospectors I specify:

  • type: log

Looking at fields.xml the default log fields do not contain fields described in the above log line, from forum posts from '16 it looks like input-type: log used to be able to parse the typical logback log lines.

In the latest filebeat though I see a lot of fields I do not need such as offset, beat.name, beat.version etc.

Can I modify fields.xml to extact the fields like thread, logger and level or is there a default module that let's me do that out of the box?

Thank you very much in advance!

Filebeat is not doing any parsing for you. Do you use logstash or Elasticsearch ingest node for parsing?

Some fields (like offset) can be removed using the drop_fields or include_fields processors.

The fields.yml file only specifies the mapping of fields (essentially defines schema of beats events), but does not configure the fields to be exported.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.