How can Filebeat send match rules to Logstash

Elastic Stack Beats
1

I want to let Logstash'gork filter use the match rules which Filebeat give

Here is my Filebeat config:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /root/Log-test/test.log
  fields:
    "@metadata":
        formatter: "%{TIMESTAMP_ISO8601:timestamp} - %{NOTSPACE:module} - %{LOGLEVEL:level} - %{NOTSPACE:filename} - %{GREEDYDATA:log_message}"
  fields_under_root: true

output.logstash:
  hosts: ["localhost:5045"]

Here is my Logstash config:

input {
    beats {
        port => "5045"
    }
}

filter {
  grok {
    match => { "message" => "%{[@metadata][formatter]}" }
  }
}

output {
  file {
    path => "/tmp/log-test.log"
    codec => rubydebug { metadata => true }
  }
}

So, i want the grok know my match rules content (the message field) is "%{TIMESTAMP_ISO8601:timestamp} - %{NOTSPACE:module} - %{LOGLEVEL:level} - %{NOTSPACE:filename} - %{GREEDYDATA:log_message}"

But the setting above do not work, I want to know how can i implement the funciton like this? or is it possible to make it?

Thanks!

2

Hi @Gary.Pan and welcome :slight_smile:

This is not possible, because grok patterns are compiled on start time, so this field is not going to be read when events are being processed.

You should be able to parse this data by using an ingest node pipeline: https://www.elastic.co/guide/en/beats/filebeat/7.0/configuring-ingest-node.html

3

Get it, thanks!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.