How can Filebeat send match rules to Logstash

I want to let Logstash'gork filter use the match rules which Filebeat give

Here is my Filebeat config:

- type: log
  enabled: true
    - /root/Log-test/test.log
        formatter: "%{TIMESTAMP_ISO8601:timestamp} - %{NOTSPACE:module} - %{LOGLEVEL:level} - %{NOTSPACE:filename} - %{GREEDYDATA:log_message}"
  fields_under_root: true

  hosts: ["localhost:5045"]

Here is my Logstash config:

input {
    beats {
        port => "5045"

filter {
  grok {
    match => { "message" => "%{[@metadata][formatter]}" }

output {
  file {
    path => "/tmp/log-test.log"
    codec => rubydebug { metadata => true }

So, i want the grok know my match rules content (the message field) is "%{TIMESTAMP_ISO8601:timestamp} - %{NOTSPACE:module} - %{LOGLEVEL:level} - %{NOTSPACE:filename} - %{GREEDYDATA:log_message}"

But the setting above do not work, I want to know how can i implement the funciton like this? or is it possible to make it?


Hi @Gary.Pan and welcome :slight_smile:

This is not possible, because grok patterns are compiled on start time, so this field is not going to be read when events are being processed.

You should be able to parse this data by using an ingest node pipeline:

Get it, thanks!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.