# How can Filebeat specify match rules to Logstash

**URL:** <https://discuss.elastic.co/t/how-can-filebeat-specify-match-rules-to-logstash/176818>\
**Category:** Logstash\
**Created:** [April 15, 2019, 3:39am UTC](https://discuss.elastic.co/t/how-can-filebeat-specify-match-rules-to-logstash/176818 "2019-04-15T03:39:37Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gary.Pan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gary.pan/32/44295_2.png) [@Gary.Pan](https://discuss.elastic.co/u/Gary.Pan)\
**Post date:** [April 15, 2019, 3:39am UTC](https://discuss.elastic.co/t/how-can-filebeat-specify-match-rules-to-logstash/176818/1 "2019-04-15T03:39:37Z")

</div>

I want to let Logstash'gork filter use the match rules which Filebeat give

Here is my Filebeat config:

```
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /root/Log-test/test.log
  fields:
    "@metadata":
        formatter: "%{TIMESTAMP_ISO8601:timestamp} - %{NOTSPACE:module} - %{LOGLEVEL:level} - %{NOTSPACE:filename} - %{GREEDYDATA:log_message}"
  fields_under_root: true

output.logstash:
  hosts: ["localhost:5045"]

```

Here is my Logstash config:

```
input {
    beats {
        port => "5045"
    }
}

filter {
  grok {
    match => { "message" => "%{[@metadata][formatter]}" }
  }
}

output {
  file {
    path => "/tmp/log-test.log"
    codec => rubydebug { metadata => true }
  }
}

```

So, i want the grok know my match rules content (the `message` field) is `"%{TIMESTAMP_ISO8601:timestamp} - %{NOTSPACE:module} - %{LOGLEVEL:level} - %{NOTSPACE:filename} - %{GREEDYDATA:log_message}"`

But the setting above do not work, I want to know how can i implement the funciton like this? or is it possible to make it?

Thanks!

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [April 15, 2019, 7:36am UTC](https://discuss.elastic.co/t/how-can-filebeat-specify-match-rules-to-logstash/176818/2 "2019-04-15T07:36:57Z")

</div>

Hi @Gary.Pan,

I have not looked at Filebeat 7 very closely yet so I might be wrong but as far as I know, you can't really pass @metadata fields to Logstash that way... There are some @metadata fields from Filebeat that Logstash does get but they (at least used to be) are more or less predefined.

I also see no mention of @metadata fields in the Filebeat [fields](https://www.elastic.co/guide/en/beats/filebeat/7.0/add-fields.html) documentation.

I think I see what you are trying to accomplice. An interesting approach. Try adding just `formater` as a _field_ and try

```auto
match => { "message" => [formatter] }

```

No idea if that will work or not though...

---

<div class="post-metadata">

**Author:** ![Gary.Pan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gary.pan/32/44295_2.png) [@Gary.Pan](https://discuss.elastic.co/u/Gary.Pan)\
**Post date:** [April 15, 2019, 8:22am UTC](https://discuss.elastic.co/t/how-can-filebeat-specify-match-rules-to-logstash/176818/3 "2019-04-15T08:22:08Z")

</div>

I have printed the data which Logstash get from Filebeat (you can see my Logstash output codec is rubydebug with metadata) and the data contain the `formatter` field, what is more i have use this way to pass my target index from Filebeat to Logstash. The problem of this case is not on the data transportation, is on how to express and assign the data to the `message` field, and i am sure the way you written `match => { "message" => [formatter] }` will raise an error, because i have tried:grinning:

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 15, 2019, 11:36am UTC](https://discuss.elastic.co/t/how-can-filebeat-specify-match-rules-to-logstash/176818/4 "2019-04-15T11:36:46Z")

</div>

I do not believe you can use a sprintf reference to a field in a grok filter. It interprets it as a pattern name reference, not a field name. So

```
    mutate { add_field => { "filename" => "foo.bar" "format" => "%{WORD:first}\.%{WORD:second}" } }
    grok { match => { "filename" => "%{format}" } }

```

just gets you an undefined pattern error for "format".

---

<div class="post-metadata">

**Author:** ![Gary.Pan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gary.pan/32/44295_2.png) [@Gary.Pan](https://discuss.elastic.co/u/Gary.Pan)\
**Post date:** [April 16, 2019, 2:03am UTC](https://discuss.elastic.co/t/how-can-filebeat-specify-match-rules-to-logstash/176818/5 "2019-04-16T02:03:40Z")

</div>

I use that way in Filebeat not in Logstash. Have you tried it in Filebeat?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 16, 2019, 12:20pm UTC](https://discuss.elastic.co/t/how-can-filebeat-specify-match-rules-to-logstash/176818/6 "2019-04-16T12:20:58Z")

</div>

> [@Gary.Pan](#):
>
> Have you tried it in Filebeat?

No, I have not.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 14, 2019, 12:21pm UTC](https://discuss.elastic.co/t/how-can-filebeat-specify-match-rules-to-logstash/176818/7 "2019-05-14T12:21:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.