# How to use grok patterns coming from event data

**URL:** <https://discuss.elastic.co/t/how-to-use-grok-patterns-coming-from-event-data/115137>\
**Category:** Logstash\
**Created:** [January 11, 2018, 4:36pm UTC](https://discuss.elastic.co/t/how-to-use-grok-patterns-coming-from-event-data/115137 "2018-01-11T16:36:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![whites11](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/whites11/32/26479_2.png) [@whites11](https://discuss.elastic.co/u/whites11)\
**Post date:** [January 11, 2018, 4:36pm UTC](https://discuss.elastic.co/t/how-to-use-grok-patterns-coming-from-event-data/115137/1 "2018-01-11T16:36:55Z")

</div>

I have an ELK stack deployed on kubernetes used to collect containers' data. Among all the rest, it is using a grok filter to parse the actual log line based on a pattern.

My wish is to be able to setup this pattern by using an annotation in the kubernetes pod.

I added an annotation called `elk-grok-pattern` in the `pod`, configured `filebeat` in order to forward the annotation and I can get the annotation value as a field in my `event` in `logstash`, so far so good.

The problem is that I am unable to use the value of my field as a `grok pattern`.

The annotation in my pod looks like this:

```
Annotations: elk-grok-pattern=%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:status} %{NUMBER:response_time}

```

The `filter` I am trying to use is similar to the following:

```
filter {
  # create a new field called "elk-grok-pattern" from the pod annotation
  mutate {
        rename => { "[kubernetes][annotations][elk-grok-pattern]" => "elk-grok-pattern" }
  }

  grok {
    pattern_definitions => {
      "CUSTOM" => "%{elk-grok-pattern}"
    }
    match => { "log" => "%{CUSTOM}" }
  }
}

```

Unluckily this leads to an error:

```
Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<Grok::PatternError: pattern %{elk-grok-pattern} not defined>

```

In practice, grok is interpreting my pattern literally, and not evaluating the string content coming from the event.

I also tried using the pattern directly, withoud defining a pattern\_definition, like this:

```
grok {
  match => { "log" => "%{elk-grok-pattern}" }
}

```

But I get the same exact error.

Is there a way to accomplish my goal?  
Any advice or possible workaround would be very appreciated.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 8, 2018, 4:37pm UTC](https://discuss.elastic.co/t/how-to-use-grok-patterns-coming-from-event-data/115137/2 "2018-02-08T16:37:10Z")

</div>

Closed on February 8, 2018, 4:37pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.