# I want to match two grok patterns in one config file?

**URL:** <https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878>\
**Category:** Logstash\
**Created:** [June 1, 2017, 10:04am UTC](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878 "2017-06-01T10:04:56Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yaswanth](https://avatars.discourse-cdn.com/v4/letter/y/94ad74/32.png) [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\
**Post date:** [June 1, 2017, 10:04am UTC](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878/1 "2017-06-01T10:04:56Z")

</div>

Hi,

I have two types of error messages in the below format:

```
[2017-05-25 01:00:00,647][ERROR][marvel.agent.exporter.local] local exporter [default_local] - failed to delete indices
RemoteTransportException[[data-0][10.0.0.8:9300][indices:admin/delete]]; nested: IndexNotFoundException[no such index];
...
[2017-05-18 00:00:06,339][DEBUG][action.admin.indices.create] [data-2] [data-may-2017,data-apr-2017,data-mar-2017] failed to create
[data-may-2017,data-apr-2017,data-mar-2017] InvalidIndexNameException[Invalid index name ..

```

My logstash configuration is like this:

```
input {
      file {
            path => "D:\logstash\logstash-2.4.0\bin\logs.txt"
            start_position => "beginning"
	    codec => multiline {
            pattern => "^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]"
            negate => true
            what => "previous"
        }
  }

}
filter {
   grok {
        match => ["message", "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}\]%{SPACE}\[%{DATA:SERVERNAME}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)", "message", "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}%{SPACE}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)"]
   }
mutate {
    remove_field => ["message","@version","path","host"]
  }

}
output {
 
  stdout { codec => rubydebug }
}

```

For Both the logs it is taking only the first grok pattern. Why it is not taking the second one?

Thanks

---

<div class="post-metadata">

**Author:** ![Jaxon\_Kochel](https://avatars.discourse-cdn.com/v4/letter/j/da6949/32.png) [@Jaxon\_Kochel](https://discuss.elastic.co/u/Jaxon_Kochel)\
**Post date:** [June 1, 2017, 1:38pm UTC](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878/2 "2017-06-01T13:38:29Z")

</div>

Im not entirely sure, but when I do multiple patters i do:

grok{

match =\> [ "message" ... # first pattern in one match  
match = \> [ "message" ... #second pattern in another

}

Also I didn't look to closely, but if both entries match the same pattern then they'll always go to that one pattern and not the other.

---

<div class="post-metadata">

**Author:** ![Yaswanth](https://avatars.discourse-cdn.com/v4/letter/y/94ad74/32.png) [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\
**Post date:** [June 1, 2017, 2:04pm UTC](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878/3 "2017-06-01T14:04:54Z")

</div>

Thanks @Jaxon_Kochel

I think what you are saying is right . But how can i overcome this?

Whether i have to give like this

```
Filter{
grok{
match=>"msg"
}
if [LEVEL] == "Error"
grok {
match = > "msg"
}
else {
drop{}
}

```

Thanks

---

<div class="post-metadata">

**Author:** ![Jaxon\_Kochel](https://avatars.discourse-cdn.com/v4/letter/j/da6949/32.png) [@Jaxon\_Kochel](https://discuss.elastic.co/u/Jaxon_Kochel)\
**Post date:** [June 1, 2017, 2:48pm UTC](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878/4 "2017-06-01T14:48:08Z")

</div>

Use this pattern for when you have a servername:

(?m)^[%{TIMESTAMP\_ISO8601:TIMESTAMP}][ERROR][%{DATA:ERRORTYPE}]%{SPACE}[%{URIHOST:SERVERNAME}]%{SPACE}(?(.|\r|\n)\*)

Ichanged it form being DATA to URIHOST

This prevents a non servername log form being matched and will solve the issue

---

<div class="post-metadata">

**Author:** ![Yaswanth](https://avatars.discourse-cdn.com/v4/letter/y/94ad74/32.png) [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\
**Post date:** [June 1, 2017, 4:09pm UTC](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878/5 "2017-06-01T16:09:13Z")

</div>

Thanks @Jaxon_Kochel

I used your suggestion by giving grok like this:

```
filter {
   grok {
        match => ["message", "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}\]%{SPACE}\[%{URIHOST:SERVERNAME}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)", "message", "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}%{SPACE}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)"]
   }
mutate {
    remove_field => ["message","@version","path","host"]
  }

} 

```

But still i am getting the same response.

But i used the these two grok patterns in [Grok constructor site](http://grokconstructor.appspot.com/do/match#result) it is working fine(i.e.First grok only matching second log and second grok only matching first log) .

Thanks

---

<div class="post-metadata">

**Author:** ![Yaswanth](https://avatars.discourse-cdn.com/v4/letter/y/94ad74/32.png) [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\
**Post date:** [June 4, 2017, 9:07am UTC](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878/6 "2017-06-04T09:07:59Z")

</div>

Finally came with a filter that satisfies my requirement.

```
input {
      file {
            path => "D:\logstash\logstash-2.4.0\bin\logs.txt"
            start_position => "beginning"
             type => "log"
        codec => multiline {
            pattern => "^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]"
            negate => true
            what => "previous"
        }
  }

}
    filter {
      if [type] == "log" {
        grok {
          match => ["message", "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}%{SPACE}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)"]
        }
    # DEBUG Logs
    if "grokked" not in [tags] and "DEBUG" == [LEVEL] {
    grok { match => ["ERRORMESSAGE", "(?m)^\[%{DATA:SERVERNAME}\]" ]
    add_tag => ["Debug Logs", "grokked"]
    tag_on_failure => []
    }
    }
    }
    }
output {

  stdout { codec => rubydebug }
}

```

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 2, 2017, 9:08am UTC](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878/7 "2017-07-02T09:08:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.