# Multile pattren in single Grok Filter

**URL:** https://discuss.elastic.co/t/multile-pattren-in-single-grok-filter/142168
**Category:** Logstash
**Created:** [July 30, 2018, 11:54am UTC](https://discuss.elastic.co/t/multile-pattren-in-single-grok-filter/142168 "2018-07-30T11:54:29Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)
#### Post date: [July 30, 2018, 11:54am UTC](https://discuss.elastic.co/t/multile-pattren-in-single-grok-filter/142168/1 "2018-07-30T11:54:29Z")

</div>

Hello All,

I have written multiple pattern in single grok filter, but my logs are parse only first filter. Below are the filter in my Grok

```auto
 match => { "message" => ["%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}",
                          "\I\,\s\[(?<date-time>[\d\-\w\:\.]+)\s\#(?<pid>[\d]+)\]\s\s(?<loglevel>[\w]+)\s\--\s\:\s\[(?<request-id>[\d\-\w]+)\]\s(?<path>[\w\s\"\/\.]+)\s(?<message>.*)",
                          "\I\,\s\[(?<date-time>[\d\-\w\:\.]+)\s\#(?<pid>[\d]+)\]\s\s(?<loglevel>[\w]+)\s\--\s\:\s\[(?<request-id>[\d\-\w]+)\]\s(?<message>.*)",
                          "\I\,\s\[(?<date-time>[\d\-\w\:\.]+)\s\#(?<pid>[\d]+)\]\s\s(?<loglevel>[\w]+)\s\--\s\:\s\[(?<request-id>[\d\-\w]+)\]\s(?<device-id>[\w\s\:]+)" ] }

```

Any suggestion will be appreciated.

Thanks in advance.

---

<div class="post-metadata">

### Author: ![Shaoranlaos](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@Shaoranlaos](https://discuss.elastic.co/u/Shaoranlaos)
#### Post date: [July 30, 2018, 12:35pm UTC](https://discuss.elastic.co/t/multile-pattren-in-single-grok-filter/142168/2 "2018-07-30T12:35:28Z")

</div>

could you post some sample lines?

Not sure if i read it correctly but are these pattern for different log lines or each for the same and they extract different fields?

if these filters should all be executed for the same log line you will need to disable the break\_on\_match option of the grok filter:  
[https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-grok.html#plugins-filters-grok-break\_on\_match](https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-grok.html#plugins-filters-grok-break_on_match)

---

<div class="post-metadata">

### Author: ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)
#### Post date: [July 31, 2018, 3:49am UTC](https://discuss.elastic.co/t/multile-pattren-in-single-grok-filter/142168/3 "2018-07-31T03:49:54Z")

</div>

Hello Shaoranlaos,

Above patterns are for different log lines. But some filelds are common in all logs. So my patterns are look generic. Please find the some sample logs:

```auto
I, [2018-07-23T08:49:09.854771 #22418] INFO -- : [81ee6e72-e1b3-4480-b664-0e215cdf23a8] Started POST "/api/v1/locations.json" for 127.0.0.1 at 2018-07-23 08:49:09 +0000
I, [2018-07-23T08:49:09.856826 #22418] INFO -- : [81ee6e72-e1b3-4480-b664-0e215cdf23a8] Processing by Api::V1::LocationsController#create as JSON
I, [2018-07-23T08:49:09.860378 #22418] INFO -- : [81ee6e72-e1b3-4480-b664-0e215cdf23a8] Current device: 1380 : 001
I, [2018-07-23T08:49:09.861850 #22418] INFO -- : [81ee6e72-e1b3-4480-b664-0e215cdf23a8] Device :: 1380 :: 5

```

Please help to fix this issue or let me know if i need to made any changes in my patterns.

Thanks.

---

<div class="post-metadata">

### Author: ![Shaoranlaos](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@Shaoranlaos](https://discuss.elastic.co/u/Shaoranlaos)
#### Post date: [July 31, 2018, 5:07am UTC](https://discuss.elastic.co/t/multile-pattren-in-single-grok-filter/142168/4 "2018-07-31T05:07:02Z")

</div>

in my tries it is the second pattern that will always match and not the first (because your log lines are not syslogs)

anyway could you try to change the order of the patterns and be more specific in the matching because all 3 pattern match the provided lines and extract because of this for the specific fields nonsense  
it needs some identifing word or character to correctly differentiate between the patterns

for this you also could try to only use the general pattern (where only message is extracted, second pattern in your starting post) and then make the specific field extraction in an extra grok filter on the message field(would probaly reduce some overhead and maintenance cost).

e.g. (like i think it will make sense)

```auto
 match => { "message" => ["%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}",
                          "\I\,\s\[(?<date-time>[\d\-\w\:\.]+)\s\#(?<pid>[\d]+)\]\s\s(?<loglevel>[\w]+)\s\--\s\:\s\[(?<request-id>[\d\-\w]+)\]\sStarted\s%{WORD:verb}\s\"(?<path>[\w\s\"\/\.]+)\"\s(?<message>.*)",
                          "\I\,\s\[(?<date-time>[\d\-\w\:\.]+)\s\#(?<pid>[\d]+)\]\s\s(?<loglevel>[\w]+)\s\--\s\:\s\[(?<request-id>[\d\-\w]+)\]\s(?:Current\s)?[dD]evice[\s:]+(?<device-id>[\w\s\:]+)",
                          "\I\,\s\[(?<date-time>[\d\-\w\:\.]+)\s\#(?<pid>[\d]+)\]\s\s(?<loglevel>[\w]+)\s\--\s\:\s\[(?<request-id>[\d\-\w]+)\]\s(?<message>.*)" ] }

```

this is only a quick shoot from my side i will see if i find the time today to take a closer look

---

<div class="post-metadata">

### Author: ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)
#### Post date: [July 31, 2018, 5:30am UTC](https://discuss.elastic.co/t/multile-pattren-in-single-grok-filter/142168/5 "2018-07-31T05:30:36Z")

</div>

Hello Shaoranlaos,

Thank you for your help. You are right that was my second pattern which match my logs. I wil also look into your approach and try to be more specific.

Thanks once again.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 28, 2018, 5:30am UTC](https://discuss.elastic.co/t/multile-pattren-in-single-grok-filter/142168/6 "2018-08-28T05:30:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
