# Logstash - Multiple grok pattern not working together

**URL:** <https://discuss.elastic.co/t/logstash-multiple-grok-pattern-not-working-together/181825>\
**Category:** Logstash\
**Created:** [May 20, 2019, 1:23pm UTC](https://discuss.elastic.co/t/logstash-multiple-grok-pattern-not-working-together/181825 "2019-05-20T13:23:02Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![achatterjee104](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/achatterjee104/32/46473_2.png) [@achatterjee104](https://discuss.elastic.co/u/achatterjee104)\
**Post date:** [May 20, 2019, 1:23pm UTC](https://discuss.elastic.co/t/logstash-multiple-grok-pattern-not-working-together/181825/1 "2019-05-20T13:23:02Z")

</div>

I am very new in using Logstash. I have two kinds of log,

Pattern 1 : --2019-05-09 08:53:45.057 -INFO 11736 --- [ntainer#1-0-C-1] c.s.s.service.MessageLogServiceImpl : [adc7fd862db5307a688817198046b284dbb12b9347bed9067320caa49d8efa381557392024151] Event =\> Message Status Change [Start Time : 09052019 08:53:44] : CUSTOM\_PROCESSING\_COMPLETED

Pattern 2 : --2019-05-09 06:49:05.590 -TRACE 6293 --- [ntainer#0-0-C-1] c.s.s.service.MessageLogServiceImpl : [41a6811cbc1c66eda0e942712a12a003d6bf4654b3edb6d24bf159b592afc64f1557384545548] Event =\> Message Failure Identified : INVALID\_STRUCTURE

Though there are many more other lines, but I want to consider only these two types. Hence I used below filter,

```
 grok {
     #Event : message status change
     match => {
         "message" => "--(?<logtime>[^\]]*) -%{LOGLEVEL:level} (?<pid>\d+) --- \[(?<thread>[^\]]+)] (?<classname>[\w.]+)\s+: \[(?<token>[^\]]+)] Event \=> Message Status Change \[Start Time : (?<start>[^\]]*)\] : (?<status>[\w]+)"
     }
     add_field => {
         "event" => "message_status_change"
     }
 }

 grok {
     #Event : message failure
     match => {
         "message" => "--(?<logtime>[^\]]*) -%{LOGLEVEL:level} (?<pid>\d+) --- \[(?<thread>[^\]]+)] (?<classname>[\w.]+)\s+: \[(?<token>[^\]]+)] Event \=> Message Failure Identified : (?<code>[\w]+)"
     }
     add_field => {
         "event" => "message_failure"
     }
 }

```

I have also noticed that both of these grok patterns work individually (if I comment one, then other one works perfectly). Logstash server also ok when both patterns are active. But it raises a grokparse error when both of them is open and a new line is added in the log file.

Also I want to know, though I am configured the input to read from a file from beginning, it is not reading even after server restart unless I add a new line in the log. Why this behaviour?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 20, 2019, 4:39pm UTC](https://discuss.elastic.co/t/logstash-multiple-grok-pattern-not-working-together/181825/2 "2019-05-20T16:39:05Z")

</div>

> [@achatterjee104](#):
>
> I have also noticed that both of these grok patterns work individually (if I comment one, then other one works perfectly). Logstash server also ok when both patterns are active. But it raises a grokparse error when both of them is open and a new line is added in the log file.

That is to be expected. If a line matches one of those two grok filters it will not match the other one, so it will get tagged with \_grokparsefailure.

> [@](#):
>
> Also I want to know, though I am configured the input to read from a file from beginning, it is not reading even after server restart unless I add a new line in the log. Why this behaviour?

If you have a persistent sincedb then start\_position is ignored once a sincedb entry exists. It is only used the first time a file is seen. You can avoid persisting the sincedb across restarts using 'sincedb\_path =\> "NUL"' on Windows, or 'sincedb\_path =\> "/dev/null" on UNIX.

---

<div class="post-metadata">

**Author:** ![achatterjee104](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/achatterjee104/32/46473_2.png) [@achatterjee104](https://discuss.elastic.co/u/achatterjee104)\
**Post date:** [May 21, 2019, 5:42am UTC](https://discuss.elastic.co/t/logstash-multiple-grok-pattern-not-working-together/181825/3 "2019-05-21T05:42:57Z")

</div>

Thanks for reply. I understand what you said. But whats the solution here? What if I want two parse two different lines? Cant I use two groks together?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 21, 2019, 12:42pm UTC](https://discuss.elastic.co/t/logstash-multiple-grok-pattern-not-working-together/181825/4 "2019-05-21T12:42:17Z")

</div>

Yes, you can use multiple groks. If you do not care that a particular grok failed you could add a mutate+remove\_tag filter to remove the \_grokparsefailure.

Alternatively, you could test the message contents before invoking the grok

```
if [message] =~ "Message Failure Identified" {
     grok { [...]

```

Alternatively, the [match](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#plugins-filters-grok-match) option can match a line against an array of patterns.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 18, 2019, 12:42pm UTC](https://discuss.elastic.co/t/logstash-multiple-grok-pattern-not-working-together/181825/5 "2019-06-18T12:42:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.