# Grok filter with multiple regex not working

**URL:** <https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031>\
**Category:** Logstash\
**Created:** [May 8, 2018, 3:12pm UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031 "2018-05-08T15:12:47Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dazith\_Kj](https://avatars.discourse-cdn.com/v4/letter/d/b77776/32.png) [@Dazith\_Kj](https://discuss.elastic.co/u/Dazith_Kj)\
**Post date:** [May 8, 2018, 3:12pm UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/1 "2018-05-08T15:12:48Z")

</div>

I have design the below simple regex pattern. And this was working fine for me.

```
input {
    beats {
        port => "5044"
        ssl => true
        ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
        ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
    }
}
 filter {
    grok {
        match => { "message" => "(?<timestamp>[0-9]{4}.[0-9]{2}.[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3})"}
    }
}

output {
  elasticsearch {
    hosts => ["192.168.200.42:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
  }
}

```

However, when I tried with 2 regex patterns below conf file failed to wirte data to the elastic seach.

```
input {
    beats {
        port => "5044"
        ssl => true
        ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
        ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
    }
}
 filter {
    grok {
        match => { "message" => "(?<timestamp>[0-9]{4}.[0-9]{2}.[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3})"}
    }
    grok {
        match => { "message" => "(?<corid> [[0-9][a-z]]{8}-[[0-9][a-z]]{4}-[[0-9][a-z]]{4}-[[0-9][a-z]]{4}-[[0-9][a-z]]{12})"}
    }

}

output {
  elasticsearch {
    hosts => ["192.168.200.42:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
  }
}

```

Since I am new to ELK stack as well as for grok and it's behaviour appriciate if someone can help me on this.

Thanks in advance 🙂

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 8, 2018, 7:33pm UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/2 "2018-05-08T19:33:22Z")

</div>

If Logstash has problems sending to ES it'll tell you about it in the log.

While debugging inputs and filters use a simple output like `stdout { codec => rubydebug }`. Once you've confirmed that things are behaving as expected you can add additional complexity like an ES output.

---

<div class="post-metadata">

**Author:** ![Dazith\_Kj](https://avatars.discourse-cdn.com/v4/letter/d/b77776/32.png) [@Dazith\_Kj](https://discuss.elastic.co/u/Dazith_Kj)\
**Post date:** [May 9, 2018, 8:21am UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/3 "2018-05-09T08:21:42Z")

</div>

Thanks @magnusbaeck , Please guide me on how to add stdout. I did something like below. It didn't work as expected

```
output {
stdout { codec => rubydebug }
  elasticsearch {
    hosts => ["192.168.200.42:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
  }
}
```

---

<div class="post-metadata">

**Author:** ![vchandrashekar](https://avatars.discourse-cdn.com/v4/letter/v/f07891/32.png) [@vchandrashekar](https://discuss.elastic.co/u/vchandrashekar)\
**Post date:** [May 9, 2018, 8:28am UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/4 "2018-05-09T08:28:21Z")

</div>

You can just try this in output section of your conf file:

output {  
stdout { codec =\> rubydebug }  
}

Regards

---

<div class="post-metadata">

**Author:** ![Dazith\_Kj](https://avatars.discourse-cdn.com/v4/letter/d/b77776/32.png) [@Dazith\_Kj](https://discuss.elastic.co/u/Dazith_Kj)\
**Post date:** [May 9, 2018, 8:30am UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/5 "2018-05-09T08:30:50Z")

</div>

Thanks @vchandrashekar ! Can't I add both some which I mentioned above ?

---

<div class="post-metadata">

**Author:** ![vchandrashekar](https://avatars.discourse-cdn.com/v4/letter/v/f07891/32.png) [@vchandrashekar](https://discuss.elastic.co/u/vchandrashekar)\
**Post date:** [May 9, 2018, 8:44am UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/6 "2018-05-09T08:44:44Z")

</div>

with the rubydebug entry, you can verify if you are seeing the expected output.

Once are OK with the data, you debug why the data is not being sent to elastic search.

If possible please paste the error that you are seeing.

---

<div class="post-metadata">

**Author:** ![Dazith\_Kj](https://avatars.discourse-cdn.com/v4/letter/d/b77776/32.png) [@Dazith\_Kj](https://discuss.elastic.co/u/Dazith_Kj)\
**Post date:** [May 9, 2018, 9:25am UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/7 "2018-05-09T09:25:50Z")

</div>

@vchandrashekar logstash.log is the file that we need to check right ?

---

<div class="post-metadata">

**Author:** ![MrNerd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mrnerd/32/30763_2.png) [@MrNerd](https://discuss.elastic.co/u/MrNerd)\
**Post date:** [May 9, 2018, 9:40am UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/8 "2018-05-09T09:40:30Z")

</div>

yes.

---

<div class="post-metadata">

**Author:** ![Dazith\_Kj](https://avatars.discourse-cdn.com/v4/letter/d/b77776/32.png) [@Dazith\_Kj](https://discuss.elastic.co/u/Dazith_Kj)\
**Post date:** [May 9, 2018, 10:02am UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/9 "2018-05-09T10:02:27Z")

</div>

HI All,

This is the complete logstash config which I am having right now.

```
input {
    beats {
        port => "5044"
        ssl => true
        ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
        ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
    }
}
 filter {
    grok {
        match => {"message" => "(?<timestamp>[0-9]{4}.[0-9]{2}.[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3})"}
        match => {"message" => "(?<corid> [[0-9][a-z]]{8}-[[0-9][a-z]]{4}-[[0-9][a-z]]{4}-[[0-9][a-z]]{4}-[[0-9][a-z]]{12})"}
    }
}

output {
  stdout { codec => rubydebug }
}

```

And below is the tail of the logstash.log. Any idea on how to troubleshoot the above ?

> root@localhost:/var/log# tail -f logstash.log  
> tail: cannot open ‘logstash.log’ for reading: No such file or directory  
> root@localhost:/var/log# cd logstash/  
> root@localhost:/var/log/logstash# tail -f logstash.log  
> {:timestamp=\>"2018-05-09T14:10:31.256000+0530", :message=\>"SIGTERM received. Shutting down the pipeline.", :level=\>:warn}  
> {:timestamp=\>"2018-05-09T14:16:28.769000+0530", :message=\>"SIGTERM received. Shutting down the pipeline.", :level=\>:warn}  
> {:timestamp=\>"2018-05-09T14:19:28.381000+0530", :message=\>"SIGTERM received. Shutting down the pipeline.", :level=\>:warn}  
> {:timestamp=\>"2018-05-09T15:04:20.584000+0530", :message=\>"SIGTERM received. Shutting down the pipeline.", :level=\>:warn}
> 
> {:timestamp=\>"2018-05-09T15:07:47.107000+0530", :message=\>"SIGTERM received. Shutting down the pipeline.", :level=\>:warn}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2018, 10:02am UTC](https://discuss.elastic.co/t/grok-filter-with-multiple-regex-not-working/131031/10 "2018-06-06T10:02:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.