# Grok filter in logstash for more than one different lines

**URL:** <https://discuss.elastic.co/t/grok-filter-in-logstash-for-more-than-one-different-lines/288108>\
**Category:** Logstash\
**Created:** [November 1, 2021, 4:53am UTC](https://discuss.elastic.co/t/grok-filter-in-logstash-for-more-than-one-different-lines/288108 "2021-11-01T04:53:33Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![root\_linux](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@root\_linux](https://discuss.elastic.co/u/root_linux)\
**Post date:** [November 1, 2021, 4:53am UTC](https://discuss.elastic.co/t/grok-filter-in-logstash-for-more-than-one-different-lines/288108/1 "2021-11-01T04:53:33Z")

</div>

I have logs in which there are different types of lines are present. And I want to write grok filter for each line. How I can I achieve it? My logs are as given below:

```auto
Oct 5 14:05:53 centos-8gb-nbg1-1 openvpn: Tue Oct 5 14:05:53 2021 TCP connection established with [AF_INET6]::ffff:192.168.10.1:55630
Oct 5 14:05:54 centos-8gb-nbg1-1 openvpn: Tue Oct 5 14:05:54 2021 192.168.10.1:55630 peer info: IV_GUI_VER=windows.NT.x86_64
Oct 5 14:05:54 centos-8gb-nbg1-1 openvpn: Tue Oct 5 14:05:54 2021 192.168.10.1:55630 peer info: IV_PLAT_VER=30_11_arm64-v8a_samsung_exynos9610_SM-A505F
Oct 5 14:31:44 centos-8gb-nbg1-1 openvpn: Tue Oct 5 14:01:44 2021 google/192.168.10.1:55418 Connection reset, restarting [0]

```

Please guide me how can I write grok filter for these lines.

```auto

```

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [November 1, 2021, 3:26pm UTC](https://discuss.elastic.co/t/grok-filter-in-logstash-for-more-than-one-different-lines/288108/2 "2021-11-01T15:26:12Z")

</div>

Use this website to help you write your patterns.  
It looks like it's already semi-well structured so you may be able to do something like this:

`%{MONTH} %{MONTHDAY} %{TIME} %{HOSTNAME:hostname} %{NOTSPACE:service} %{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{GREEDYDATA:log_message}`

> {  
> "MONTH": [  
> [  
> "Oct",  
> "Oct"  
> ]  
> ],  
> "MONTHDAY": [  
> [  
> "5",  
> "5"  
> ]  
> ],  
> "TIME": [  
> [  
> "14:05:53",  
> "14:05:53"  
> ]  
> ],  
> "HOUR": [  
> [  
> "14",  
> "14"  
> ]  
> ],  
> "MINUTE": [  
> [  
> "05",  
> "05"  
> ]  
> ],  
> "SECOND": [  
> [  
> "53",  
> "53"  
> ]  
> ],  
> "hostname": [  
> [  
> "centos-8gb-nbg1-1"  
> ]  
> ],  
> "service": [  
> [  
> "openvpn:"  
> ]  
> ],  
> "DAY": [  
> [  
> "Tue"  
> ]  
> ],  
> "log\_message": [  
> [  
> "2021 TCP connection established with [AF\_INET6]::ffff:192.168.10.1:55630"  
> ]  
> ]  
> }

---

<div class="post-metadata">

**Author:** ![root\_linux](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@root\_linux](https://discuss.elastic.co/u/root_linux)\
**Post date:** [November 2, 2021, 4:51am UTC](https://discuss.elastic.co/t/grok-filter-in-logstash-for-more-than-one-different-lines/288108/3 "2021-11-02T04:51:40Z")

</div>

Hi @AquaX , Thanks for your response.  
I am beginner in ELK. So could you guide me how can write different grok patterns in `logstash.conf` for each line?

```auto
%{SYSLOGTIMESTAMP:time} %{HOSTNAME:hostname} %{SYSLOGHOST:service}: %{DAY:day} %{SYSLOGTIMESTAMP:connection_est_time} %{YEAR:year} %{IP:server_ip}:%{NUMBER:port} peer info: IV_GUI_VER=%{GREEDYDATA:client_version}
%{SYSLOGTIMESTAMP:time} %{HOSTNAME:hostname} %{SYSLOGHOST:service}: %{DAY:day} %{SYSLOGTIMESTAMP:conn_est_time} %{YEAR:year} %{IP:server_ip}:%{NUMBER:port} peer info: IV_PLAT_VER=%{GREEDYDATA:device_details}

```

I have written these grok pattern for 2nd and 3rd lines of above logs. How can I use it in logstash configuration file?

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [November 2, 2021, 11:27am UTC](https://discuss.elastic.co/t/grok-filter-in-logstash-for-more-than-one-different-lines/288108/4 "2021-11-02T11:27:48Z")

</div>

Yes, you can have multiple patterns per line.

Logstash.conf

```auto

filter{
    grok {
      patterns_dir => "/pathtopaterns/filepatterns.txt"
      match => { 
        "message" => ["%{CUSTOM_1}", "%{CUSTOM_2}"]
      }
    }
}

```

filepatterns.txt content:

```auto
# Pattern 1
CUSTOM_1 
%{SYSLOGTIMESTAMP:time} %{HOSTNAME:hostname} %{SYSLOGHOST:service}: %{DAY:day} %{SYSLOGTIMESTAMP:connection_est_time} %{YEAR:year} %{IP:server_ip}:%{NUMBER:port} peer info: IV_GUI_VER=%{GREEDYDATA:client_version}

# Pattern 2
CUSTOM_2 %{SYSLOGTIMESTAMP:time} %{HOSTNAME:hostname} %{SYSLOGHOST:service}: %{DAY:day} %{SYSLOGTIMESTAMP:conn_est_time} %{YEAR:year} %{IP:server_ip}:%{NUMBER:port} peer info: IV_PLAT_VER=%{GREEDYDATA:device_details}

```

Also you can you use "if" instead of multiple match:

```auto
if [field1] == "something" {
mutate {rename, update, replace ...}
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 2, 2021, 4:58pm UTC](https://discuss.elastic.co/t/grok-filter-in-logstash-for-more-than-one-different-lines/288108/5 "2021-11-02T16:58:55Z")

</div>

You can match against an array of patterns.

```
grok {
    match => {
        "message" => [
            "%{SYSLOGTIMESTAMP:time} %{HOSTNAME:hostname} %{SYSLOGHOST:service}: %{DAY:day} %{SYSLOGTIMESTAMP:connection_est_time} %{YEAR:year} %{IP:server_ip}:%{NUMBER:port} peer info: IV_GUI_VER=%{GREEDYDATA:client_version}",
            "%{SYSLOGTIMESTAMP:time} %{HOSTNAME:hostname} %{SYSLOGHOST:service}: %{DAY:day} %{SYSLOGTIMESTAMP:conn_est_time} %{YEAR:year} %{IP:server_ip}:%{NUMBER:port} peer info: IV_PLAT_VER=%{GREEDYDATA:device_details}"
        ]
    }
}

```

If the lines start with the SYSLOGTIMESTAMP then I strongly recommend that you anchor your patterns to the start of line: "^%{SYSLOGTIMESTAMP:time} ...". This makes it much cheaper to evaluate them when they do not match, and if you need an array of patterns there will always be some that do not match.

---

<div class="post-metadata">

**Author:** ![root\_linux](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@root\_linux](https://discuss.elastic.co/u/root_linux)\
**Post date:** [November 3, 2021, 6:37am UTC](https://discuss.elastic.co/t/grok-filter-in-logstash-for-more-than-one-different-lines/288108/6 "2021-11-03T06:37:56Z")

</div>

@Rios and @Badger, thanks for solution 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 1, 2021, 6:38am UTC](https://discuss.elastic.co/t/grok-filter-in-logstash-for-more-than-one-different-lines/288108/7 "2021-12-01T06:38:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.