# Logstash - parse/ filter lines having a specific string

**URL:** https://discuss.elastic.co/t/logstash-parse-filter-lines-having-a-specific-string/288103
**Category:** Logstash
**Created:** [October 31, 2021, 11:38pm UTC](https://discuss.elastic.co/t/logstash-parse-filter-lines-having-a-specific-string/288103 "2021-10-31T23:38:08Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)
#### Post date: [October 31, 2021, 11:38pm UTC](https://discuss.elastic.co/t/logstash-parse-filter-lines-having-a-specific-string/288103/1 "2021-10-31T23:38:08Z")

</div>

Hi All,

We are running ELK 7.6.2 stack in our environment.

The log generated by our application throws out messages with different Log Levels. Please see below example:

```auto
2021-10-31 19:00:01,062|DEBUG|DispatcherServlet|anonymous-1.2.3.4.106-TVZ-PCVLSB69GCBUWD1NCMAVMYJJJ4YC6KG1744901501|Exiting from "ASYNC" dispatch
2021-10-31 19:00:01,059| INFO|HttpRequestUtil|anonymous-1.2.3.4.106-TVZ-PCVLSB69GCBUWD1NCMAVMYJJJ4YC6KG1744901501|ERROR STATUS CODE 500
2021-10-31 19:00:01,060| INFO|ErrorController|anonymous-1.2.3.4.106-TVZ-PCVLSB69GCBUWD1NCMAVMYJJJ4YC6KG1744901501|ERROR ASYNC HANDLER async isCommitted=false
2021-10-31 19:20:01,286| INFO|HttpRequestUtil|anonymous-1.2.3.4-1-01-CV-PCVY73MMUJJ3LBIMAMGJ97EBIPLWJWG1745536201@1-1163547#17|ERROR STATUS CODE 401
2021-10-31 19:20:32,606|ERROR|SlUtil|1.2.3.4-TVZ-PCVZXJECFBRFBM43FFEREJUAFEUWUMG4732288802@1-1163852#17|Parsing the request failed

```

Filebeat agent running on the server pushes this content to Logstash.

I want Logstash to just parse/ filter the line that has "ERROR" as Log Level (Last line in the sample log above) and push it to Elasticsearch. The rest should be discarded.

Please guide on how this could be achieved.

Thanks

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 31, 2021, 11:55pm UTC](https://discuss.elastic.co/t/logstash-parse-filter-lines-having-a-specific-string/288103/2 "2021-10-31T23:55:03Z")

</div>

It could be as simple as

```
if "|ERROR|" not in [message] { drop {} }
```

---

<div class="post-metadata">

### Author: ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)
#### Post date: [November 3, 2021, 8:40pm UTC](https://discuss.elastic.co/t/logstash-parse-filter-lines-having-a-specific-string/288103/3 "2021-11-03T20:40:55Z")

</div>

Thanks @Badger as always!

I introduced this in my logtstash config file as the following snippet:

```auto
        if [type] == "tv_proxy_log" {
                mutate {
                        split => ["message", "|"]
                        add_field =>{
                           "createdTime" => "%{[message][0]}"
                           "logLevel" => "%{[message][1]}"
                           "className" => "%{[message][2]}"
                           "reqID" => "%{[message][3]}"
                           "message" => "%{[message][4]}"

                       }
                }
        if "|ERROR|" not in [message] { drop {} }

                      -------------------------------
                      -------------------------------
                      -------------------------------

```

As a result it has stopped showing "all" messages in Kibana, whereas I expect it to show the following message:

```auto
2021-11-03 16:34:49,208|ERROR|UserController|TV-UCVZ3CJK4CCJAJ9JCT4DFYFCBZM6FWG3013461403|Not sufficient auth level to get user profile

```

Please guide

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 3, 2021, 8:57pm UTC](https://discuss.elastic.co/t/logstash-parse-filter-lines-having-a-specific-string/288103/4 "2021-11-03T20:57:51Z")

</div>

> [@zaeemmasood](#):
>
> `split => ["message", "|"]`

After this [message] is an array that does not contain |, so everything gets dropped. Try

```
if [logLevel] != "ERROR" { drop {} }

```

---

<div class="post-metadata">

### Author: ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)
#### Post date: [November 4, 2021, 1:35pm UTC](https://discuss.elastic.co/t/logstash-parse-filter-lines-having-a-specific-string/288103/5 "2021-11-04T13:35:26Z")

</div>

Thanks @Badger. It worked!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 2, 2021, 1:35pm UTC](https://discuss.elastic.co/t/logstash-parse-filter-lines-having-a-specific-string/288103/6 "2021-12-02T13:35:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
