# How to exclude bad output (lines not matching 'grok' pattern) from logstash?

**URL:** <https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459>\
**Category:** Logstash\
**Created:** [January 29, 2016, 9:56am UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459 "2016-01-29T09:56:25Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![punit\_naik1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/punit_naik1/32/7239_2.png) [@punit\_naik1](https://discuss.elastic.co/u/punit_naik1)\
**Post date:** [January 29, 2016, 9:56am UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/1 "2016-01-29T09:56:25Z")

</div>

I have a log file and I am parsing it through Logstash and storing it in some place. But the the problem is that some lines in the log file do not always match my grok pattern and are therefore tagged as 'grokparsefailure' etc automatically. I do not tag any line explicitly, so the lines which are automatically tagged by logstash are the wrongly structured lines and I want these lines to be skipped i.e. I don't want these output lines to appear in the output at all (lines which have tags).

Can anyone help me to achieve this?  
Please help!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 29, 2016, 9:59am UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/2 "2016-01-29T09:59:25Z")

</div>

Use the drop filter to, well, drop events you don't want.

```auto
if "_grokparsefailure" in [tags] {
  drop { }
}

```

---

<div class="post-metadata">

**Author:** ![punit\_naik1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/punit_naik1/32/7239_2.png) [@punit\_naik1](https://discuss.elastic.co/u/punit_naik1)\
**Post date:** [January 29, 2016, 10:04am UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/3 "2016-01-29T10:04:19Z")

</div>

Thanks a lot @magnusbaeck! It totally worked!🙂

---

<div class="post-metadata">

**Author:** ![Tr\_ng\_Trang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tr_ng_trang/32/13053_2.png) [@Tr\_ng\_Trang](https://discuss.elastic.co/u/Tr_ng_Trang)\
**Post date:** [December 6, 2016, 6:59am UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/4 "2016-12-06T06:59:04Z")

</div>

hey guys, so i add it on block filter and after grok or where, pls reply soon. thanks you so much

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 6, 2016, 7:05am UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/5 "2016-12-06T07:05:55Z")

</div>

@Tr_ng_Trang, please open a new thread and supply more details.

---

<div class="post-metadata">

**Author:** ![UnitedMarsupials](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/unitedmarsupials/32/19429_2.png) [@UnitedMarsupials](https://discuss.elastic.co/u/UnitedMarsupials)\
**Post date:** [June 28, 2017, 7:38pm UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/6 "2017-06-28T19:38:58Z")

</div>

This construct is present in many examples online, but what if I still want the line logged, just differently?

If my `grok`-filter failed, I do not want any other filters applied, but still want to record the entire message in a separate output -- how would I achieve that?

The separate output part is easy -- by using checking for the `[tags]` -- but what about avoiding all the other filters _without dropping the event_? Must I move them all into a condition checking for `_grokparsefailure` _not_ being among the `[tags]`, or is there some other way? Thank you!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 28, 2017, 7:47pm UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/7 "2017-06-28T19:47:42Z")

</div>

> Must I move them all into a condition checking for `_grokparsefailure` not being among the `[tags]`

Yes, that's the way to do it.

---

<div class="post-metadata">

**Author:** ![bingbing](https://avatars.discourse-cdn.com/v4/letter/b/53a042/32.png) [@bingbing](https://discuss.elastic.co/u/bingbing)\
**Post date:** [August 15, 2019, 3:28am UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/8 "2019-08-15T03:28:46Z")

</div>

@magnusbaeck this method works for me too, but I want to know why the line with `_grokparsefailure` line are still sent to ES. I am new to use logstash and ES, from my perspective, the **filter** is to filter some lines which will be **grok-failed** and not sent to ES, is there something wrong, thanks a lot.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 7, 2019, 12:07pm UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/9 "2019-09-07T12:07:08Z")

</div>

Please start a new thread and provide additional details about your configuration.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2022, 4:13am UTC](https://discuss.elastic.co/t/how-to-exclude-bad-output-lines-not-matching-grok-pattern-from-logstash/40459/10 "2022-11-04T04:13:42Z")

</div>


