# Discarding unnecessary data

**URL:** <https://discuss.elastic.co/t/discarding-unnecessary-data/215677>\
**Category:** Logstash\
**Created:** [January 20, 2020, 7:24am UTC](https://discuss.elastic.co/t/discarding-unnecessary-data/215677 "2020-01-20T07:24:02Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![RichardH](https://avatars.discourse-cdn.com/v4/letter/r/9de0a6/32.png) [@RichardH](https://discuss.elastic.co/u/RichardH)\
**Post date:** [January 20, 2020, 7:24am UTC](https://discuss.elastic.co/t/discarding-unnecessary-data/215677/1 "2020-01-20T07:24:02Z")

</div>

Hello,

I am just starting with logstash and writing filters, and trying to better understand how it works. We want to push all syslog messages to logstash but I want to ensure that events are only sent to elastic if they match a grok filter. So if (using the Cisco example) I have a grok {  
match =\> [  
# IOS  
"message", "%{SYSLOG5424PRI}(%{NUMBER:log\_sequence#})? .... etc

What happens to messages that do not match the Grok pattern specified - are they discarded?

Also, what if I want to discard a portion of a message. I know that %{NUMBER:log\_sequence#} would assign the number to "log\_sequence), What if I want to drop the number altogether?

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [January 20, 2020, 2:08pm UTC](https://discuss.elastic.co/t/discarding-unnecessary-data/215677/2 "2020-01-20T14:08:06Z")

</div>

Hi there,

to answer your questions:

> What happens to messages that do not match the Grok pattern specified - are they discarded?

Not by default. By default they are ingested together with a field `tags` with value `_grokparsefailure` in it.

> We want to push all syslog messages to logstash but I want to ensure that events are only sent to elastic if they match a grok filter

If the event matches a grok, as you said, given fields will be populated (for example that `log_sequence`). Then, you can either drop the event in the filter section if a given field is not popuated or ingest the doc only if that field is populated. So:

```
filter {
  ...
  if ![log_sequence] {
    drop{}
  }
}

```

or

```
output {
  if [log_sequence] {
    ..whatever output you want..
  }
}

```

Choose the solution that better fits your needs.

> Also, what if I want to discard a portion of a message. I know that %{NUMBER:log\_sequence#} would assign the number to "log\_sequence), What if I want to drop the number altogether?

You can simply not assign anything to that part. In that case leaving %{NUMBER} without the `log_sequence` part will cause that part to go lost and not stored in your final document.

---

<div class="post-metadata">

**Author:** ![RichardH](https://avatars.discourse-cdn.com/v4/letter/r/9de0a6/32.png) [@RichardH](https://discuss.elastic.co/u/RichardH)\
**Post date:** [January 21, 2020, 7:34am UTC](https://discuss.elastic.co/t/discarding-unnecessary-data/215677/3 "2020-01-21T07:34:23Z")

</div>

Got it - Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 18, 2020, 7:34am UTC](https://discuss.elastic.co/t/discarding-unnecessary-data/215677/4 "2020-02-18T07:34:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.