# Where should I drop records? And how?

**URL:** <https://discuss.elastic.co/t/where-should-i-drop-records-and-how/252383>\
**Category:** Logstash\
**Created:** [October 16, 2020, 6:30pm UTC](https://discuss.elastic.co/t/where-should-i-drop-records-and-how/252383 "2020-10-16T18:30:29Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mbuster](https://avatars.discourse-cdn.com/v4/letter/m/3d9bf3/32.png) [@mbuster](https://discuss.elastic.co/u/mbuster)\
**Post date:** [October 16, 2020, 6:30pm UTC](https://discuss.elastic.co/t/where-should-i-drop-records-and-how/252383/1 "2020-10-16T18:30:29Z")

</div>

Background:

- Sending Sonicwall syslog to Logstash in order to keep records on SSL VPN logins/logouts
- Sonicwall indices are accounting for a majority of my storage usage

Need:

- Drop Sonicwall records based on specific criteria

Detail:

- I'm using the generic Sonicwall filter that's been floating around for a while now and it works just fine for getting documents/events into Elasticsearch.
- In Elasticsearch, I'm able to filter events based on the 'msg' field. I'm only using two queries:
  - msg is SSL VPN zone remote user login allowed
  - msg is User logged out\*

- I'd like to cut down on the Sonicwall logging data I'm storing and I figure I can drop records at logstash before they even coming into Elasticsearch
  - Thought I could tag records that have a specific 'msg' and then drop any records that don't have that tag. I can't seem to make this work...

Questions:

- Should I be dropping records in Logstash or Elasticsearch?
- Is Logstash breaking up the syslog event into its parsable chunks or is Elasticsearch?

I've attempted to add an 'if' statement to my logstash filter:

```auto
  if [msg] == \"SSL VPN zone remote user login allowed\" {
    mutate {
      add_tag => ["valid"]
    }
  }
  if "valid" not in [tags] {            
    drop { }
  }

```

But the above just crashed logstash... Even if I take out the second 'if' to do the drop, logstash doesn't like it.

Looking for guidance. Thanks!!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 16, 2020, 8:27pm UTC](https://discuss.elastic.co/t/where-should-i-drop-records-and-how/252383/2 "2020-10-16T20:27:21Z")

</div>

> [@mbuster](#):
>
> `if [msg] == \"SSL VPN zone remote user login allowed\" {`

Remove the backslashes before the double quotes.

Note that == is a string equality test. That would match the string

```
SSL VPN zone remote user login allowed

```

but not the string

```
SSL VPN zone remote user login allowed again

```

You could use a regexp match

```
if [msg] =~ "SSL VPN zone remote user login allowed" {

```

even through it is not a regular expression, or you could use a sub-string match

```
if "SSL VPN zone remote user login allowed" in [msg] {

```

---

<div class="post-metadata">

**Author:** ![mbuster](https://avatars.discourse-cdn.com/v4/letter/m/3d9bf3/32.png) [@mbuster](https://discuss.elastic.co/u/mbuster)\
**Post date:** [October 19, 2020, 6:18pm UTC](https://discuss.elastic.co/t/where-should-i-drop-records-and-how/252383/3 "2020-10-19T18:18:59Z")

</div>

Badger -

Thank you so much for your reply and help!

It got me on the right track and now I'm successfully tagging records and dropping records that don't have the tag!

Thanks again!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 16, 2020, 6:19pm UTC](https://discuss.elastic.co/t/where-should-i-drop-records-and-how/252383/4 "2020-11-16T18:19:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.