# \[Logstash\] How to drop message if field is not a number

**URL:** <https://discuss.elastic.co/t/logstash-how-to-drop-message-if-field-is-not-a-number/333324>\
**Category:** Logstash\
**Created:** [May 12, 2023, 3:43pm UTC](https://discuss.elastic.co/t/logstash-how-to-drop-message-if-field-is-not-a-number/333324 "2023-05-12T15:43:14Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![catalin.bulancea](https://avatars.discourse-cdn.com/v4/letter/c/e56c9b/32.png) [@catalin.bulancea](https://discuss.elastic.co/u/catalin.bulancea)\
**Post date:** [May 12, 2023, 3:43pm UTC](https://discuss.elastic.co/t/logstash-how-to-drop-message-if-field-is-not-a-number/333324/1 "2023-05-12T15:43:14Z")

</div>

Hi Logstash gurus,

I need to drop the messages that contain specific fields that are not a number.

The filter I have is:

```auto
filter {
  csv {
      separator => ","
      skip_header => "true"
      columns => ["process-name","upload-bw","download-bw","process-owner","filename","hostname"]
	  convert => {
		"upload-bw" => "float"
		"download-bw" => "float"
	  }
  }
}

```

Sometimes, upload-bw and download-bw contain characters, so I get conflicts in the index pattern and the dashboards don't render.

So I want to drop all those messages with upload-bw and download-bw that are not a float.

Can you help?

Thank you,  
Catalin

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 12, 2023, 4:29pm UTC](https://discuss.elastic.co/t/logstash-how-to-drop-message-if-field-is-not-a-number/333324/2 "2023-05-12T16:29:39Z")

</div>

You could try

```
    ruby {
        code => '
            ["upload-bw", "download-bw"].each { |x|
                if ! event.get(x).is_a? Float; event.remove(x); end
            }
        '
    }

```

---

<div class="post-metadata">

**Author:** ![Anton\_H](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anton_h/32/10200_2.png) [@Anton\_H](https://discuss.elastic.co/u/Anton_H)\
**Post date:** [May 13, 2023, 11:57am UTC](https://discuss.elastic.co/t/logstash-how-to-drop-message-if-field-is-not-a-number/333324/3 "2023-05-13T11:57:42Z")

</div>

Or you could use a grok filter in stead of csv to parse your lines, if the upload-bw and download-bw do not contain numbers then you have no match and can drop the event based on the \_grokparsefailure tag.

```auto
filter {
    grok {
        match => { "message" => "%{WORD:process-name}, %{NUMBER:upload-bw}, %{NUMBER:download-bw}, (?<process-owner>([a-zA-Z]*)), (?<filename>([a-zA-Z\-\_\.]*)), %{WORD:hostname}" }
    }
    if "_grokparsefailure" in [tags] {
        drop { }
    }
}

```

This is just an example, you would have to spend time debugging your grok/regex to match your data, while your csv solution with the ruby code works out of the box.  
I'm adding this comment as an option because it might be worth to investage what solution is more "expensive" in resources.

---

<div class="post-metadata">

**Author:** ![catalin.bulancea](https://avatars.discourse-cdn.com/v4/letter/c/e56c9b/32.png) [@catalin.bulancea](https://discuss.elastic.co/u/catalin.bulancea)\
**Post date:** [May 16, 2023, 7:36am UTC](https://discuss.elastic.co/t/logstash-how-to-drop-message-if-field-is-not-a-number/333324/4 "2023-05-16T07:36:12Z")

</div>

Hi Badger, Anton, thank you!  
Let me try to test both approaches and will get back to you...

Catalin

---

<div class="post-metadata">

**Author:** ![catalin.bulancea](https://avatars.discourse-cdn.com/v4/letter/c/e56c9b/32.png) [@catalin.bulancea](https://discuss.elastic.co/u/catalin.bulancea)\
**Post date:** [May 25, 2023, 3:59pm UTC](https://discuss.elastic.co/t/logstash-how-to-drop-message-if-field-is-not-a-number/333324/5 "2023-05-25T15:59:49Z")

</div>

Hi Badger,

I tried the ruby code you suggested and it works! Well, it's not dropping the whole message, but it empties the upload-bw and download-bw fields so the index doesn't conflict anymore.

I changed it a bit, instead of remove I put event.set(x, 0.0).

Thank you!  
Catalin

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 22, 2023, 4:00pm UTC](https://discuss.elastic.co/t/logstash-how-to-drop-message-if-field-is-not-a-number/333324/6 "2023-06-22T16:00:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.