# Data Validation in Logstash

**URL:** <https://discuss.elastic.co/t/data-validation-in-logstash/277922>\
**Category:** Logstash\
**Created:** [July 6, 2021, 7:57am UTC](https://discuss.elastic.co/t/data-validation-in-logstash/277922 "2021-07-06T07:57:12Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pallavibhushan](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pallavibhushan](https://discuss.elastic.co/u/Pallavibhushan)\
**Post date:** [July 6, 2021, 7:57am UTC](https://discuss.elastic.co/t/data-validation-in-logstash/277922/1 "2021-07-06T07:57:12Z")

</div>

Hi,  
I am trying data validation in logstash.  
Example : column 1 in CSV should only allow number (double data type).  
No other text should be allowed if any other string found then trigger mail.  
Similarly, need date validation. if date is not in required format trigger mail and do not process the CSV file until correct data isn't placed.

How to handle this in Logstash.

Please help me with this.

Thanks in advance

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 6, 2021, 4:08pm UTC](https://discuss.elastic.co/t/data-validation-in-logstash/277922/2 "2021-07-06T16:08:27Z")

</div>

You can use a date filter to check whether the date is in the required format. If it fails to parse the date it will add a \_dateparsefailure tag to the event.

If you want to check whether a field is a number you could use something like

```
if [someField] !~ /^[\d\.]+$/ { mutate { add_tag => ["_numberformatfailure"] } }

```

(depending on your number format you may need to adjust the pattern).

Then in the output section route the event based on the tags...

```
if "_dateparsefailure" in [tags] or "_numberformatfailure" in [tags] {
    mail { ... }
} else {
    some other destination
}

```

Note that this is per-event, not per-file. Validating all the rows in a file before processing it can probably be done, but it is not a use case that logstash is well adapted to.

---

<div class="post-metadata">

**Author:** ![Pallavibhushan](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pallavibhushan](https://discuss.elastic.co/u/Pallavibhushan)\
**Post date:** [July 6, 2021, 4:43pm UTC](https://discuss.elastic.co/t/data-validation-in-logstash/277922/3 "2021-07-06T16:43:58Z")

</div>

> [@Badger](#):
>
> ```auto
> if "_dateparsefailure" in [tags] or "_numberformatfailure" in [tags] {
> mail { ... }
> } else {
> some other destination
> }
> 
> ```

Thank you so much . I'll check this

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 3, 2021, 4:44pm UTC](https://discuss.elastic.co/t/data-validation-in-logstash/277922/4 "2021-08-03T16:44:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.