# Null validation in logstash

**URL:** <https://discuss.elastic.co/t/null-validation-in-logstash/49456>\
**Category:** Logstash\
**Created:** [May 7, 2016, 7:58am UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456 "2016-05-07T07:58:07Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![baneshbanna](https://avatars.discourse-cdn.com/v4/letter/b/5e9695/32.png) [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\
**Post date:** [May 7, 2016, 7:58am UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456/1 "2016-05-07T07:58:07Z")

</div>

How to check null dates in logstash.  
Null validation in logstash.  
I added below configuration is it correct  
if[datetime]!='null' and [datetime]!='NULL'{

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 7, 2016, 8:16am UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456/2 "2016-05-07T08:16:30Z")

</div>

No, because you are are now looking for those actual values.

You should be able to use a ruby filter, something like;

```auto
        ruby {
                code => "
                        hash = event.to_hash
                        hash.each do |k,v|
                                if v == nil
                                        event.remove(k)
                                end
                        end
                "
        }

```

You can also do `if [datetime] and [datetime]` if the fields aren't even being created.

---

<div class="post-metadata">

**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\
**Post date:** [May 7, 2016, 8:19am UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456/3 "2016-05-07T08:19:58Z")

</div>

To test if a field does not exist, the syntax is :  
`if ![datetime]`

---

<div class="post-metadata">

**Author:** ![baneshbanna](https://avatars.discourse-cdn.com/v4/letter/b/5e9695/32.png) [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\
**Post date:** [May 7, 2016, 10:41am UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456/4 "2016-05-07T10:41:45Z")

</div>

in csv i have 4 fields .i wnat to check null validation on one field  
for example:  
transid,status,datetime

i want add null validation on datetime field.  
how to add.

---

<div class="post-metadata">

**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\
**Post date:** [May 7, 2016, 8:12pm UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456/5 "2016-05-07T20:12:54Z")

</div>

can you give csv data example where you have "null" values ?

---

<div class="post-metadata">

**Author:** ![baneshbanna](https://avatars.discourse-cdn.com/v4/letter/b/5e9695/32.png) [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\
**Post date:** [May 7, 2016, 9:06pm UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456/6 "2016-05-07T21:06:32Z")

</div>

"clientid"=\>238, "groupid"=\>550, "teamid"=\>843, "userid"=\>28, "trans\_id"=\>"1462098183\_90927569",  
"eventtransid"=\>"1462098183\_90927569", "serviceid"=\>2948, "servicekey"=\>"cc01cce5-0aad-11e6-a89d-005056944c2b",  
"servicename"=\>"DVP GW testing", "crn"=\>"2036", "eventid"=\>8, "eventtype"=\>"MISSED\_CALL", "pulsecount"=\>1,  
"ani"=\>"919440324831", "dnis"=\>"+914066977906", "answeredon"=\>"null", "releasedon"=\>"2016-05-01T10:23:03.565Z",  
"acceptedon"=\>"2016-05-01T10:23:03.501Z", "offeredon"=\>"2016-05-01T10:23:03.373Z", "statusid"=\>837,  
"statusname"=\>"voice call summary", "ostype"=\>0, "osname"=\>"null", "dvpdropstate"=\>2, "dvpdropcode"=\>2006,

in this answered on is null

---

<div class="post-metadata">

**Author:** ![baneshbanna](https://avatars.discourse-cdn.com/v4/letter/b/5e9695/32.png) [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\
**Post date:** [May 7, 2016, 9:09pm UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456/7 "2016-05-07T21:09:27Z")

</div>

238550843281462098183\_909275691462098183\_909275692948cc01cce5-0aad-11e6-a89d-005056944c2bDVP GW testing20368MISSED\_CALL1919440324831+914066977906null2016-05-01 15:53:03.5652016-05-01 15:53:03.5012016-05-01 15:53:03.373837voice call summary0null22006OthersAnswer/logdata/apps/dvp/callflows/missedcallflow/missedcall.txt102016-05-01 15:53:14.659null00null

---

<div class="post-metadata">

**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\
**Post date:** [May 7, 2016, 10:26pm UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456/8 "2016-05-07T22:26:55Z")

</div>

OK, in your case, date conversion fails, so field is filled with "null".

=\> To detect this case, I advice this if :  
`if [datetime] != 'null' {`

No need to check 'NULL'.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:58am UTC](https://discuss.elastic.co/t/null-validation-in-logstash/49456/9 "2017-07-06T04:58:53Z")

</div>