Conditional for empty field

[Lukas_Tilch]

Hello I'm trying to parse a date in a field which sometimes contains no data

I'm getting this error log when I try to parse it.
[2017-06-01T16:43:23,036][WARN ][logstash.outputs.elasticsearch] Failed action. {:status=>400, :action=>["update", {:_id=>"7345936", :_index=>"logstash-issue", :_type=>"issue", :_routing=>nil, :_retry_on_conflict=>1}, 2017-06-01T14:43:18.681Z Lukass-MBP.fritz.box %{message}], :response=>{"update"=>{"_index"=>"logstash-issue", "_type"=>"issue", "_id"=>"7345936", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [PlannedCompletionDate]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"\""}}}}}

That leads to the Data not being indexed. So I thought of checking beforehand if the field contains anything.

i've tried conditionals like
if !([PlannedCompletionDate]=~/.+/) { date {...} }

if [PlannedCompletionDate] != ~/.+/ { date {...} }

if [PlannedCompletionDate] != "" { date {...} }

none of which worked at all.

any Ideas?
Thanks in advance

[magnusbaeck (Magnus Bäck)]

The last conditional should work. Please use a stdout { codec => rubydebug } output instead of your elasticsearch output so we can see exactly what your event looks like.

But the problem here isn't that the date filter is getting told to parse an empty string, it's that Elasticsearch is given an empty string. Perhaps you should delete the PlannedCompletionDate field if it's empty?

[Lukas_Tilch]

thank you,

that was in fact why elasticsearch threw the error.

rubydebug dind't show any errors, but I didn't quite make the connection that the Format might be the problem for ES

solved it like this.

thanks again

if [PlannedCompletionDate] != "" {
    date {
      match => ["PlannedCompletionDate", "MM/dd/YYYY"]
      target => "PlannedCompletionDate"
    }
    }

    if [PlannedCompletionDate] == "" {
      mutate{
      remove_field => ["PlannedCompletionDate"]
    }
    }