Logstash mutate - convert empty string to null

Elastic Stack Logstash
1

There is a field in log file, which contains IP address or empty string if IP address is not available.
Elasticsearch index has mapping, that maps "ip" type to this field. When field value is empty string, logstash can not save data to index:

[WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch
...
"error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [doc.ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'' is not an IP string literal."}}}}}

To fix this, I need to convert empty string to null with logstash. This doesn't work:

if [doc][ip] == "" {
  mutate {
    replace => { "[doc][ip]" => null }
  }
}

Because it converts value to string "null". "nil" doesn't work either.

What is correct syntax to convert field to null value?

2

The Logstash configuration language doesn't support null values. You'll have to use a ruby filter. Or could you just remove the field?

1 Like
3

Why not just delete the field? It's more straightforward than trying to force null values from the logstash side.

if [doc][ip] == "" {
  mutate {
   remove_field => ["[doc][ip]"]
  }
}

You could also take a look on this mapping setting

3 Likes
4

Thank you, @paz and @magnusbaeck. Removing the field instead of setting it to null solves the problem :thumbsup:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.