How to deal with optionals fields in Logstash/ElasticSearch

have a problem when i try to import the following XML file in ElasticSearch with Logstash:

<param name="param1" raw="0x01">
<param name="param2" raw="0x02" eng="2">
<param name="param3" raw="0x03" eng="3">

My Logstash conf file is the following (extract):

xml {
    store_xml => false
    source => "message"
    xpath => [
        "/test/param/@name", "name",
        "/test/param/@raw", "raw",
        "/test/param/@eng", "eng"

After running Logstash, the ElasticSearch database contains:

"name": [
"raw": [
"eng": [

I want this:

"eng": [

How can i force a null (or None or NaN) value in a non existing field ?

Thanks in advance !

I try to set the suppress_empty option to false but the result is the same...

Any idea ?

Have you tried using a conditional statement for the empty fields? So say that eng is the field that is always empty

if [eng] == ""{mutate{update=>{"eng"=>"null"}}}

Or something along that line (code may vary)?

I already try but nothing change...

I think that after the xpath command the eng is not empty but equal to "eng": [2, 3]. So the code do not enter in the if statement.

So if the field eng is not even in your log message then the if statement above needs to be changed. I am sorry I did not notice that at first. If if is not within the message but you still want it nulled in the filter section of the code try this:

if ![eng]{
      add_field => {"eng" => "null"}
1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.