# How to deal with optionals fields in Logstash/ElasticSearch

**URL:** <https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965>\
**Category:** Logstash\
**Created:** [July 28, 2017, 2:09pm UTC](https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965 "2017-07-28T14:09:20Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![NicolasHennion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nicolashennion/32/20561_2.png) [@NicolasHennion](https://discuss.elastic.co/u/NicolasHennion)\
**Post date:** [July 28, 2017, 2:09pm UTC](https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965/1 "2017-07-28T14:09:20Z")

</div>

have a problem when i try to import the following XML file in ElasticSearch with Logstash:

```
<test>
<param name="param1" raw="0x01">
<param name="param2" raw="0x02" eng="2">
<param name="param3" raw="0x03" eng="3">
</test>

```

My Logstash conf file is the following (extract):

```
xml {
    store_xml => false
    source => "message"
    xpath => [
        "/test/param/@name", "name",
        "/test/param/@raw", "raw",
        "/test/param/@eng", "eng"
    ]
}

```

After running Logstash, the ElasticSearch database contains:

```
"name": [
      "param1",
      "param2",
      "param3",
    ],
"raw": [
      "0x01",
      "0x02",
      "0x02",
    ],
"eng": [
      2,
      3,
    ],

```

I want this:

```
...
"eng": [
      null,
      2,
      3,
    ],

```

How can i force a null (or None or NaN) value in a non existing field ?

Thanks in advance !

---

<div class="post-metadata">

**Author:** ![NicolasHennion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nicolashennion/32/20561_2.png) [@NicolasHennion](https://discuss.elastic.co/u/NicolasHennion)\
**Post date:** [August 3, 2017, 2:27pm UTC](https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965/2 "2017-08-03T14:27:25Z")

</div>

I try to set the suppress\_empty option to false but the result is the same...

Any idea ?

---

<div class="post-metadata">

**Author:** ![CDR](https://avatars.discourse-cdn.com/v4/letter/c/d9b06d/32.png) [@CDR](https://discuss.elastic.co/u/CDR)\
**Post date:** [August 3, 2017, 2:48pm UTC](https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965/3 "2017-08-03T14:48:52Z")

</div>

Have you tried using a conditional statement for the empty fields? So say that eng is the field that is always empty

`if [eng] == ""{mutate{update=>{"eng"=>"null"}}}`

Or something along that line (code may vary)?

---

<div class="post-metadata">

**Author:** ![NicolasHennion](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nicolashennion/32/20561_2.png) [@NicolasHennion](https://discuss.elastic.co/u/NicolasHennion)\
**Post date:** [August 4, 2017, 7:14am UTC](https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965/4 "2017-08-04T07:14:09Z")

</div>

I already try but nothing change...

I think that after the xpath command the eng is not empty but equal to "eng": [2, 3]. So the code do not enter in the if statement.

---

<div class="post-metadata">

**Author:** ![CDR](https://avatars.discourse-cdn.com/v4/letter/c/d9b06d/32.png) [@CDR](https://discuss.elastic.co/u/CDR)\
**Post date:** [August 4, 2017, 12:08pm UTC](https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965/5 "2017-08-04T12:08:33Z")

</div>

So if the field eng is not even in your log message then the if statement above needs to be changed. I am sorry I did not notice that at first. If if is not within the message but you still want it nulled in the filter section of the code try this:

```
if ![eng]{
   mutate{
      add_field => {"eng" => "null"}
   }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 1, 2017, 12:08pm UTC](https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965/6 "2017-09-01T12:08:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.