# Issue with Elasticsearch mapping

**URL:** <https://discuss.elastic.co/t/issue-with-elasticsearch-mapping/112903>\
**Category:** Logstash\
**Created:** [December 21, 2017, 10:42pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-mapping/112903 "2017-12-21T22:42:53Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [December 21, 2017, 10:42pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-mapping/112903/1 "2017-12-21T22:42:53Z")

</div>

I've been getting these errors nonstop all day and haven't been able to figure out a solution. From investigating the issue, it looks like our data input is giving us a JSON object that I want to parse with the JSON filter, but the issue (I think) is that one of the object's properties is another object, which has a property that is sometimes set to `null`. The parent object is the field `[extras]` and then that contains a child field called `[visitors]` and that contains a child object called `[user_data]`.

`[2017-12-21T22:31:48,253][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"metrics-buffalonewscom-2017-09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x6d9f48f6>], :response=>{"index"=>{"_index"=>"metrics-buffalonewscom-2017-09", "_type"=>"doc", "_id"=>"_cw0e2ABluZYKCJJOIpQ", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [visitor.user_data] tried to parse field [user_data] as object, but found a concrete value"}}}}`

My pipeline first uses the JSON filter on `[extras]` and then on `[visitors][user_data]`, but it shows this warning when it runs into issues with empty or null values for it. I've tried removing the field `if [visitor][user_data] == "null"` but that evidently isn't working.

Is there a simpler solution? I would love it if the JSON filter just had an option that handled this kind of issue (i.e. if Elasticsearch won't be able to index it correctly then it just removes the field).

What is the best way to handle this?

Thank you.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [December 22, 2017, 10:30am UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-mapping/112903/2 "2017-12-22T10:30:37Z")

</div>

Before you actually send the data to elasticsearch try to experiment with a simple config.  
e.g.

```auto
input {
  generator {
    message => '{"from_ip": "10.2.3.40", "app": "bazfoo", "amount": 22.95,
         "extras":{"visitors":{"user_data":null}}}'
    count => 1
  }
}

filter {
  json {
    source => "message"
  }

  if ! [extras][visitors][user_data] {
    mutate {
      add_tag => ["user_data-empty"]
    }
  }
}

output {
  stdout {
    codec => rubydebug {metadata => true}
  }
}

```

The if conditional is a kind of `not exists` test when done like above.

```auto
{
           "app" => "bazfoo",
      "sequence" => 0,
        "amount" => 22.95,
    "@timestamp" => 2017-12-22T10:26:37.214Z,
          "host" => "Elastics-MacBook-Pro.local",
      "@version" => "1",
        "extras" => {
        "visitors" => {
            "user_data" => nil
        }
    },
       "message" => "{\"from_ip\": \"10.2.3.40\", \"app\": \"bazfoo\", \"amount\": 22.95, \"extras\":{\"visitors\":{\"user_data\":null}}}",
          "tags" => [
        [0] "user_data-empty"
    ],
       "from_ip" => "10.2.3.40"
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 19, 2018, 10:30am UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-mapping/112903/3 "2018-01-19T10:30:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.