# XML filter causing ES mapping issues

**URL:** <https://discuss.elastic.co/t/xml-filter-causing-es-mapping-issues/126367>\
**Category:** Logstash\
**Created:** [April 2, 2018, 1:55am UTC](https://discuss.elastic.co/t/xml-filter-causing-es-mapping-issues/126367 "2018-04-02T01:55:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [April 2, 2018, 1:55am UTC](https://discuss.elastic.co/t/xml-filter-causing-es-mapping-issues/126367/1 "2018-04-02T01:55:24Z")

</div>

At least, I think that is the problem.

I am getting a lot of rejected docs from Elasticsearch, and when I look at the reason in the dead letter queue file, it says "Can't get text on a START\_OBJECT". The only thing I'm doing using the XML filter and then sending to Elasticsearch.

I think what is happening is that at one point in my XML documents, some of them have one level of a `text` element:

```auto
<text>Here is some text.</text>

```

while others have nested levels:

```auto
<text>Here is some text.
  <text> Here is some more text.</text>
</text>

```

I think that this means the first file's output will cause Elasticsearch to set the mapping for `text` to be `text` and then on later docs it will be an object.

Is that correct? If so, how can I handle this issue?

Thank you!

---

<div class="post-metadata">

**Author:** ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)\
**Post date:** [April 2, 2018, 2:04am UTC](https://discuss.elastic.co/t/xml-filter-causing-es-mapping-issues/126367/2 "2018-04-02T02:04:34Z")

</div>

What's your pipeline config?

---

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [April 2, 2018, 2:16am UTC](https://discuss.elastic.co/t/xml-filter-causing-es-mapping-issues/126367/3 "2018-04-02T02:16:57Z")

</div>

```
input{
  s3 {
    bucket => "mybucketname"
    access_key_id => "removed"
    secret_access_key => "removed"
    exclude_pattern => "^((?!XML$).)*$"
    region => "us-east-2"
    sincedb_path => "/etc/logstash/conf.d/.sincedb_files"
    codec => multiline {
      pattern => "<rootElement>"
      negate => true
      what => "previous"
      max_lines => 10000
      max_bytes => "100 MiB"
    }
  }
}

filter {
  xml {
    source => "message"
    target => "message"
    force_array => false
  }
}

output {
  stdout { codec => rubydebug }
  elasticsearch {
    hosts => "removed"
    index => "index_pattern-%{+YYYY.MM.dd}"
  }
}
```

---

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [April 3, 2018, 12:29pm UTC](https://discuss.elastic.co/t/xml-filter-causing-es-mapping-issues/126367/4 "2018-04-03T12:29:12Z")

</div>

Anyone have any ideas? I would imagine this would be a common issue if I'm correct about what's happening. No idea how to handle it though.

---

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [April 15, 2018, 10:44pm UTC](https://discuss.elastic.co/t/xml-filter-causing-es-mapping-issues/126367/5 "2018-04-15T22:44:05Z")

</div>

For anyone else experiencing this: I stopped trying to parse the entire XML file and just went with selecting each section of it with xpaths.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 13, 2018, 10:44pm UTC](https://discuss.elastic.co/t/xml-filter-causing-es-mapping-issues/126367/6 "2018-05-13T22:44:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.