# XML filter on Logstash

**URL:** https://discuss.elastic.co/t/xml-filter-on-logstash/143174
**Category:** Logstash
**Created:** [August 6, 2018, 1:28pm UTC](https://discuss.elastic.co/t/xml-filter-on-logstash/143174 "2018-08-06T13:28:09Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Fram\_Souza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fram_souza/32/24833_2.png) [@Fram\_Souza](https://discuss.elastic.co/u/Fram_Souza)
#### Post date: [August 6, 2018, 1:28pm UTC](https://discuss.elastic.co/t/xml-filter-on-logstash/143174/1 "2018-08-06T13:28:10Z")

</div>

I'm indexing xml on Elasticsearch using `xml filter`, below a simple example (a original xml is very large):

`<?xml version="1.0" encoding="ISO-8859-1"?><FATURA><DADOS_CADASTRAIS><CLIENTE Nome="john" A_C="tex" Id_Cliente="001" CPF_CNPJ="919191"></CLIENTE></DADOS_CADASTRAIS></FATURA?`

My input/filter is logstash is:

input {  
s3 {  
"bucket" =\> "xx"  
}  
}

filter {  
xml {  
source =\> "message" store\_xml =\> true target =\> "theXML" force\_array =\> false  
remove\_field =\> ["message"] }  
}

When I indexing xml on encoding ISO-8859-1, I receveid this error:

`/NF></NOTAS_FISCAIS></FATURA>", :exception=>#<REXML::ParseException: missing attribute quote Line: 1 Position: 98173 Last 80 unconsumed characters: <CLIENTE Nome=\"john\" A_C=\"x\" Id_C>, :backtrace=>["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rexml/parsers/baseparser.rb:374:in`pull\_event'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rexml/parsers/baseparser.rb:185:in `pull'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rexml/parsers/treeparser.rb:23:in`parse'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rexml/document.rb:288:in `build'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rexml/document.rb:45:in`initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/xml-simple-1.1.5/lib/xmlsimple.rb:971:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/xml-simple-1.1.5/lib/xmlsimple.rb:164:in`xml\_in'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/xml-simple-1.1.5/lib/xmlsimple.rb:203:in `xml_in'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-xml-4.0.5/lib/logstash/filters/xml.rb:182:in`filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164:in`block in multi\_filter'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161:in`multi\_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filter\_delegator.rb:47:in `multi_filter'", "(eval):69:in`block in filter\_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:445:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:424:in`worker\_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:386:in `block in start_workers'"]}`

BUT! When I change manualy a encondig to UTF-8 I don't received error and XML is indexed the way I need it.

I try add filter mutate to change, but, "encondig" is change, but I receveid same error, like this: 😕  
`filter { mutate { gsub => ["message", "ISO-8859-1", "UTF-8"] } }`

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 6, 2018, 1:30pm UTC](https://discuss.elastic.co/t/xml-filter-on-logstash/143174/2 "2018-08-06T13:30:02Z")

</div>

The mutate does come before the xml filter, right?

---

<div class="post-metadata">

### Author: ![Fram\_Souza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fram_souza/32/24833_2.png) [@Fram\_Souza](https://discuss.elastic.co/u/Fram_Souza)
#### Post date: [August 6, 2018, 1:32pm UTC](https://discuss.elastic.co/t/xml-filter-on-logstash/143174/3 "2018-08-06T13:32:15Z")

</div>

Yes @Badger

Like this:

```auto
  s3 {
    "bucket" => "x"
}
}

filter {
  mutate { gsub => ["message", "ISO-8859-1", "UTF-8"] }
}

filter {
  xml {
      source => "message" store_xml => true target => "theXML" force_array => false
        remove_field => ["message"] }
}

I see a change in a field, but I received error in Logstash
```

---

<div class="post-metadata">

### Author: ![Fram\_Souza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fram_souza/32/24833_2.png) [@Fram\_Souza](https://discuss.elastic.co/u/Fram_Souza)
#### Post date: [August 6, 2018, 1:52pm UTC](https://discuss.elastic.co/t/xml-filter-on-logstash/143174/4 "2018-08-06T13:52:37Z")

</div>

I solved this add codec on input 😉

Thanks

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 3, 2018, 1:52pm UTC](https://discuss.elastic.co/t/xml-filter-on-logstash/143174/5 "2018-09-03T13:52:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.