Xml plugin getting text with characters


I'm getting the XML values from a QUEUE and when i get an element that have the '&' character it transforms into the &amp string.

Here are the before and after:



Here are the code of the filter, the field in question is the service_key:

source => "message"
store_xml => false
remove_namespaces => true
force_array => false
xpath => [
"/LogMessage/TransactionID/text()", "transaction_id",
"/LogMessage/ServiceKey/text()", "service_key"

Any ideas why this happens and how can i resolve this?


Strange the field after passing to the interpreter of the browser, is correct.
The second value instead of a '&' it has the '&amp'.
This field is used after in a jdbc_streaming plugin and it has the wrong value.

I do not see any difference between the before and after. You may need to use appropriate markdown to quote your examples.

This is what i get from the XML:

            "service_key" => "gz$LDQ@vnbkj7DNUVwwjrnGa0&|2,a)",

and instead of & it should be '&'

You could always setup a mutate pass on the field and use gsub to replace &amp with &

mutate {
  gsub => [
  "service_key", "&amp", "&"


The problem using that is that i can't predict that's the only character that needs to be treated.
There's no other way of doing something in the XML filter plugin?

The source of the problem is the parsing engine the XML plugin uses and it's mistreatment of a special character. You can safely assuming that alphanumeric characters wont have this occur, and based on your current results, $@ are safe as well. All that in mind, unless we get some input from the plugin dev (open a ticket on github?), you could perform a test ingest of each special character and then add to the mutate rule for those that are improperly parsed.

If you have HTML entities in a string then there is a third-party filter that can remove them.

The solution wasn't pretty, but solves the problem.

What was your solution, a mutate filter?

The solution that you provided in the first place, a mutate filter with a lot of characters to be gsubed.


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.