Escaped JSON in XML

[ChristianCH]

Hi all,

I have an escaped JSON in XML structure, so the XML looks like this:

<?xml version="1.0"?>
<root>
	<test>test</test>
	<test>test</test>
	<test>
		<event>
			<item key="mainCat">
				<item key="subCat">
					<item key="content">{&quot;bid&quot;:&quot;extension&quot;,&quot;data&quot;:&quot;test&quot;}</item>
					<item key="metainfo">test</item>
				</item>
			</item>
		</event>
	</test>
</root>

I'm not able to parse this into pure JSON.

My test logstash config looks like this:

input { stdin {}}

filter {
	
	xml {
		source => "message"
		target => "doc"		
		store_xml => true
		force_array => false
	}	
	
	mutate
	{	
		remove_field => [ "message","host"]		
	}	
}

output { stdout { codec => json } }

When I use the XML filter, it outputs the escaped JSON:

{
	"@version" : "1",
	"@timestamp" : "2018-11-08T09:51:04.309Z",
	"doc" : {
		"test" : ["test", "test", {
				"event" : {
					"item" : {
						"item" : {
							"item" : [{
									"key" : "content",
									"content" : "{\"bid\":\"extension\",\"data\":\"test\"}"
								}, {
									"key" : "metainfo",
									"content" : "test"
								}
							],
							"key" : "subCat"
						},
						"key" : "mainCat"
					}
				}
			}
		]
	}
}

I tried to add a json source, but it complains about JSON parsing error then. Also a gsub replacing the backslash doesn't work.

Any idea how to get this converted into correct JSON?

Regards,
Christian

[wwalker (Walker)]

What's the gsub pattern you tried to use? I would think you could gsub the field removing the \ characters and then use the json filter to pull that data out into fields.

[ChristianCH]

Hi Walker,

yes, I had the same idea, but unfortunately it isn't working.

filter {
	
	xml {
		source => "message"
		target => "doc"		
		store_xml => true
		force_array => false
	}	
	    		
mutate {
	gsub => [ "doc", "[\\]", "" ]
}
	
	mutate
	{	
		remove_field => [ "message","host"]		
	}	
}

brings us to the same output.

These patterns don't work at all:

gsub => [ "doc", "\", "" ]
gsub => [ "doc", "[\]", "" ]
gsub => [ "doc", "\\", "" ]

Regards
Christian

Regards,
Christian