Hi all,
I want to insert chats into Elasticsearch and have a problem in converting the chat XML transcript to the proper JSON using logstash.
This is the XML:
<?xml version="1.0" encoding="UTF-8"?> This is just a text from A This is a text from B This is a text from C This is a reply from A This is another text from CThis is my basic starting Logstash config to test with:
input { stdin {
}
}
filter {
xml {
source => "message"
target => "doc"
store_xml => true
force_array => false
remove_namespaces =>true
}
mutate
{
remove_field => [ "message","host" ]
}
}
output { stdout { codec => rubydebug } }
This is the output of the JSON:
{
"@timestamp" => 2018-11-02T15:31:14.901Z,
"@version" => "1",
"doc" => {
"message" => [
[0] {
"userId" => "123",
"msgText" => "This is just a text from A"
},
[1] {
"userId" => "456",
"msgText" => "This is a text from B"
},
[2] {
"userId" => "789",
"msgText" => "This is a text from C"
},
[3] {
"userId" => "123",
"msgText" => "This is a reply from A"
},
[4] {
"userId" => "789",
"msgText" => "This is another text from C"
}
],
"newParty" => [
[0] {
"userInfo" => {
"userNick" => "A",
"userType" => "CLIENT"
},
"userId" => "123"
},
[1] {
"userInfo" => {
"userNick" => "B",
"userType" => "AGENT"
},
"userId" => "456"
},
[2] {
"userInfo" => {
"userNick" => "C",
"userType" => "AGENT"
},
"userId" => "789"
}
]
}
}
What I want to achieve is, that in the message fields there isn't the userID, but the userNickName being stored to ElasticSearch.
How could I match those fields together?
Thanks in advance.
Regards,
Christian