Mutiple xml file with same fields (How to handle?) any suggestion


(Saket Kumar) #1

xml {
xpath => [

"/response/data/median/firstView/visualComplete/text()","FV_visualComplete",
"/response/data/median/firstView/lastVisualChange/text()","FV_lastVisualChange",
"/response/data/median/firstView/loadTime/text()","FV_loadTime",
"/response/data/median/firstView/fullyLoaded/text()","FV_fullyLoaded",
"/response/data/median/firstView/SpeedIndex/text()","FV_SpeedIndex",
"/response/data/median/firstView/TTFB/text()","FV_TTFB",
"/response/data/median/firstView/titleTime/text()","FV_titleTime",
] }

I have multiple XML file with above fields. When Logstash parses them, last file's value gets reflected in above fields.

Is there any way to handle?


(Saket Kumar) #2

Is possible to parse multiple XML file containing same fields to parse at once using logstash? any help will save my day.


(Mark Walkom) #3

Each file should be processed as a new message, but it sounds like LS things they are the same message and is just updating them.

Can you show us your config?


(Saket Kumar) #4

Yes i found a way. Now using below config file i am able to get them as a new message.
input {
file {
path => "/opt/Log/WebPage10/RUN*/*_XML_WebpageSummary.xml"
start_position => "beginning"
}
}

filter {
if [message] =~ "^<?xml .*" {
drop {}
}
multiline {
pattern => "^</response>"
negate => true
what => "next"
}

xml {
source => "message"
target => "videoFrames"
store_xml => false
xpath => ["/response/data/median/firstView/visualComplete/text()","visualComplete"]
xpath => ["/response/data/run/firstView/videoFrames/frame[]/time/text()","Time_FV"]
xpath => ["/response/data/run/repeatView/videoFrames//frame[
]/time/text()","Time_RV"]
}

Now I am struggling with "Time_FV" & "Time_RV" array fields. Each new message contains these two fields. I have to find the length of array(message which has this array field of maximum length) to split respective field. Any help?
Thanks!


(system) #5