I am receiving XML data which contains an array. The array can be cleaned up with gsub so it looks like a key=value pair, then parsed accordingly. My issue is that the pairs are in data:name order, so backwards for using the name as a field. After going through the xml filter, and gsub, the data winds up looking like:
"SMF"="EventName", "17.02.00"="EventVersion", "HCDEV0"="JobName", "HCI"="StepName", "S0626722"="JobJESID", "A"="JobClass", "000"="JobPriority", "CW09"="SystemID", "Y"="UsesOS/390", "00000000"="ProgramOffset", "00000000"="CSECTOffset", "//"="ProgramLinkDate", "000629A83907"="CPUID", "HCDEV0"="JobUserID", "10/12/2018"="ErrorOccurredAtDate", "14:20:11"="ErrorOccurredAtTime", "-0240"="UTCOffset", "2018"="ErrYear", "41"="ErrWeek", "10"="ErrMonth", "SA03"="ErrorCode1", "00000000"="ErrorCode2", "z/OS"="OperatingSystemName", "02.03.00"="OperatingSystemRelease", "HCIMMAIN"="ProgramName", "10.10.0.205"="PublisherID", "BATCH"="EventType", "17.02.00"="PublisherVersion", "Cfg\OS390_Fault.xml"="ODBC_DataMapName", "SMF"="PublisherName", "00000750"="CPUTime", "00000750"="JobCPUTime", "00644891"="ElapsedTime", "00644890"="JobElapsedTime", "D511911C-40D0-2812-DCBA-000629A83907"="EventID"
This backwards. Is there a relatively easy way to flip them?
What I tried to do before going down this path, was to use the xml filter initially, then xpath to the array'd node. What I wind up with though are two individual comma delimited arrays/strings. I have no idea how I can use Ruby to extract each "Name", use that for the field name, then assign the "Data" to it as its value. That bit of the config looks like:
xml {
source => "message"
target => "parsedxml"
force_array => "true"
force_content => "false"
store_xml => "true"
xpath => [
"//SMF/EventData/Field/@Name", "smfnamex",
"//SMF/EventData/Field/@Data", "smfdatax",
]
}
mutate {
replace => { "name" => "%{smfnamex}" }
replace => { "data" => "%{smfdatax}" }
}
which gives me:
for data:
SMF,17.02.00,HCDEV0,HCI,S0626722,A,000,CW09,Y,00000000,00000000,//,000629A83907,HCDEV0,10/12/2018,14:20:11,-0240,2018,41,10,SA03,00000000,z/OS,02.03.00,HCIMMAIN,10.10.0.205,BATCH,17.02.00,Cfg\OS390_Fault.xml,SMF,00000750,00000750,00644891,00644890,D511911C-40D0-2812-DCBA-000629A83907
and for name:
EventName,EventVersion,JobName,StepName,JobJESID,JobClass,JobPriority,SystemID,UsesOS/390,ProgramOffset,CSECTOffset,ProgramLinkDate,CPUID,JobUserID,ErrorOccurredAtDate,ErrorOccurredAtTime,UTCOffset,ErrYear,ErrWeek,ErrMonth,ErrorCode1,ErrorCode2,OperatingSystemName,OperatingSystemRelease,ProgramName,PublisherID,EventType,PublisherVersion,ODBC_DataMapName,PublisherName,CPUTime,JobCPUTime,ElapsedTime,JobElapsedTime,EventID
I also noticed that when I changed the source XML document to change the order of the two fields (Data and Name), they wind up coming out in the same order... i.e. changing their order in the source has no impact on the results after the xml parsing. It seems to be arbitrarily putting them in alphabetical order, so not sure what's going on there.
Any thoughts, suggestions?