Xml filter does not work

[Necus]

Hi,
Do you have any idea why this xml filter does not work?

filter{
if [type] == "abc" {
dissect {
mapping => {
message => '<%{abc}>%{syslog_timestamp} %{+syslog_timestamp} %{+syslog_timestamp} %{logsource} %{rest}'
}
}
if[rest] =~ "123" {
xml {
source => "rest"
target => "rest"
xpath =>
["/1/2/3/333/text()", "333"
]
}
}
}
}

XML:
<1>
<2 ProductName="" ProductVersion="" ProductFamily="">
<3>
<333>19136</333>
</3>
</2>
</1>

[Badger]

It does not work because it is not valid XML. Element names must start with underscore or a letter..

input { generator { count => 1 message => '' } }
filter {
    mutate { add_field => { "rest" => '<a1> <a2 ProductName="" ProductVersion="" ProductFamily=""> <a3> <a333>19136</a333> </a3> </a2> </a1>' } }
    xml {
        source => "rest"
        target => "rest"
        xpath => { "/a1/a2/a3/a333/text()" => "333" }
    }
}

works just fine

       "333" => [
    [0] "19136"
]

[Necus]

Thank you, you are obviously right it was my mistake.

The exact fields below:

<?xml version="1.0" encoding="UTF-8"?>

After simple xml {} without xpath everty kibana shows right fields (with start rest.EventList and rest.MachineInfo) but after using xpath i do not get enything in newly created field, its empty

So, for some reason any of below does not work:
xpath => ["/Main/MachineInfo/MachineName/text()"]
xpath => ["/rest.MachineInfo/MachineName/text()"]
any idea?

[Badger]

@Necus

xpath => ["/Main/MachineInfo/MachineName/text()"]
xpath => ["/rest.MachineInfo/MachineName/text()"]

That's not the right syntax. You can only have one xpath option, and it takes pairs of values, either as a hash or an array.

[Necus]

Hi,
No matter what I will choose, whether
xpath => ["/Main/MachineInfo/MachineName/text()", "MachineName"]
or
xpath => ["//MachineName/text()", "MachineName"]
the field is empty, any idea why?

[Badger]

Not without seeing the actual XML, no.

[Necus]

<?xml version="1.0" encoding="UTF-8"?><Main><MachineInfo><MachineName></MachineName><AgentGUID></AgentGUID><IPAddress></IPAddress><OSName></OSName><UserName></UserName><TimeZoneBias></TimeZoneBias><RawMACAddress></RawMACAddress></MachineInfo><EventList ProductName="" ProductVersion="" ProductFamily=""><Event><EventID></EventID><Severity></Severity><GMTTime></GMTTime><OPGData></OPGData><UserInfo></UserInfo><ThreatName></ThreatName><PolicyName></PolicyName><TimeSZone></TimeSZone></Event></EventList></Main>

I hope that this will help, thank you in advance

[Badger]

Well, for that XML,

        xpath => [ "//MachineName/text()" , "MachineName" ]

produces

"MachineName" => [],

If you add some text to the MachineName element it will extract it

 "MachineName" => [
    [0] "Foo"
]

It is an array, always an array, so references to MachineName may not do what you expect.

[Necus]

I have tried that and nothing, field is empty. To be honest, no matter which field I will choose, all of them appear to be empty after xpath.
Is there some way to debug this?
Like I said earlier xml filter works fine, there is something wrong with xpath.

One more thing
XML splitted output into 3 main fields:
rest2.EventList and there rest
rest2.EventList.Event.GMTTime
and rest2.MachineInfo

From Kibana:
rest2.MachineInfo {
"TimeZoneBias": [
"-60"
],
"AgentGUID": [
"{dhkkjkh}"
],
"IPAddress": [
"x.x.x.x"
],
"RawMACAddress": [
"abc"
],
"OSName": [
"Windows 10 Workstation"
],
"UserName": [
"SYSTEM"
],
"MachineName": [
"xxx"
]
}

[Badger]

Clearly the example XML you posted is not the XML you are working on (no rest. anywhere). I understand you may be unable to post proprietary information but it is making impossible for me to see what the problem might be.

[Necus]

Current config:
filter{
if [type] == "abc" {
dissect {
mapping => {
message => '<%{abc}>%{syslog_timestamp} %{+syslog_timestamp} %{+syslog_timestamp} %{logsource} %{rest}'
}
}
if[rest] =~ "123" {
xml {
source => "rest"
target => "rest2"
xpath => {"//Main/rest2.MachineInfo/MachineName/text()" => "MachineName"}
}
}
}
}

xpath option is my variable, I have tried many, and field is still empty.

[Badger]

If your syslog timestamps look like "Feb 1 01:02:03" with two spaces between Feb and 1 then this will break tomorrow. You need to tell dissect that there are extra delimiters used for visual alignment. The -> after syslog_timestamp does that.

message => '<%{abc}>%{syslog_timestamp->} %{+syslog_timestamp} %{+syslog_timestamp} %{logsource} %{rest}'

You should not add rest2. to the xpath expression. I don't know what else to say. Given the XML format you show, if, and only if, there is a value in the MachineName element, then

        xpath => [
            "/Main/MachineInfo/MachineName/text()", "MachineName"
        ]

will extract it. I have to infer that the XML format you show is not actually what you are using.

[Necus]

Thank you for all your help, for some reason this xpath does not want to work. Grok works just fine, I will stick with it for this kind of logs.