# Xml filter does not work

**URL:** <https://discuss.elastic.co/t/xml-filter-does-not-work/166047>\
**Category:** Logstash\
**Created:** [January 28, 2019, 8:27pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047 "2019-01-28T20:27:59Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Necus](https://avatars.discourse-cdn.com/v4/letter/n/e47774/32.png) [@Necus](https://discuss.elastic.co/u/Necus)\
**Post date:** [January 28, 2019, 8:27pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/1 "2019-01-28T20:27:59Z")

</div>

Hi,  
Do you have any idea why this xml filter does not work? 🙂

filter{  
if [type] == "abc" {  
dissect {  
mapping =\> {  
message =\> '\<%{abc}\>%{syslog\_timestamp} %{+syslog\_timestamp} %{+syslog\_timestamp} %{logsource} %{rest}'  
}  
}  
if[rest] =~ "123" {  
xml {  
source =\> "rest"  
target =\> "rest"  
xpath =\>  
["/1/2/3/333/text()", "333"  
]  
}  
}  
}  
}

XML:  
\<1\>  
\<2 ProductName="" ProductVersion="" ProductFamily=""\>  
\<3\>  
\<333\>19136\</333\>  
\</3\>  
\</2\>  
\</1\>

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 28, 2019, 9:33pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/3 "2019-01-28T21:33:07Z")

</div>

It does not work because it is not valid XML. Element names must start with underscore or a letter..

```
input { generator { count => 1 message => '' } }
filter {
    mutate { add_field => { "rest" => '<a1> <a2 ProductName="" ProductVersion="" ProductFamily=""> <a3> <a333>19136</a333> </a3> </a2> </a1>' } }
    xml {
        source => "rest"
        target => "rest"
        xpath => { "/a1/a2/a3/a333/text()" => "333" }
    }
}

```

works just fine

```
       "333" => [
    [0] "19136"
]
```

---

<div class="post-metadata">

**Author:** ![Necus](https://avatars.discourse-cdn.com/v4/letter/n/e47774/32.png) [@Necus](https://discuss.elastic.co/u/Necus)\
**Post date:** [January 29, 2019, 7:47am UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/4 "2019-01-29T07:47:24Z")

</div>

Thank you, you are obviously right it was my mistake.

The exact fields below:

\<?xml version="1.0" encoding="UTF-8"?\>

After simple xml {} without xpath everty kibana shows right fields (with start rest.EventList and rest.MachineInfo) but after using xpath i do not get enything in newly created field, its empty

So, for some reason any of below does not work:  
xpath =\> ["/Main/MachineInfo/MachineName/text()"]  
xpath =\> ["/rest.MachineInfo/MachineName/text()"]  
any idea?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 29, 2019, 1:51pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/5 "2019-01-29T13:51:41Z")

</div>

> [@Necus](#):
>
> xpath =\> ["/Main/MachineInfo/MachineName/text()"]  
> xpath =\> ["/rest.MachineInfo/MachineName/text()"]

That's not the right syntax. You can only have one xpath option, and it takes pairs of values, either as a hash or an array.

---

<div class="post-metadata">

**Author:** ![Necus](https://avatars.discourse-cdn.com/v4/letter/n/e47774/32.png) [@Necus](https://discuss.elastic.co/u/Necus)\
**Post date:** [January 30, 2019, 8:27am UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/6 "2019-01-30T08:27:12Z")

</div>

Hi,  
No matter what I will choose, whether  
xpath =\> ["/Main/MachineInfo/MachineName/text()", "MachineName"]  
or  
xpath =\> ["//MachineName/text()", "MachineName"]  
the field is empty, any idea why?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 30, 2019, 1:13pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/7 "2019-01-30T13:13:55Z")

</div>

Not without seeing the actual XML, no.

---

<div class="post-metadata">

**Author:** ![Necus](https://avatars.discourse-cdn.com/v4/letter/n/e47774/32.png) [@Necus](https://discuss.elastic.co/u/Necus)\
**Post date:** [January 30, 2019, 7:51pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/8 "2019-01-30T19:51:22Z")

</div>

```
<?xml version="1.0" encoding="UTF-8"?><Main><MachineInfo><MachineName></MachineName><AgentGUID></AgentGUID><IPAddress></IPAddress><OSName></OSName><UserName></UserName><TimeZoneBias></TimeZoneBias><RawMACAddress></RawMACAddress></MachineInfo><EventList ProductName="" ProductVersion="" ProductFamily=""><Event><EventID></EventID><Severity></Severity><GMTTime></GMTTime><OPGData></OPGData><UserInfo></UserInfo><ThreatName></ThreatName><PolicyName></PolicyName><TimeSZone></TimeSZone></Event></EventList></Main>

```

I hope that this will help, thank you in advance

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 30, 2019, 8:20pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/9 "2019-01-30T20:20:02Z")

</div>

Well, for that XML,

```
        xpath => ["//MachineName/text()" , "MachineName"]

```

produces

```
"MachineName" => [],

```

If you add some text to the MachineName element it will extract it

```
 "MachineName" => [
    [0] "Foo"
]

```

It is an array, always an array, so references to MachineName may not do what you expect.

---

<div class="post-metadata">

**Author:** ![Necus](https://avatars.discourse-cdn.com/v4/letter/n/e47774/32.png) [@Necus](https://discuss.elastic.co/u/Necus)\
**Post date:** [January 30, 2019, 8:25pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/10 "2019-01-30T20:25:42Z")

</div>

I have tried that and nothing, field is empty. To be honest, no matter which field I will choose, all of them appear to be empty after xpath.  
Is there some way to debug this?  
Like I said earlier xml filter works fine, there is something wrong with xpath.

One more thing  
XML splitted output into 3 main fields:  
rest2.EventList and there rest  
rest2.EventList.Event.GMTTime  
and rest2.MachineInfo

From Kibana:  
rest2.MachineInfo {  
"TimeZoneBias": [  
"-60"  
],  
"AgentGUID": [  
"{dhkkjkh}"  
],  
"IPAddress": [  
"x.x.x.x"  
],  
"RawMACAddress": [  
"abc"  
],  
"OSName": [  
"Windows 10 Workstation"  
],  
"UserName": [  
"SYSTEM"  
],  
"MachineName": [  
"xxx"  
]  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 30, 2019, 9:00pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/11 "2019-01-30T21:00:19Z")

</div>

Clearly the example XML you posted is not the XML you are working on (no rest. anywhere). I understand you may be unable to post proprietary information but it is making impossible for me to see what the problem might be.

---

<div class="post-metadata">

**Author:** ![Necus](https://avatars.discourse-cdn.com/v4/letter/n/e47774/32.png) [@Necus](https://discuss.elastic.co/u/Necus)\
**Post date:** [January 31, 2019, 7:49pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/12 "2019-01-31T19:49:03Z")

</div>

Current config:  
filter{  
if [type] == "abc" {  
dissect {  
mapping =\> {  
message =\> '\<%{abc}\>%{syslog\_timestamp} %{+syslog\_timestamp} %{+syslog\_timestamp} %{logsource} %{rest}'  
}  
}  
if[rest] =~ "123" {  
xml {  
source =\> "rest"  
target =\> "rest2"  
xpath =\> {"//Main/rest2.MachineInfo/MachineName/text()" =\> "MachineName"}  
}  
}  
}  
}

xpath option is my variable, I have tried many, and field is still empty.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 31, 2019, 8:06pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/13 "2019-01-31T20:06:25Z")

</div>

If your syslog timestamps look like "Feb 1 01:02:03" with two spaces between Feb and 1 then this will break tomorrow. You need to tell dissect that there are extra delimiters used for visual alignment. The -\> after syslog\_timestamp does that.

```
message => '<%{abc}>%{syslog_timestamp->} %{+syslog_timestamp} %{+syslog_timestamp} %{logsource} %{rest}'

```

You should not add rest2. to the xpath expression. I don't know what else to say. Given the XML format you show, if, and only if, there is a value in the MachineName element, then

```
        xpath => [
            "/Main/MachineInfo/MachineName/text()", "MachineName"
        ]

```

_will_ extract it. I have to infer that the XML format you show is not actually what you are using.

---

<div class="post-metadata">

**Author:** ![Necus](https://avatars.discourse-cdn.com/v4/letter/n/e47774/32.png) [@Necus](https://discuss.elastic.co/u/Necus)\
**Post date:** [January 31, 2019, 8:45pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/14 "2019-01-31T20:45:54Z")

</div>

Thank you for all your help, for some reason this xpath does not want to work. Grok works just fine, I will stick with it for this kind of logs.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 28, 2019, 8:46pm UTC](https://discuss.elastic.co/t/xml-filter-does-not-work/166047/15 "2019-02-28T20:46:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.