# Logstash problems with xml filter

**URL:** <https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667>\
**Category:** Logstash\
**Created:** [February 18, 2021, 9:16am UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667 "2021-02-18T09:16:02Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Er1csson](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@Er1csson](https://discuss.elastic.co/u/Er1csson)\
**Post date:** [February 18, 2021, 9:16am UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/1 "2021-02-18T09:16:02Z")

</div>

Hi all,  
I want to collect only certain tags from my xml file (data with Parameter tag). Here is the example of xml file I want to parse:

```
// <Instrument Name="GLPO" DisplayName="AAAA" HeartBeat="BBBB">
// <Component Name="AutoCtf" DisplayName="AutoCtf" ServiceCategory="None">
// <Parameter ID="495" EventID="497" Name="Defocus" />
 // <Parameter ID="496" EventID="497" Name="Astigmatism" />
// <Parameter ID="497" EventID="497" Name="AstigmatismOrientation" />
 // </Component>
 // </Instrument>

```

` The problem is, beside right data, logstash index tags which does not have right strings. Here is what I get when run logstash:`

```
//{
// "@version" => "1",
// "event_id" => [
// [0] "498"
// ],
// "host" => "NLEIN-GZCVWZ1",
// "type" => "healthmonitoring",
// "health_id" => [
// [0] "512"
// ],
// "@timestamp" => 2021-02-18T09:04:20.528Z,
// "path" => //"C:/Users/aleksei.poliakov/Desktop/Internship/Logs/HealthMonitorCmd_20200817_153946.xml",
// "message" => " <Parameter ID=\"512\" EventID=\"498\" Name=\"Iteration\" //DisplayName=\"Iteration\" Type=\"Int\" StorageUnit=\"\" DisplayUnit=\"\" DisplayScale=\"\" //FormatString=\"\" ServiceCategory=\"None\" MaxLogInterval=\"00:00:00\" //AbsoluteMinimum=\"-1.7976931348623157E+308\" //AbsoluteMaximum=\"1.7976931348623157E+308\" />\r"
//}
//{
// "@version" => "1",
// "host" => "NLEIN-GZCVWZ1",
// "type" => "healthmonitoring",
// "@timestamp" => 2021-02-18T09:04:20.528Z,
// "path" => //"C:/Users/aleksei.poliakov/Desktop/Internship/Logs/HealthMonitorCmd_20200817_153946.xml",
// "message" => " </Component>\r"
//}

```

` My config file:`

```
// input {
// file {
// path => ["C:/Users/aleksei.poliakov/Desktop/Internship/Logs/HealthMonitorCmd_20200817//_153946.xml"]
// start_position => "beginning"
// sincedb_path => "NUL"
// type => "healthmonitoring"
// exclude => "*.gz"
// }
//}
// filter {
// xml {
// store_xml => false
// source => "message"
// target => "Parameter"
// xpath => 
// [
// "//Parameter/@ID", "health_id",
// "//Parameter/@EventID", "event_id"
// ]
// }
//}
// output {
// if [type] == "healthmonitoring" {
// elasticsearch {
// hosts => ["localhost:9200"]
// index => "health-monitoring-%{+DDMMYYYY}"
// }
// }
// stdout { }
//}

Thank you in advance!
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 18, 2021, 6:40pm UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/2 "2021-02-18T18:40:28Z")

</div>

With that data and that filter I get

```
 "health_id" => [
    [0] "495",
    [1] "496",
    [2] "497"
],
  "event_id" => [
    [0] "497",
    [1] "497",
    [2] "497"
],

```

So I do not think you are doing what you think you are doing.

---

<div class="post-metadata">

**Author:** ![Er1csson](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@Er1csson](https://discuss.elastic.co/u/Er1csson)\
**Post date:** [February 19, 2021, 10:17am UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/3 "2021-02-19T10:17:13Z")

</div>

Hi Badger,  
Thank you for reply.  
How do your logs look like? If you check my post you can see that some logs do not have data I interested for. I even tried to use different computer, the result is the same.

P.S. I'm new at this so I don't really know if it is normal or not.

I want to get something like this:

> ```
> {
> "health_id" = "495",
> "event_id" = "497"
> }
> {
> "health_id" = "496",
> "event_id" = "497"
> }
> {
> "health_id" = "497",
> "event_id" = "497"
> }
> 
> ```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 19, 2021, 6:15pm UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/4 "2021-02-19T18:15:30Z")

</div>

> [@Er1csson](#):
>
> How do your logs look like?

I used

```
 <Instrument Name="GLPO" DisplayName="AAAA" HeartBeat="BBBB">
  <Component Name="AutoCtf" DisplayName="AutoCtf" ServiceCategory="None">
    <Parameter ID="495" EventID="497" Name="Defocus" />
    <Parameter ID="496" EventID="497" Name="Astigmatism" />
    <Parameter ID="497" EventID="497" Name="AstigmatismOrientation" />
  </Component>
</Instrument>

```

---

<div class="post-metadata">

**Author:** ![Er1csson](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@Er1csson](https://discuss.elastic.co/u/Er1csson)\
**Post date:** [February 19, 2021, 8:50pm UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/5 "2021-02-19T20:50:40Z")

</div>

Sorry, I meant indexes. How do they look like?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 19, 2021, 9:04pm UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/6 "2021-02-19T21:04:09Z")

</div>

I do not have indexes. I do not run elasticsearch.

---

<div class="post-metadata">

**Author:** ![Er1csson](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@Er1csson](https://discuss.elastic.co/u/Er1csson)\
**Post date:** [February 19, 2021, 9:34pm UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/7 "2021-02-19T21:34:47Z")

</div>

Can you show me your config file in case you used different one? Because I get completely different output:

```
{
      "@version" => "1",
      "event_id" => [
       [0] "498"
    ],
      "type" => "healthmonitoring",
      "health_id" => [
        [0] "512"
    ],
    "@timestamp" => 2021-02-18T09:04:20.528Z,
}

{
      "@version" => "1",
      "type" => "healthmonitoring",
      "@timestamp" => 2021-02-18T09:04:20.528Z,
}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 19, 2021, 10:15pm UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/8 "2021-02-19T22:15:59Z")

</div>

Well you would, since your [message] field is

"message" =\> " \<Parameter ID="512" EventID="498" Name="Iteration" //DisplayName="Iteration" Type="Int" StorageUnit="" DisplayUnit="" DisplayScale="" //FormatString="" ServiceCategory="None" MaxLogInterval="00:00:00" //AbsoluteMinimum="-1.7976931348623157E+308" //AbsoluteMaximum="1.7976931348623157E+308" /\>\r"

That matches the health\_id and event\_id in your event.

I wonder if your problem is that you are consuming the XML line by line. You may need a multiline codec. Read [this](https://discuss.elastic.co/t/ingest-xml-file-using-logstash/255113/4) post and the posts linked to.

---

<div class="post-metadata">

**Author:** ![Er1csson](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@Er1csson](https://discuss.elastic.co/u/Er1csson)\
**Post date:** [February 19, 2021, 11:15pm UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/9 "2021-02-19T23:15:26Z")

</div>

Thank you, I will check that post. Somehow, I thought that xml filter can do that by default. What the point in xml filter if you can do literally the same with grok filter?!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 19, 2021, 11:15pm UTC](https://discuss.elastic.co/t/logstash-problems-with-xml-filter/264667/10 "2021-03-19T23:15:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.