# Using logstash

**URL:** <https://discuss.elastic.co/t/using-logstash/32454>\
**Category:** Logstash\
**Created:** [October 19, 2015, 6:21am UTC](https://discuss.elastic.co/t/using-logstash/32454 "2015-10-19T06:21:18Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![fereshteh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fereshteh/32/5926_2.png) [@fereshteh](https://discuss.elastic.co/u/fereshteh)\
**Post date:** [October 19, 2015, 6:21am UTC](https://discuss.elastic.co/t/using-logstash/32454/1 "2015-10-19T06:21:18Z")

</div>

I am trying to use the logstash documentation to set up a xml filter, but I just cant seem to get it right.

My XML format is pretty straigth forward;

---

<div class="post-metadata">

**Author:** ![PhaedrusTheGreek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phaedrusthegreek/32/4884_2.png) [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)\
**Post date:** [October 19, 2015, 1:57pm UTC](https://discuss.elastic.co/t/using-logstash/32454/2 "2015-10-19T13:57:21Z")

</div>

@fereshteh,

XML parsing should work easily, even with multiline. Consider the following input (xml.log):

```auto
<html lang="en" class="sample">
  <head mycompany="http://url.me/profile#">
    <meta charset='utf-8'>
       <mytag mykey='myval'/>
    </meta>
  </head>
</html>

<html lang="en" class="sample">
  <head mycompany="http://url.me/profile#">
    <meta charset='utf-8'>
       <mytag mykey2='myval2'/>
    </meta>
  </head>
</html>

```

And this Logstash Config:

```auto
input
{
        file
        {
                path => "/sample-inputs/xml.log"
                sincedb_path => "/dev/null"
                start_position => "beginning"
        }
}

filter
{
        multiline
        {
                pattern => "^<html"
                negate => true
                what => previous
        }
        xml
        {
                source => ["message"]
                target => ["x"]
        }
}

output
{
        stdout
        {
                codec => rubydebug
        }
}

```

Produces:

```auto
{
       "message" => "<html lang=\"en\" class=\"sample\">\n <head mycompany=\"http://url.me/profile#\">\n <meta charset='utf-8'>\n <mytag mykey='myval'/> \n </meta>\n </head>\n</html>\n",
      "@version" => "1",
    "@timestamp" => "2015-10-19T13:54:45.774Z",
          "tags" => [
        [0] "multiline"
    ],
             "x" => {
         "lang" => "en",
        "class" => "sample",
         "head" => [
            [0] {
                "mycompany" => "http://url.me/profile#",
                     "meta" => [
                    [0] {
                        "charset" => "utf-8",
                          "mytag" => [
                            [0] {
                                "mykey" => "myval"
                            }
                        ]
                    }
                ]
            }
        ]
    }
}
{
       "message" => "<html lang=\"en\" class=\"sample\">\n <head mycompany=\"http://url.me/profile#\">\n <meta charset='utf-8'>\n <mytag mykey2='myval2'/> \n </meta>\n </head>\n</html>",
      "@version" => "1",
    "@timestamp" => "2015-10-19T13:54:45.777Z",
          "tags" => [
        [0] "multiline"
    ],
             "x" => {
         "lang" => "en",
        "class" => "sample",
         "head" => [
            [0] {
                "mycompany" => "http://url.me/profile#",
                     "meta" => [
                    [0] {
                        "charset" => "utf-8",
                          "mytag" => [
                            [0] {
                                "mykey2" => "myval2"
                            }
                        ]
                    }
                ]
            }
        ]
    }
}

```

---

<div class="post-metadata">

**Author:** ![PhaedrusTheGreek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phaedrusthegreek/32/4884_2.png) [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)\
**Post date:** [October 20, 2015, 7:06pm UTC](https://discuss.elastic.co/t/using-logstash/32454/4 "2015-10-20T19:06:39Z")

</div>

@fereshteh,

You can do that too. If you want to parse out the attributes by [XPath](http://www.tizag.com/xmlTutorial/xpathattribute.php), do it like this:

```auto
 xml
        {
                store_xml => false
                source => ["message"]
                xpath => [
                        "/html/@lang","lang",
                        "/html/@class","class",
                ]
        }

```

Yields:

```auto
{
  "lang" => [
        [0] "en"
    ],
  "class" => [
        [0] "sample"
    ]
}

```

Note that the attributes come out as arrays, because there can be multiple instances of the node specified by the path. If you like, you could do something like this to convert them to strings.

```auto
mutate
{
         join => { "lang" => "," }
         join => { "class" => "," }
}

```

Yields:

```auto
{
         "lang" => "en",
         "class" => "sample"
}

```

HTH

---

<div class="post-metadata">

**Author:** ![PhaedrusTheGreek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phaedrusthegreek/32/4884_2.png) [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)\
**Post date:** [October 26, 2015, 6:10pm UTC](https://discuss.elastic.co/t/using-logstash/32454/6 "2015-10-26T18:10:57Z")

</div>

The multiline filter requires that the html tag is at the beginning of the line - make sure your input file is exactly as mine is.

Let me know how it that helps!

Jay

---

<div class="post-metadata">

**Author:** ![PhaedrusTheGreek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phaedrusthegreek/32/4884_2.png) [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)\
**Post date:** [October 27, 2015, 4:58pm UTC](https://discuss.elastic.co/t/using-logstash/32454/8 "2015-10-27T16:58:25Z")

</div>

In windows, use stdin input instead, and redirect the input of xml.log:

```auto
input {  
 stdin { }  
}

```

```auto
bin\logstash -f test.cfg < xml.log

```

I have not tested this, but it should work.

---

<div class="post-metadata">

**Author:** ![PhaedrusTheGreek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phaedrusthegreek/32/4884_2.png) [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)\
**Post date:** [November 4, 2015, 1:49pm UTC](https://discuss.elastic.co/t/using-logstash/32454/10 "2015-11-04T13:49:20Z")

</div>

@fereshteh,

In your initial question, the XML was formatted on multiple lines, so we used the multiline filter. Is your expected input always on a single line? Or do you still expect to see events over multiple lines?

---

<div class="post-metadata">

**Author:** ![PhaedrusTheGreek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phaedrusthegreek/32/4884_2.png) [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)\
**Post date:** [November 4, 2015, 2:07pm UTC](https://discuss.elastic.co/t/using-logstash/32454/11 "2015-11-04T14:07:41Z")

</div>

You can use the following to split your single line XML into multiple lines:

```auto
 mutate {
                        gsub => ["message", "</html><html", "</html>
<html"]
                }

                split {
                }

                xml {
                        source => ["message"]
                        target => ["x"]
                }

```

---

<div class="post-metadata">

**Author:** ![PhaedrusTheGreek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phaedrusthegreek/32/4884_2.png) [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)\
**Post date:** [November 10, 2015, 3:17pm UTC](https://discuss.elastic.co/t/using-logstash/32454/13 "2015-11-10T15:17:37Z")

</div>

@fereshteh,

In that case, you need to first extract the XML from the rest of the message using GROK like so:

```auto
filter
{
        grok {
                match => { "message" => ".*(?<the_xml><html.*</html>).*" }        
        }
        xml {
                store_xml => false
                source => ["the_xml"]
                xpath => [
                        "/html/@lang","lang",
                        "/html/@class","class"
                ]
        }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:22am UTC](https://discuss.elastic.co/t/using-logstash/32454/15 "2017-07-06T05:22:43Z")

</div>


