# Logstash-XML parser

**URL:** https://discuss.elastic.co/t/logstash-xml-parser/121641
**Category:** Logstash
**Created:** [February 27, 2018, 11:29am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641 "2018-02-27T11:29:53Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![rattisyam](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rattisyam](https://discuss.elastic.co/u/rattisyam)
#### Post date: [February 27, 2018, 11:29am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/1 "2018-02-27T11:29:53Z")

</div>

Hi,  
I am new to ELK.  
I am trying to parse jenkins build.xml and collecting output using elastic search.  
I am using codec =\> multiline for the input to get it as a single event but i could not able to get tag in the message.

some one please help me on this. Thank you.

my input build.xml

\<?xml version='1.0' encoding='UTF-8'?\> rattisyam 1 8 refs/remotes/origin/master 0eae8a5c898e67eab676b5e95983ac11a20e7534 refs/remotes/origin/master 6 https://github.com/rattisyam/maven-project.git refs/remotes/origin/master /var/lib/jenkins/workspace/PackageJob 35 1517408055456 1517408055457 SUCCESS 10748 UTF-8 false /var/lib/jenkins/workspace/PackageJob 2.73.3 false 

logstash.conf file:  
input {  
file {  
path =\> "/var/lib/jenkins/jobs/PackageJob/builds/8/build.xml"  
sincedb\_path =\> "/dev/null"  
start\_position =\> "beginning"  
type =\> "buildxml"

```
    codec => multiline {
          pattern => "<?xml version='1.0' encoding='UTF-8'?>"
          negate => true
          what => "next"
          max_lines => 1000
        }
 }

```

}

filter {

}

output {  
elasticsearch {  
hosts =\> ["172.31.33.209:9200"]  
index =\> "buildxml1"  
document\_type =\> "demobuildxml"  
}

stdout { codec =\> rubydebug }

}

my output which i am getting now :

\<?xml version='1.0' encoding='UTF-8'?\> rattisyam 1 17 refs/remotes/origin/master 0eae8a5c898e67eab676b5e95983ac11a20e7534 refs/remotes/origin/master 8 https://github.com/rattisyam/maven-project.git refs/remotes/origin/master /var/lib/jenkins/workspace/PackageJob 39 1517409456533 1517409456534 SUCCESS 10641 UTF-8 false /var/lib/jenkins/workspace/PackageJob 2.73.3 false 

--here end tag is missing so that i canot apply xml parser on input message.

and my goal is to get username, buil duration,build number, build status and project name from the build.xml file

please help me how to get these details in kibana.

Regards,  
Syam

---

<div class="post-metadata">

### Author: ![rattisyam](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rattisyam](https://discuss.elastic.co/u/rattisyam)
#### Post date: [February 27, 2018, 11:36am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/2 "2018-02-27T11:36:56Z")

</div>

output:  
output which i got now:

\<?xml version='1.0' encoding='UTF-8'?\> rattisyam 1 17 refs/remotes/origin/master 0eae8a5c898e67eab676b5e95983ac11a20e7534 refs/remotes/origin/master 8 https://github.com/rattisyam/maven-project.git refs/remotes/origin/master /var/lib/jenkins/workspace/PackageJob 39 1517409456533 1517409456534 SUCCESS 10641 UTF-8 false /var/lib/jenkins/workspace/PackageJob 2.73.3 false

---

<div class="post-metadata">

### Author: ![rattisyam](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rattisyam](https://discuss.elastic.co/u/rattisyam)
#### Post date: [February 28, 2018, 9:02am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/3 "2018-02-28T09:02:58Z")

</div>

please help me on this config.Thank you

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [February 28, 2018, 4:04pm UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/4 "2018-02-28T16:04:59Z")

</div>

When you post preformatted code, please use the preformatted text button to ensure the forums don't mess with it, there's an icon that looks like `</>`, just highlight your text and click that button, helps in understanding what you paste in.

As far as the issue, I'm not quite sure how it processes the `what` action `next`...does it stick it at the beginning of the line or at the end? Regardless, have you tried setting your pattern to `<build>` and then what to `previous`? Seems like it would give you something like below.

```
<?xml version='1.0' encoding='UTF-8'?>
<build><actions><hudson.model.CauseAction><causeBag class="linked-hash-map">...</build>

```

Afterwards, you could use an if expression with the drop filter to get rid of the xml versioning/encoding line from the event. For the problem with the end tag missing, add the multiline codec option `auto_flush_interval`

---

<div class="post-metadata">

### Author: ![rattisyam](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rattisyam](https://discuss.elastic.co/u/rattisyam)
#### Post date: [March 1, 2018, 9:12am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/5 "2018-03-01T09:12:10Z")

</div>

Thank you very much for u r answer.  
I changed my logstash.conf file to below. even now i am getting the output as same as above.  
even i tried with pattern and still i am not getting the end tag  
input {  
file {  
path =\> "/var/lib/jenkins/jobs/PackageJob/builds/7/build.xml"  
sincedb\_path =\> "/dev/null"  
start\_position =\> "beginning"  
type =\> "buildxml"

```
    codec => multiline {
          pattern => "^<?xml version='1.0' encoding='UTF-8'?>"
          negate => true
          what => "previous"
          max_lines => 1000
          auto_flush_interval => 3
        }
 }

```

}

filter {

}

output {  
elasticsearch {  
hosts =\> ["172.31.33.209:9200"]  
index =\> "abcdef"  
document\_type =\> "demobuildxml"  
}

stdout { codec =\> rubydebug }

}

* * *

Output :  
"message" =\> "\<?xml version='1.0' encoding='UTF-8'?\>\n\n \n \<hudson.model.CauseAction\>\n \<causeBag class="linked-hash-map"\>\n \n \<hudson.model.Cause\_-UserIdCause\>\n rattisyam\n \</hudson.model.Cause\_-UserIdCause\>\n 1\n \n \n \</hudson.model.CauseAction\>\n \<hudson.plugins.jobConfigHistory.JobConfigBadgeAction plugin="jobConfigHistory@2.18"\>\n \n 2018-01-31\_14-29-10\n 2018-01-31\_14-02-54\n \n \</hudson.plugins.jobConfigHistory.JobConfigBadgeAction\>\n \<jenkins.metrics.impl.TimeInQueueAction [plugin="metrics@3.1.2.10](mailto:plugin=%22metrics@3.1.2.10)"\>\n 7\n \</jenkins.metrics.impl.TimeInQueueAction\>\n \<hudson.plugins.git.util.BuildData plugin="git@3.6.4"\>\n \n \n refs/remotes/origin/master\n \<hudson.plugins.git.util.Build\>\n \<marked plugin="git-client@2.6.0"\>\n 0eae8a5c898e67eab676b5e95983ac11a20e7534\n \<branches class="list"\>\n \<hudson.plugins.git.Branch\>\n \<sha1 reference="../../../sha1"/\>\n refs/remotes/origin/master\n \</hudson.plugins.git.Branch\>\n \n \n \<revision reference="../marked"/\>\n 7\n \</hudson.plugins.git.util.Build\>\n \n \n \<lastBuild reference="../buildsByBranchName/entry/hudson.plugins.git.util.Build"/\>\n \n [https://github.com/rattisyam/maven-project.git](https://github.com/rattisyam/maven-project.git)\n \n \</hudson.plugins.git.util.BuildData\>\n \<hudson.plugins.git.GitTagAction plugin="git@3.6.4"\>\n \<tags class="hudson.util.CopyOnWriteMap$Tree"\>\n \n refs/remotes/origin/master\n \n \n \n /var/lib/jenkins/workspace/PackageJob\n \</hudson.plugins.git.GitTagAction\>\n \<hudson.scm.SCMRevisionState\_-None/\>\n \n 37\n 1517408954042\n 1517408954042\n SUCCESS\n 10718\n UTF-8\n false\n \n /var/lib/jenkins/workspace/PackageJob\n 2.73.3\n \<scm class="hudson.plugins.git.GitChangeLogParser" plugin="git@3.6.4"\>\n false\n \n \<culprits class="com.google.common.collect.EmptyImmutableSortedSet"/\>",  
"@version" =\> "1",  
"host" =\> "ip-172-31-43-243",  
"type" =\> "buildxml",  
"tags" =\> [  
[0] "multiline"  
],  
"path" =\> "/var/lib/jenkins/jobs/PackageJob/builds/7/build.xml",  
"@timestamp" =\> 2018-03-01T09:03:07.930Z  
}

Note: Still I am unable to get tag in the message. Please help me how to get that last tag. and even i changed what==\> previous and auto\_flush\_interval =\>3 seconds.

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [March 1, 2018, 2:13pm UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/6 "2018-03-01T14:13:17Z")

</div>

Honestly I'm not sure..what version of ElasticStack products are you using?

---

<div class="post-metadata">

### Author: ![rattisyam](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rattisyam](https://discuss.elastic.co/u/rattisyam)
#### Post date: [March 1, 2018, 11:32pm UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/7 "2018-03-01T23:32:43Z")

</div>

6.1-elk version.thnx.

---

<div class="post-metadata">

### Author: ![rattisyam](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rattisyam](https://discuss.elastic.co/u/rattisyam)
#### Post date: [March 5, 2018, 5:14am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/8 "2018-03-05T05:14:23Z")

</div>

hi team, can nay one help me on this. Thank you.

---

<div class="post-metadata">

### Author: ![rattisyam](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rattisyam](https://discuss.elastic.co/u/rattisyam)
#### Post date: [March 9, 2018, 4:48am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/9 "2018-03-09T04:48:32Z")

</div>

Can any one help on this. Plz.Thnx.

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [March 9, 2018, 6:20pm UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/10 "2018-03-09T18:20:43Z")

</div>

blah...not sure why I didn't see this earlier. You aren't using the XML filter to parse the data so it's ingesting the file, the multiline codec is sticking it all together onto a single line, and then it's being output to Elasticsearch. In the filter section you need to do a couple things, this example is not all inclusive or functional, you'll need to read the Logstash [XML plugin filter documentation](https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-xml.html) and learn some [xpath basics](https://www.w3schools.com/xml/xpath_intro.asp) to tailor it to your needs:

```
filter {
  xml {
    source => "message"
    xpath => [
      "xpath", "field1",
      "xpath", "field2"
    ]
  }
}
```

---

<div class="post-metadata">

### Author: ![rattisyam](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rattisyam](https://discuss.elastic.co/u/rattisyam)
#### Post date: [March 12, 2018, 9:48am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/11 "2018-03-12T09:48:18Z")

</div>

Thanks for u r reply.

The multiline codec is not giving complete XML (it is missing tag. So that I cannot parse it to XML and XPATH.

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [March 13, 2018, 1:01pm UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/12 "2018-03-13T13:01:03Z")

</div>

What output are you getting now?

---

<div class="post-metadata">

### Author: ![rattisyam](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@rattisyam](https://discuss.elastic.co/u/rattisyam)
#### Post date: [March 16, 2018, 10:30am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/13 "2018-03-16T10:30:03Z")

</div>

I am getting same output as previous.  
stdout { codec =\> rubydebug }

}

Output :  
"message" =\> "\<?xml version='1.0' encoding='UTF-8'?\>\n\n \n \<hudson.model.CauseAction\>\n \n \n \<hudson.model.Cause\_-UserIdCause\>\n rattisyam\n \</hudson.model.Cause\_-UserIdCause\>\n 1\n \n \n \</hudson.model.CauseAction\>\n \<hudson.plugins.jobConfigHistory.JobConfigBadgeAction plugin="jobConfigHistory@2.18"\>\n \n 2018-01-31\_14-29-10\n 2018-01-31\_14-02-54\n \n \</hudson.plugins.jobConfigHistory.JobConfigBadgeAction\>\n \<jenkins.metrics.impl.TimeInQueueAction [plugin="metrics@3.1.2.10](mailto:plugin=%22metrics@3.1.2.10)"\>\n 7\n \</jenkins.metrics.impl.TimeInQueueAction\>\n \<hudson.plugins.git.util.BuildData plugin="git@3.6.4"\>\n \n \n refs/remotes/origin/master\n \<hudson.plugins.git.util.Build\>\n \n 0eae8a5c898e67eab676b5e95983ac11a20e7534\n \n \<hudson.plugins.git.Branch\>\n \n refs/remotes/origin/master\n \</hudson.plugins.git.Branch\>\n \n \n \n 7\n \</hudson.plugins.git.util.Build\>\n \n \n \n \n [https://github.com/rattisyam/maven-project.git\n](https://github.com/rattisyam/maven-project.git%5Cn) \n \</hudson.plugins.git.util.BuildData\>\n \<hudson.plugins.git.GitTagAction plugin="git@3.6.4"\>\n \n \n refs/remotes/origin/master\n \n \n \n /var/lib/jenkins/workspace/PackageJob\n \</hudson.plugins.git.GitTagAction\>\n \<hudson.scm.SCMRevisionState\_-None/\>\n \n 37\n 1517408954042\n 1517408954042\n SUCCESS\n 10718\n UTF-8\n false\n \n /var/lib/jenkins/workspace/PackageJob\n 2.73.3\n \n false\n \n ",  
"@version" =\> "1",  
"host" =\> "ip-172-31-43-243",  
"type" =\> "buildxml",  
"tags" =\> [  
[0] "multiline"  
],  
"path" =\> "/var/lib/jenkins/jobs/PackageJob/builds/7/build.xml",  
"@timestamp" =\> 2018-03-01T09:03:07.930Z  
}

Note: Still I am unable to get tag in the message. Please help me how to get that last tag. and even i changed what==\> previous and auto\_flush\_interval =\>3 seconds.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 13, 2018, 10:30am UTC](https://discuss.elastic.co/t/logstash-xml-parser/121641/14 "2018-04-13T10:30:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.