# Unable to parse XML through Logstash

**URL:** <https://discuss.elastic.co/t/unable-to-parse-xml-through-logstash/307144>\
**Category:** Logstash\
**Created:** [June 14, 2022, 11:29am UTC](https://discuss.elastic.co/t/unable-to-parse-xml-through-logstash/307144 "2022-06-14T11:29:41Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bineeta\_Das\_IN](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bineeta_das_in/32/107005_2.png) [@Bineeta\_Das\_IN](https://discuss.elastic.co/u/Bineeta_Das_IN)\
**Post date:** [June 14, 2022, 11:29am UTC](https://discuss.elastic.co/t/unable-to-parse-xml-through-logstash/307144/1 "2022-06-14T11:29:42Z")

</div>

I am new to ELK stack.  
Trying to parse below XML code snippet through Logstash:

\<?xml version="1.0"?\> Gambardella, Matthew XML Developer's Guide Computer 44.95 2000-10-01 An in-depth look at creating applications with XML. Ralls, Kim Midnight Rain Fantasy 5.95 2000-12-16 A former architect battles corporate zombies, an evil sorceress, and her own childhood to become queen of the world. 

Below is my config:

input {  
file {  
path =\> "/home/testuser/test/test.xml"  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
codec =\> multiline  
{  
pattern =\> "^\<?book .\*\>"  
negate =\> true  
what =\> "previous"  
}  
}  
}

filter  
{  
xml {  
source =\> "message"  
target =\> "parsed"  
}  
split {  
field =\> "[parsed][book]"  
add\_field =\> {  
bookAuthor =\> "%{[parsed][book][author]}"  
title =\> "%{[parsed][book][title]}"  
genre =\> "%{[parsed][book][genre]}"  
price =\> "%{[parsed][book][price]}"  
publish\_date =\> "%{[parsed][book][publish\_date]}"  
description =\> "%{[parsed][book][description]}"  
}  
}  
}

output {  
Elasticsearch {  
hosts =\> "127.0.0.1"  
index =\> "xmlnew-test"  
codec =\> rubydebug  
}  
}

Although Logstash runs without any errors , no index is created.

I changed user and group permissions for my xml file and restarted Logstash but this is not helping.

[testuser@test ~]$ ls -la /home/testuser/test/test.xml  
-rwxrwxrwx. 1 logstash logstash 4405 Jun 14 10:23 /home/testuser/test/test.xml

Kindly suggest.

---

<div class="post-metadata">

**Author:** ![Bineeta\_Das\_IN](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bineeta_das_in/32/107005_2.png) [@Bineeta\_Das\_IN](https://discuss.elastic.co/u/Bineeta_Das_IN)\
**Post date:** [June 14, 2022, 11:31am UTC](https://discuss.elastic.co/t/unable-to-parse-xml-through-logstash/307144/2 "2022-06-14T11:31:11Z")

</div>

```auto
<?xml version="1.0"?>
<catalog>
   <book id="bk101">
      <author>Gambardella, Matthew</author>
      <title>XML Developer's Guide</title>
      <genre>Computer</genre>
      <price>44.95</price>
      <publish_date>2000-10-01</publish_date>
      <description>An in-depth look at creating applications 
      with XML.</description>
   </book>
   <book id="bk102">
      <author>Ralls, Kim</author>
      <title>Midnight Rain</title>
      <genre>Fantasy</genre>
      <price>5.95</price>
      <publish_date>2000-12-16</publish_date>
      <description>A former architect battles corporate zombies, 
      an evil sorceress, and her own childhood to become queen 
      of the world.</description>
   </book>
</catalog>

```

---

<div class="post-metadata">

**Author:** ![Bineeta\_Das\_IN](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bineeta_das_in/32/107005_2.png) [@Bineeta\_Das\_IN](https://discuss.elastic.co/u/Bineeta_Das_IN)\
**Post date:** [June 14, 2022, 11:32am UTC](https://discuss.elastic.co/t/unable-to-parse-xml-through-logstash/307144/3 "2022-06-14T11:32:11Z")

</div>

Pasted xml code snippet above.

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [June 14, 2022, 12:47pm UTC](https://discuss.elastic.co/t/unable-to-parse-xml-through-logstash/307144/4 "2022-06-14T12:47:11Z")

</div>

Hello,

What does the logstash logs says when picking up the conf or starting to read your file ?

---

<div class="post-metadata">

**Author:** ![Bineeta\_Das\_IN](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bineeta_das_in/32/107005_2.png) [@Bineeta\_Das\_IN](https://discuss.elastic.co/u/Bineeta_Das_IN)\
**Post date:** [June 14, 2022, 1:11pm UTC](https://discuss.elastic.co/t/unable-to-parse-xml-through-logstash/307144/5 "2022-06-14T13:11:33Z")

</div>

This is what I see on the logs:

Jun 14 10:53:31 test logstash[14253]: [2022-06-14T10:53:31,519][INFO][logstash.outputs.Elasticsearch][main] Elasticsearch version determined (7.17.4) {:es\_version=\>7}  
Jun 14 10:53:31 test logstash[14253]: [2022-06-14T10:53:31,521][WARN][logstash.outputs.Elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7}  
Jun 14 10:53:31 test logstash[14253]: [2022-06-14T10:53:31,593][INFO][logstash.outputs.Elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`  
Jun 14 10:53:31 test logstash[14253]: [2022-06-14T10:53:31,604][INFO][logstash.outputs.Elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`  
Jun 14 10:53:31 test logstash[14253]: [2022-06-14T10:53:31,737][INFO][logstash.outputs.Elasticsearch][main] Using a default mapping template {:es\_version=\>7, :ecs\_compatibility=\>:disabled}  
Jun 14 10:53:32 test logstash[14253]: [2022-06-14T10:53:32,275][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>2, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>250, "pipeline.sources"=\>["/etc/logstash/conf.d/test.conf"], :thread=\>"#\<Thread:0x7c4680c4 run\>"}  
Jun 14 10:53:33 test logstash[14253]: [2022-06-14T10:53:33,210][INFO][logstash.javapipeline][main] Pipeline Java execution initialization time {"seconds"=\>0.93}  
Jun 14 10:53:33 test logstash[14253]: [2022-06-14T10:53:33,321][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=\>"main"}  
Jun 14 10:53:33 test logstash[14253]: [2022-06-14T10:53:33,378][INFO][filewatch.observingtail][main][f22f45860093b5e6671036e486fe4177ee0847e7bf0d38553424c322662bf783] START, creating Discoverer, Watch with file and sincedb collections  
Jun 14 10:53:33 test logstash[14253]: [2022-06-14T10:53:33,411][INFO][logstash.agent] Pipelines running {:count=\>1, :running\_pipelines=\>[:main], :non\_running\_pipelines=\>}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 14, 2022, 3:56pm UTC](https://discuss.elastic.co/t/unable-to-parse-xml-through-logstash/307144/6 "2022-06-14T15:56:29Z")

</div>

Your multiline codec is waiting for a line that matches `/^<?book .*>/`. Once it sees one it will flush an event onto the pipeline. You probably need to change the pattern, and also add the auto\_flush\_interval option to the codec, otherwise you will never get an event for the last book in the catalog.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 12, 2022, 3:56pm UTC](https://discuss.elastic.co/t/unable-to-parse-xml-through-logstash/307144/7 "2022-07-12T15:56:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.