# Loading a file having multiple XML elements, but document is not having ROOT element

**URL:** <https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423>\
**Category:** Logstash\
**Created:** [June 14, 2017, 4:42pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423 "2017-06-14T16:42:58Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![bhattji007](https://avatars.discourse-cdn.com/v4/letter/b/dc4da7/32.png) [@bhattji007](https://discuss.elastic.co/u/bhattji007)\
**Post date:** [June 14, 2017, 4:42pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/1 "2017-06-14T16:42:58Z")

</div>

I am not able to find any post/documents/blogs/search to load details from XML file without root element.  
Here is my structure:  
rt-logs.gz file is having file containing daily real time messages, and each message have pre-defined structure with Namespaces. Only thing is that the file is just multiple elements, and no root element for whole file.

\< tns1:Message xmlns:tns1="...." timestamp=".." messageSize="..." \>  
\< child 1\>....\< /child 1\>  
\< child n\>..\< /child n\>  
\< /tns1:Message\>  
\< tns1:Message xmlns:tns1="...." timestamp=".." messageSize="..." \>  
\< child 1\>....\< /child 1\>  
\< child n\>..\< /child n\>  
\< /tns1:Message\>

How can I parse these files, and get loaded to elastic?

---

<div class="post-metadata">

**Author:** ![bhattji007](https://avatars.discourse-cdn.com/v4/letter/b/dc4da7/32.png) [@bhattji007](https://discuss.elastic.co/u/bhattji007)\
**Post date:** [June 15, 2017, 5:40am UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/2 "2017-06-15T05:40:22Z")

</div>

Any one yet on xml file without ROOT element?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 15, 2017, 7:45pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/3 "2017-06-15T19:45:20Z")

</div>

By definition, XML documents have exactly one root document. Your file therefore contains multiple documents. I suggest you use a multiline codec to join all the lines of each document into a single event. Perhaps the configuration could look something like this:

```nohighlight
codec => multiline {
  pattern => "^<tns1:"
  negate => true
  what => "previous"
}

```

---

<div class="post-metadata">

**Author:** ![bhattji007](https://avatars.discourse-cdn.com/v4/letter/b/dc4da7/32.png) [@bhattji007](https://discuss.elastic.co/u/bhattji007)\
**Post date:** [June 20, 2017, 6:49pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/4 "2017-06-20T18:49:01Z")

</div>

Thanks Magnus, now problem is I can't get all the attributes and other child nodes populated.

filter{  
xml {  
source =\> "message"  
store\_xml =\> "false"  
target =\> "Message"  
}  
mutate {  
add\_field =\> { "msg\_timestamp" =\> "%{[Message][timeStamp]}" }  
add\_field =\> { "priority" =\> "%{[Message][priority]}" }  
add\_field =\> { "id" =\> "%{[Message][Identifiers][msgID]}" }  
add\_field =\> { "transid" =\> "%{[Message][Identifiers][TransID]}" }  
}   
}

I am getting like "transid" =\> "%{[Message][Identifiers][TransID]}", instead of value.

Even tried to put split, but it gives error for split - need to be String or Array...

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 21, 2017, 5:29am UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/5 "2017-06-21T05:29:09Z")

</div>

What does an event processed by the xml filter look like then? Use a `stdout { codec => rubydebug }` output.

---

<div class="post-metadata">

**Author:** ![bhattji007](https://avatars.discourse-cdn.com/v4/letter/b/dc4da7/32.png) [@bhattji007](https://discuss.elastic.co/u/bhattji007)\
**Post date:** [June 21, 2017, 2:40pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/6 "2017-06-21T14:40:49Z")

</div>

Hi Magnus,

Similar as what I gave last line, like this:

[WARN][logstash.filters.split] Only String and Array types are splittable. field:[Message] is of type = NilClass

{  
"path" =\> "C:/xmlTest/xmlnoroot.xml",  
"@timestamp" =\> 2017-06-21T14:26:38.063Z,  
"transid" =\> "%{[Message][Identifiers][TransID]}",  
"@version" =\> "1",  
"host" =\> "",  
"id" =\> "%{[Message][Identifiers][msgID]}",  
"message" =\> "\< Message timeStamp="2016-05-02T03:11:39Z" pr  
iority="High" xmlns="........."\> \< Identifiers \>\n \< msgID\>......\< /msgID\>\n \< TransID\>..........\< /Tran  
sID\>\n \< /Identifiers\>\n\< /Message\>",  
"type" =\> "xml\_log",  
"priority" =\> "%{[Message][priority]}",  
"msg\_timestamp" =\> "%{[Message][timeStamp]}",  
"tags" =\> [  
[0] "multiline",  
[1] "\_split\_type\_failure"  
]  
}

And If remove split filter, split type failure is gone, but output is same.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 22, 2017, 5:18am UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/7 "2017-06-22T05:18:32Z")

</div>

If you look at your event you'll notice that there is no `Message` field to split.

The reason your xml filter does nothing is that haven't set the `xpath` option _and_ you've disabled `store_xml`.

---

<div class="post-metadata">

**Author:** ![bhattji007](https://avatars.discourse-cdn.com/v4/letter/b/dc4da7/32.png) [@bhattji007](https://discuss.elastic.co/u/bhattji007)\
**Post date:** [July 6, 2017, 7:33pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/8 "2017-07-06T19:33:57Z")

</div>

Hi Magnus,

I tried xpath options but no luck yet.

xpath =\>  
["/MessageTraceEvent/priority/text()", "testTs"]

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 6, 2017, 7:55pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/9 "2017-07-06T19:55:09Z")

</div>

Where's the MessageTraceEvent element in your document? The XML abbreviated sample you posted earlier doesn't contain one.

---

<div class="post-metadata">

**Author:** ![bhattji007](https://avatars.discourse-cdn.com/v4/letter/b/dc4da7/32.png) [@bhattji007](https://discuss.elastic.co/u/bhattji007)\
**Post date:** [July 6, 2017, 10:12pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/10 "2017-07-06T22:12:44Z")

</div>

Typo it's Message.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 11, 2017, 6:38pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/11 "2017-07-11T18:38:29Z")

</div>

"priority" isn't a subelement but an attribute, so the correct XPath expression is probably /Message/@priority/text().

---

<div class="post-metadata">

**Author:** ![bhattji007](https://avatars.discourse-cdn.com/v4/letter/b/dc4da7/32.png) [@bhattji007](https://discuss.elastic.co/u/bhattji007)\
**Post date:** [July 11, 2017, 7:35pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/12 "2017-07-11T19:35:07Z")

</div>

Ahh my bad..Thanks for pointing it, I will try that and will let you know.

Regards  
AB

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 8, 2017, 7:50pm UTC](https://discuss.elastic.co/t/loading-a-file-having-multiple-xml-elements-but-document-is-not-having-root-element/89423/13 "2017-08-08T19:50:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.