# Xml Filter Not Extracting All Values From XML Document

**URL:** <https://discuss.elastic.co/t/xml-filter-not-extracting-all-values-from-xml-document/155074>\
**Category:** Logstash\
**Created:** [November 1, 2018, 8:00pm UTC](https://discuss.elastic.co/t/xml-filter-not-extracting-all-values-from-xml-document/155074 "2018-11-01T20:00:10Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sankalan\_Banerjee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sankalan_banerjee/32/45660_2.png) [@Sankalan\_Banerjee](https://discuss.elastic.co/u/Sankalan_Banerjee)\
**Post date:** [November 1, 2018, 8:00pm UTC](https://discuss.elastic.co/t/xml-filter-not-extracting-all-values-from-xml-document/155074/1 "2018-11-01T20:00:10Z")

</div>

I have an xml document having nested array elements for Test Case Run Summary . Since a Test Case contains multiple Test Steps, so in the document you will see multiple node and each node contains corresponding step information like \<step\_name\>, \<execution\_status\> etc.

Now when I used XML Filter to parse and extract step information from all the nodes, it generates output as an array element for all the nodes present under node like :-

"step\_name" =\> [  
[0] "Subprocess Login - Login",  
[1] "Subprocess Login",  
[2] "Subprocess Initiate\_Reservation - Initiate\_Reservation",  
[3] "Subprocess Initiate\_Reservation",  
[4] "Subprocess Select\_CarClass~1 - Select\_CarClass",  
[5] "Subprocess Select\_CarClass~1",  
[6] "Subprocess Get\_CarClass\_Details - Get\_CarClass\_Details",  
[7] "Subprocess Update\_Extras - Update\_Extras",  
[8] "Subprocess Commit\_Reservation - Commit\_Reservation",  
[9] "Subprocess Commit\_Reservation",  
[10] "Subprocess Retrieve\_Reservation - Retrieve\_Reservation",  
[11] "Subprocess Retrieve\_Reservation",  
[12] "Subprocess Cancel\_Reservation - Cancel\_Reservation",  
[13] "Subprocess Cancel\_Reservation",  
[14] "Subprocess Retrieve\_Reservation~1 - Retrieve\_Reservation",  
[15] "Subprocess Retrieve\_Reservation~1",  
[16] "Subprocess Cancel\_Reservation~1 - Cancel\_Reservation"  
],  
"execution\_status" =\> [  
[0] "PASSED",  
[1] "PASSED",  
[2] "PASSED",  
[3] "PASSED",  
[4] "PASSED",  
[5] "PASSED",  
[6] "PASSED",  
[7] "PASSED",  
[8] "PASSED",  
[9] "PASSED",  
[10] "PASSED",  
[11] "PASSED",  
[12] "PASSED",  
[13] "PASSED",  
[14] "PASSED",  
[15] "PASSED"  
]

Original XML document contains 18 step nodes. So XML Filter should extract 18 elements for step\_name and execution\_status". But as you can see from the above example, XML Filter extracted 18 elements for step\_name and 16 elements for execution\_status.

Need your help to understand, why the XML Filter is behaving like that?

**Logstash CONF File:-**  
input {  
file {  
path =\> "[//devcorp.deverac.local/dfsroot/NAS/data01\_RQATtesting/ARS/Results/EMAGorilla/EGBO-RCQA/EGBO\_Sanity\_Suite/Run\_10-15-2018\_09-59-39-PM/EGBO\_SANITY\_02\_AuthRes/Step\_Detail\_XML.xml](https://devcorp.deverac.local/dfsroot/NAS/data01_RQATtesting/ARS/Results/EMAGorilla/EGBO-RCQA/EGBO_Sanity_Suite/Run_10-15-2018_09-59-39-PM/EGBO_SANITY_02_AuthRes/Step_Detail_XML.xml)"  
start\_position =\> "beginning"  
sincedb\_path =\> "NUL"  
type =\> "xml"  
codec =\> multiline {  
pattern =\> ""  
negate =\> "true"  
what =\> previous  
auto\_flush\_interval =\> 1  
}   
}  
}

filter  
{  
xml {  
source =\> "message"  
target =\> "xmldata"  
store\_xml =\> false  
xpath =\> [  
"/testCase/tc\_start\_time/text()", "tc\_start\_time",  
"/testCase/step/step\_name/text()", "step\_name",  
"/testCase/step/execution\_time\_sec/text()","execution\_time\_sec",  
"/testCase/step/execution\_status/text()","execution\_status"  
]  
}

mutate {  
remove\_field =\> ["@timestamp","host","tags","@version","type","message"]  
}

}

output {  
stdout { codec =\> rubydebug }  
}

**Original XML Doc:-**  
  
\<tc\_id\>C608690E65444B3F8A3B9DA25BA1493E\</tc\_id\>  
D08B65929BEC48B5BB4B5FF2843CA3B5  
\<tc\_name\>EGBO\_SANITY\_02\_AuthRes\</tc\_name\>  
SERVICE  
\<tc\_start\_time\>10/15/2018 09:56:43 PM\</tc\_start\_time\>  
\<tc\_end\_time\>10/15/2018 09:57:10 PM\</tc\_end\_time\>  
\<tc\_status\>PASSED\</tc\_status\>  
\<exec\_time\_sec\>26 sec\</exec\_time\_sec\>  
\<tc\_desc\>\</tc\_desc\>  
false  
  
\<step\_id\>B6059B588D8D49F2A75B972720BF6B6C\</step\_id\>  
\<tc\_id\>C608690E65444B3F8A3B9DA25BA1493E\</tc\_id\>  
\<session\_id\>D08B65929BEC48B5BB4B5FF2843CA3B5\</session\_id\>  
\<step\_name\>Subprocess Login - Login\</step\_name\>  
\<step\_desc\>\</step\_desc\>  
\<execution\_time\_sec\>1 sec\</execution\_time\_sec\>  
\<execution\_status\>PASSED\</execution\_status\>  
\<step\_request\>RQ\_STEP-1.txt\</step\_request\>  
\<step\_response\>RS\_STEP-1.txt\</step\_response\>  
\<image\_content\>\</image\_content\>  
\<step\_err\>\</step\_err\>  
\<step\_warn\>\</step\_warn\>  
false  
\<step\_response\_header\>\</step\_response\_header\>  
\<step\_start\_time\>1969-12-31T18:00:00-06:00\</step\_start\_time\>  
  
Check HTTP Response Code  
PASSED  
  
  
  
\<step\_id\>4116E443708D4B8C9F08CDF70127C307\</step\_id\>  
\<tc\_id\>C608690E65444B3F8A3B9DA25BA1493E\</tc\_id\>  
\<session\_id\>D08B65929BEC48B5BB4B5FF2843CA3B5\</session\_id\>  
\<step\_name\>Subprocess Cancel\_Reservation~1 - Cancel\_Reservation\</step\_name\>  
\<step\_desc\>\</step\_desc\>  
\<execution\_time\_sec\>\< 1 sec\</execution\_time\_sec\>  
\<execution\_status\>PASSED\</execution\_status\>  
\<step\_request\>RQ\_STEP-17.txt\</step\_request\>  
\<step\_response\>RS\_STEP-17.txt\</step\_response\>  
\<image\_content\>\</image\_content\>  
\<step\_err\>\</step\_err\>  
\<step\_warn\>\</step\_warn\>  
false  
\<step\_response\_header\>\</step\_response\_header\>  
\<step\_start\_time\>1969-12-31T18:00:00-06:00\</step\_start\_time\>  
  
Check HTTP Response Code  
PASSED  
  
  
  
\<step\_id\>4BD6A6C384F840209E30E1CAC2002D21\</step\_id\>  
\<tc\_id\>C608690E65444B3F8A3B9DA25BA1493E\</tc\_id\>  
\<session\_id\>D08B65929BEC48B5BB4B5FF2843CA3B5\</session\_id\>  
\<step\_name\>Subprocess Cancel\_Reservation~1\</step\_name\>  
\<step\_desc\>\</step\_desc\>  
\<execution\_time\_sec\>\< 1 sec\</execution\_time\_sec\>  
\<execution\_status\>PASSED\</execution\_status\>  
\<step\_request\>\</step\_request\>  
\<step\_response\>\</step\_response\>  
\<image\_content\>\</image\_content\>  
\<step\_err\>\</step\_err\>  
\<step\_warn\>\</step\_warn\>  
false  
\<step\_response\_header\>\</step\_response\_header\>  
\<step\_start\_time\>1969-12-31T18:00:00-06:00\</step\_start\_time\>  
  
Match Presence Of Error Code  
PASSED  
  
  
Match Presence Of Error Message  
PASSED  
  
  
\<tc\_sequence\>3\</tc\_sequence\>

Cannot provide you the complete XML Doc due to space limitation.It will be very helpful if you can help me out to resolve this weird issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 29, 2018, 8:00pm UTC](https://discuss.elastic.co/t/xml-filter-not-extracting-all-values-from-xml-document/155074/2 "2018-11-29T20:00:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.