# Extract data with Logstash and Xpath

**URL:** <https://discuss.elastic.co/t/extract-data-with-logstash-and-xpath/134617>\
**Category:** Logstash\
**Created:** [June 5, 2018, 12:50pm UTC](https://discuss.elastic.co/t/extract-data-with-logstash-and-xpath/134617 "2018-06-05T12:50:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![neo](https://avatars.discourse-cdn.com/v4/letter/n/9fc348/32.png) [@neo](https://discuss.elastic.co/u/neo)\
**Post date:** [June 5, 2018, 12:50pm UTC](https://discuss.elastic.co/t/extract-data-with-logstash-and-xpath/134617/1 "2018-06-05T12:50:43Z")

</div>

Hi ,  
I want to extract data (timestamp and message) via Xpath plugin in Logstash from XML files to display only them in fields in kibana.

> [@neo](#):
>
> Hi ,  
> I want to extract data (timestamp and message) via Xpath plugin in Logstash from XML files to display only them in fields in kibana.
> 
> XML sample:
> 
> ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/f/5fe540daed00d328268b385ef99d41ee47c93c76.png)  
> here is my LOGSTASH conf :
> 
> input {  
> beats {  
> port =\> 5044  
> }  
> }  
> filter{  
> xml{  
> target =\> "doc"  
> store\_xml =\> false  
> source =\> "message"  
> xpath =\>  
> ["/E2ETraceEvent/System/EventID/@EventID", "event\_id",  
> "/E2ETraceEvent/System/Type/@Type", "type",  
> "/E2ETraceEvent/System/SubType/@SubType", "name",  
> "/E2ETraceEvent/System/Level/@Level", "level",  
> "/E2ETraceEvent/System/TimeCreated/@TimeCreated", "time"]  
> }  
> }  
> output {  
> stdout { codec =\> rubydebug }  
> elasticsearch {  
> hosts =\> "100.101.15.181:9200"  
> manage\_template =\> false  
> index =\> "t11-%{+YYYY.MM}"  
> }  
> }  
> in my conf I'm tryig to parse only couple of columns because of the complexity of log, I need to parse all the log data.
> 
> I'm on it couple of day and i don't get any errors on logstash log or filebeat but no data inserted into ELASTIC.
> 
> can someone help me to understand how to work it out?
> 
> Thanks in advance.

here is my LOGSTASH conf :

input {  
beats {  
port =\> 5044  
}  
}  
filter{  
xml{  
target =\> "doc"  
store\_xml =\> false  
source =\> "message"  
xpath =\>  
["/E2ETraceEvent/System/EventID/@EventID", "event\_id",  
"/E2ETraceEvent/System/Type/@Type", "type",  
"/E2ETraceEvent/System/SubType/@SubType", "name",  
"/E2ETraceEvent/System/Level/@Level", "level",  
"/E2ETraceEvent/System/TimeCreated/@TimeCreated", "time"]  
}  
}  
output {  
stdout { codec =\> rubydebug }  
elasticsearch {  
hosts =\> "100.101.15.181:9200"  
manage\_template =\> false  
index =\> "t11-%{+YYYY.MM}"  
}  
}  
in my conf I'm tryig to parse only couple of columns because of the complexity of log, I need to parse all the log data.

I'm on it couple of day and i don't get any errors on logstash log or filebeat but no data inserted into ELASTIC.

can someone help me to understand how to work it out?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2018, 1:46pm UTC](https://discuss.elastic.co/t/extract-data-with-logstash-and-xpath/134617/2 "2018-06-05T13:46:16Z")

</div>

That is not XML. Can you post the XML between line contains three backticks, like this: ```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2018, 3:05pm UTC](https://discuss.elastic.co/t/extract-data-with-logstash-and-xpath/134617/3 "2018-06-05T15:05:44Z")

</div>

To pull out the value of individual nodes, use

```
xpath => { "/E2ETraceEvent/System/EventID/text()" => "event_id" }

```

If you do that, you may also want to

```
if [event_id] { mutate { replace => { "event_id" => "%{[event_id][0]}" } } }

```

However, if you want to parse all of the xml, use

```
store_xml => true target => "someField"
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 3, 2018, 3:05pm UTC](https://discuss.elastic.co/t/extract-data-with-logstash-and-xpath/134617/4 "2018-07-03T15:05:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.