# Xml processing

**URL:** https://discuss.elastic.co/t/xml-processing/129533
**Category:** Logstash
**Created:** [April 25, 2018, 4:02pm UTC](https://discuss.elastic.co/t/xml-processing/129533 "2018-04-25T16:02:35Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![sandeepnarla22322](https://avatars.discourse-cdn.com/v4/letter/s/57b2e6/32.png) [@sandeepnarla22322](https://discuss.elastic.co/u/sandeepnarla22322)
#### Post date: [April 25, 2018, 4:02pm UTC](https://discuss.elastic.co/t/xml-processing/129533/1 "2018-04-25T16:02:36Z")

</div>

\<?xml version="1.0" encoding="UTF-8"?\> 1794480 20180425195021 SES 10.174.79.158 2018-04-25 07:20:24.741642168 Wed, 25 Apr 2018 14:20:24 GMT 10.174.79.158 Apache-HttpClient/4.1.1 (java 1.5) VG1 10.174.131.158 2018-04-25 07:20:24 1794481 SES 10.174.79.158 2018-04-25 07:20:24.741642168 198.24.6.220 3758 200 Apache-HttpClient/4.1.1 (java 1.5) 2018-04-25 07:20:24.802608803 2018-04-25 07:20:24.861197008 VG1 10.174.131.158 2018-04-25 07:20:24 1 registeredDevices R2VuZXJpY19DUjEzMFBhcnQyXzIwMTgwNDI1XzE5NDkxNA== 1004 ownerIdtoIMSIandMSISDNmappingCfg.json is not configured. digits 1794482 20180425195021 SES 10.174.79.158 2018-04-25 07:20:24.741642168 Wed, 25 Apr 2018 14:20:24 GMT 10.174.79.158 Apache-HttpClient/4.1.1 (java 1.5) VG1 10.174.131.158 2018-04-25 07:20:24

Hello ALl,

I want to filter data between I have multiple occurences of in a given file

I have tried using xpath and mapping each and indiviudal file

mapping individual fields

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 25, 2018, 4:26pm UTC](https://discuss.elastic.co/t/xml-processing/129533/2 "2018-04-25T16:26:21Z")

</div>

Did you actually read what you posted? Unless you format the XML as preformatted text (use Markdown or the `</>` toolbar button) the tags will be stripped.

---

<div class="post-metadata">

### Author: ![sandeepnarla22322](https://avatars.discourse-cdn.com/v4/letter/s/57b2e6/32.png) [@sandeepnarla22322](https://discuss.elastic.co/u/sandeepnarla22322)
#### Post date: [April 25, 2018, 5:53pm UTC](https://discuss.elastic.co/t/xml-processing/129533/3 "2018-04-25T17:53:14Z")

</div>

179448020180425195021SES10.174.79.158 2018-04-25 07:20:24.741642168Wed, 25 Apr 2018 14:20:24 GMT10.174.79.158Apache-HttpClient/4.1.1 (java 1.5)VG110.174.131.1582018-04-25 07:20:241794481SES10.174.79.158 2018-04-25 07:20:24.741642168198.24.6.2203758200Apache-HttpClient/4.1.1 (java 1.5)2018-04-25 07:20:24.8026088032018-04-25 07:20:24.861197008VG110.174.131.1582018-04-25 07:20:241registeredDevicesR2VuZXJpY19DUjEzMFBhcnQyXzIwMTgwNDI1XzE5NDkxNA==1004ownerIdtoIMSIandMSISDNmappingCfg.json is not configured.digits179448220180425195021SES10.174.79.158 2018-04-25 07:20:24.741642168Wed, 25 Apr 2018 14:20:24 GMT10.174.79.158Apache-HttpClient/4.1.1 (java 1.5)VG110.174.131.1582018-04-25 07:20:24

Thanks for getting back to me this the xml file I have, my bad I have posted wrong data earlier

if I want to extract data between trafficeventData tags is there a easier way to do

I am used to processing csv files, this is the first time i am trying to process xml file

I have a requirement to retain all the fields between these tags

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 25, 2018, 6:29pm UTC](https://discuss.elastic.co/t/xml-processing/129533/4 "2018-04-25T18:29:08Z")

</div>

Seriously, read what you're posting or use the preview pane. What you posted isn't shown XML and unless you fix this it's impossible to help.

---

<div class="post-metadata">

### Author: ![sandeepnarla22322](https://avatars.discourse-cdn.com/v4/letter/s/57b2e6/32.png) [@sandeepnarla22322](https://discuss.elastic.co/u/sandeepnarla22322)
#### Post date: [April 25, 2018, 7:56pm UTC](https://discuss.elastic.co/t/xml-processing/129533/5 "2018-04-25T19:56:00Z")

</div>

```
* <?xml version="1.0" encoding="UTF-8"?>
* <TrafficEventData>
* <TE1>
* <E60>1794480</E60>
* <E21>20180425195021</E21>
* <E61>SES</E61>
* <E65>10.174.79.158 2018-04-25 07:20:24.741642168</E65>
* <E91>Wed, 25 Apr 2018 14:20:24 GMT</E91>
* <E66>10.174.79.158</E66>
* <E69>Apache-HttpClient/4.1.1 (java 1.5)</E69>
* <E72>VG1</E72>
* <E73>10.174.131.158</E73>
* <E74>2018-04-25 07:20:24</E74>
* </TE1>
* <TE0>
* <E60>1794481</E60>
* <E61>SES</E61>
* <E65>10.174.79.158 2018-04-25 07:20:24.741642168</E65>
* <E66>198.24.6.220</E66>
* <E67>3758</E67>
* <E24>200</E24>
* <E69>Apache-HttpClient/4.1.1 (java 1.5)</E69>
* <E70>2018-04-25 07:20:24.802608803</E70>
* <E71>2018-04-25 07:20:24.861197008</E71>
* <E72>VG1</E72>
* <E73>10.174.131.158</E73>
* <E74>2018-04-25 07:20:24</E74>
* <S77>
* <S121>
* <E1000>1</E1000>
* <E1001>registeredDevices</E1001>
* <E1002>R2VuZXJpY19DUjEzMFBhcnQyXzIwMTgwNDI1XzE5NDkxNA==</E1002>
* <E1007>1004</E1007>
* <E206>ownerIdtoIMSIandMSISDNmappingCfg.json is not configured.</E206>
* <E2409>digits</E2409>
* </S121>
* </S77>
* </TE0>
* <TE2>
* <E60>1794482</E60>
* <E21>20180425195021</E21>
* <E61>SES</E61>
* <E65>10.174.79.158 2018-04-25 07:20:24.741642168</E65>
* <E91>Wed, 25 Apr 2018 14:20:24 GMT</E91>
* <E66>10.174.79.158</E66>
* <E69>Apache-HttpClient/4.1.1 (java 1.5)</E69>
* <E72>VG1</E72>
* <E73>10.174.131.158</E73>
* <E74>2018-04-25 07:20:24</E74>
* </TE2>
* </TrafficEventData>

```

sorry for the inconvenience i was not paying attention to the preview this the xml data

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 25, 2018, 8:45pm UTC](https://discuss.elastic.co/t/xml-processing/129533/6 "2018-04-25T20:45:06Z")

</div>

Can't you just use the xml filter with the default `store_xml` option? Then you'll basically convert the XML data to a JSON representation and thus retain all data.

---

<div class="post-metadata">

### Author: ![sandeepnarla22322](https://avatars.discourse-cdn.com/v4/letter/s/57b2e6/32.png) [@sandeepnarla22322](https://discuss.elastic.co/u/sandeepnarla22322)
#### Post date: [April 26, 2018, 4:09pm UTC](https://discuss.elastic.co/t/xml-processing/129533/7 "2018-04-26T16:09:01Z")

</div>

thanks Magnus

---

<div class="post-metadata">

### Author: ![sandeepnarla22322](https://avatars.discourse-cdn.com/v4/letter/s/57b2e6/32.png) [@sandeepnarla22322](https://discuss.elastic.co/u/sandeepnarla22322)
#### Post date: [April 26, 2018, 4:30pm UTC](https://discuss.elastic.co/t/xml-processing/129533/8 "2018-04-26T16:30:00Z")

</div>

Hello Magnus,

Thanks a to one small help, I want to extract a nodename(TNTMO071.) from the following input path

TNTMO071.2018-04-26\_07%3A00.log

I have tried the following pattern and cannot see it in Kibana even though there is no error while running logstash

the following is the input and filter

file {  
path =\>"/opt/LogStashOutputFormatted/TMOES/\*.log"  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
type =\> "tmoes"  
}  
grok {  
match =\> ["path", "/%{WORD:nodeName}.log$"]  
}

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 26, 2018, 8:00pm UTC](https://discuss.elastic.co/t/xml-processing/129533/9 "2018-04-26T20:00:34Z")

</div>

So what **do** you see in Kibana? Copy/paste from the JSON tab so we can see an example document.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 24, 2018, 8:02pm UTC](https://discuss.elastic.co/t/xml-processing/129533/10 "2018-05-24T20:02:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
