# Help Logstash XML Parsing to Xpath
**URL:** https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051
**Category:** Logstash
**Created:** [July 14, 2020, 4:17am UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051 "2020-07-14T04:17:24Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [July 14, 2020, 4:17am UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/1 "2020-07-14T04:17:25Z")
Hello!
I need help how to parse this kind of XML File:
```auto
2020
4
13
2020
4
13
true
true
true
true
true
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 14, 2020, 5:27pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/2 "2020-07-14T17:27:13Z")
You would use an xml filter. Either store the whole XML
```
xml { source => "message" target => "theXML" force_array => false }
```
or pull parts of it out using xpath
```
xml {
source => "message"
store_xml => false
xpath => {
"/NewData/Data/@ID" => "ID"
"/NewData/Data/ClientConfig/ClientSetting/text()" => "Setting"
}
}
```
would get you
```
"Setting" => [
[0] "true",
[1] "true",
[2] "true",
[3] "true",
[4] "true"
],
"ID" => [
[0] "1234"
]
```
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [July 23, 2020, 1:33pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/3 "2020-07-23T13:33:27Z")
> [@Badger](#):
>
> ```auto
> ata/Data/@ID" => "ID"
> "/NewData/Data/C
>
> ```
Hello thank you! How to parse if the ID is unique for certain file as well as client setting? Thanks!
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 10, 2020, 7:23pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/4 "2020-08-10T19:23:24Z")
> [@akim](#):
>
> How to parse if the ID is unique for certain file as well as client setting?
I do not understand the question.
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [August 11, 2020, 5:45pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/5 "2020-08-11T17:45:14Z")
```auto
2020
4
13
2020
4
13
true
true
true
true
true
Some text here
Some Text here B
```
I'll just rephrase. How to parse all text inside unique Document, this could be variable and can be looped --- eg: Document 1 to 40 but unique texts each.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 11, 2020, 6:34pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/6 "2020-08-11T18:34:49Z")
Again, either use store\_xml =\> true, or use xpath and deal with merging all the arrays of data.
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [August 13, 2020, 5:55pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/7 "2020-08-13T17:55:52Z")
Can you help me how it's going to be stored and merged?
Example, I want to show:
Data ID = 1234
Document ID 1= Some text here
Document ID2 = Somet text here B
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 13, 2020, 6:44pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/8 "2020-08-13T18:44:39Z")
You will need ruby to iterate over the documents
```
xml { source => "message" target => "[@metadata][theXML]" xpath => { "/NewData/Data/@ID" => "DataID" } }
mutate { replace => { "DataID" => "%{[DataID][0]}" } }
ruby {
code => '
docs = event.get("[@metadata][theXML][Document]")
if docs.is_a? Array
docs.each { |x|
id = x["ID"]
text = x["Text"][0]
event.set("documentId#{id}", text)
}
end
'
}
```
will get you
```
"documentId1" => " Some text here \n ",
"DataID" => "1234",
"documentId2" => " \nSome Text here B \n ",
```
The metadata field looks like this:
```
"theXML" => {
"Document" => [
[0] {
"ID" => "1",
"Instructions" => "THIS",
"Description" => "XXX",
"Type" => "-1",
"Text" => [
[0] " Some text here \n "
]
},
[1] {
"ID" => "2",
"Instructions" => "THIS",
"Description" => "YYY",
"Type" => "-1",
"Text" => [
[0] " \nSome Text here B \n "
]
}
],
"Data" => [
[0] {
"Instructions" => "Only Here",
"Description" => "This is a sample",
"OtherInstructions" => "NONE",
"EntryDate" => [
[0] {
"CCYY" => [
[0] "2020"
],
"Day" => [
[0] "13"
],
"Month" => [
[0] "4"
]
}
],
"ID" => "1234",
"ClientConfig" => [
[0] {
"ClientSetting" => [
[0] {
"name" => "Some_Value1",
"content" => "true"
},
[1] {
"name" => "Some_Value2",
"content" => "true"
},
[2] {
"name" => "Some_Value3",
"content" => "true"
},
[3] {
"name" => "Some_Value4",
"content" => "true"
},
[4] {
"name" => "Some_Value5",
"content" => "true"
}
]
}
],
"OutDate" => [
[0] {
"CCYY" => [
[0] "2020"
],
"Day" => [
[0] "13"
],
"Month" => [
[0] "4"
]
}
],
"OtherID" => "48",
"Type" => "Inside"
}
]
}
```
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [August 13, 2020, 7:46pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/9 "2020-08-13T19:46:46Z")
> [@Badger](#):
>
> `documentId1`
Looks like my texts are parsed separately..
If I have a paragraph under text do I have to specify anything in the input?
Example Text:
```auto
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nihil acciderat ei, quod nollet, nisi quod anulum, quo delectabatur, in mari abiecerat. Ne amores quidem sanctos a sapiente alienos esse arbitrantur. Quid censes in Latino fore? Is ita vivebat, ut nulla tam exquisita posset inveniri voluptas, qua non abundaret. Primum cur ista res digna odio est, nisi quod est turpis? Duo Reges: constructio interrete.
Quid dubitas igitur, inquam, summo bono a te ita constituto, ut id totum in non dolendo sit, id tenere unum, id tueri, id defendere? Sunt etiam turpitudines plurimae, quae, nisi honestas natura plurimum valeat, cur non cadant in sapientem non est facile defendere. Sed tempus est, si videtur, et recta quidem ad me. At iam decimum annum in spelunca iacet. Hinc ceteri particulas arripere conati suam quisque videro voluit afferre sententiam. Vobis autem, quibus nihil est aliud propositum nisi rectum atque honestum, unde officii, unde agendi principlum nascatur non reperietis. An quod ita callida est, ut optime possit architectari voluptates? Nam diligi et carum esse iucundum est propterea, quia tutiorem vitam et voluptatem pleniorem efficit. Sed mehercule pergrata mihi oratio tua. In enumerandis autem corporis commodis si quis praetermissam a nobis voluptatem putabit, in aliud tempus ea quaestio differatur. Ita finis bonorum existit secundum naturam vivere sic affectum, ut optime is affici possit ad naturamque accommodatissime.
Octavio fuit, cum illam severitatem in eo filio adhibuit, quem in adoptionem D. Ea, quae dialectici nunc tradunt et docent, nonne ab illis instituta sunt aut inventa sunt? Quod autem ratione actum est, id officium appellamus. Quis est enim aut quotus quisque, cui, mora cum adpropinquet, non refugiat timido sanguen átque exalbescát metu? Dat enim intervalla et relaxat. Est autem eius generis actio quoque quaedam, et quidem talis, ut ratio postulet agere aliquid et facere eorum. Sed quia studebat laudi et dignitati, multum in virtute processerat. Restatis igitur vos; Diodorus, eius auditor, adiungit ad honestatem vacuitatem doloris.
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 13, 2020, 8:31pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/10 "2020-08-13T20:31:22Z")
I do not understand how that relates to the \ element in the XML.
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [August 13, 2020, 9:25pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/11 "2020-08-13T21:25:01Z")
I mean, the texts are not parsed correctly if there are too many.
Right now, I'm still having issue errors like:
```auto
Error parsing xml with XmlSimple {:source=>"message", :value=>"", :exception=>#
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 13, 2020, 9:52pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/12 "2020-08-13T21:52:29Z")
That suggests your input has broken up a single XML object into multiple events.
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [August 13, 2020, 10:02pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/13 "2020-08-13T22:02:02Z")
Hmmm. Any reason why this is broken?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 13, 2020, 10:11pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/14 "2020-08-13T22:11:57Z")
You have said nothing about your inputs so I could not possibly say.
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [August 13, 2020, 10:17pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/15 "2020-08-13T22:17:24Z")
Ok sorry for that. My input is basically from google cloud storage with a similar structure above including the Text value that basically is part of Document\>
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [August 13, 2020, 10:26pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/16 "2020-08-13T22:26:22Z")
Here's how it looks like:
```auto
input {
google_cloud_storage {
interval => 60
bucket_id => "somebucket-id"
json_key_file => "/etc/logstash/conf.d/serviceaccount.json"
file_matches => ".*\.xml"
type => "xml"
}
}
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 13, 2020, 10:36pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/17 "2020-08-13T22:36:19Z")
The google\_cloud\_storage input appears to consume "files" a line at a time. If an XML object is split across multiple lines I would expect that to fail.
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [August 13, 2020, 10:46pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/18 "2020-08-13T22:46:50Z")
what's the best approach so that this would not split? Should I use the normal input instead?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 13, 2020, 10:48pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/19 "2020-08-13T22:48:10Z")
Not sure. I have never used Google cloud storage.
---
### Author:  [@akim](https://discuss.elastic.co/u/akim)
#### Post date: [August 13, 2020, 10:55pm UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051/20 "2020-08-13T22:55:06Z")
I mean should I just store the .xml file not in GCS bucket instead? Like inside a server?
```auto
input {
file
{
path => "/etc/logstash/source/*.xml"
start_position => "beginning"
codec => multiline { pattern => "" negate => true what => "previous" }
sincedb_path => "/dev/null"
}
}
```
Is this correct?
[Next page](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051.md?page=2)