### Author:  [@abhisek](https://discuss.elastic.co/u/abhisek)
#### Post date: [July 17, 2017, 9:06am UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/1 "2017-07-17T09:06:09Z")
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 17, 2017, 9:30am UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/2 "2017-07-17T09:30:09Z")
Please don't post screenshots when you can use normal copy/paste of text.
Use a grok or csv filter to separate timestamp and other stuff from the XML payload, then use an xml filter to process the field containing the XML. The exact look of the configuration depends on what you want the end result to look like.
---
### Author:  [@abhisek](https://discuss.elastic.co/u/abhisek)
#### Post date: [July 17, 2017, 1:09pm UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/3 "2017-07-17T13:09:17Z")
Thanks for the reply magnus and sorry for the screenshot
The XML payload contains 'request' and 'response' which is "|" sperated
for which i have a config file shown below :
> input {
> file {
> path =\> "/opt/test5/practice\_new/data.xml"
> start\_position =\> "beginning"
> codec =\> multiline
> {
> pattern =\> "^\<'soapenv:Envelope\>|\<'soapenv:Envelope\>"
> negate =\> true
> what =\> "next"
> }
> }
> }
> filter {
> xml {
> store\_xml =\> false
> source =\> "req"
> }
> }
> output {
> stdout { codec =\> rubydebug }
> elasticsearch {
> index =\> "req\_res"
> hosts =\> ["localhost:9200"]
> }
> stdout {}
> }
In which i have not included timestamp and other stuff , was trying to load 'request' and 'response' data, first in two sperate field.
For config file mentioned above the whole xml data gets loaded in one field.
My output should look like this :
> **id** - 1499871540
> **timestamp** - 2017-07-12 14:59:00.789398
> **success** - 1
> **host** - [htintra.net](http://htintra.net)
> **status** - read
> **request** -
> \ xmlns:web="[http://webservices.lookup.sdp.bharti.ibm.com](http://webservices.lookup.sdp.bharti.ibm.com)"\>
> ......
> \<'/soapenv:Envelope\>
> **response** -
> \<'soapenv:Envelope xmlns:soapenv="[http://schemas.xmlsoap.org/soap/](http://schemas.xmlsoap.org/soap/)
> envelope/"\>
> ......
> \<'/soapenv:Envelope\>
---
### Author:  [@abhisek](https://discuss.elastic.co/u/abhisek)
#### Post date: [July 18, 2017, 5:49am UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/4 "2017-07-18T05:49:01Z")
Please reply anyone.
thanks in advance
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 18, 2017, 6:26am UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/5 "2017-07-18T06:26:03Z")
Your multiline configuration is wrong. The logic should be "unless the line begins with something like "1499871540|2007-07-12..." join with the previous line.
Once that's working, look into using a csv filter to parse the line and split on "|" (and pray that the XML documents don't contain such a character).
---
### Author:  [@abhisek](https://discuss.elastic.co/u/abhisek)
#### Post date: [July 18, 2017, 10:46am UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/6 "2017-07-18T10:46:49Z")
Thanks for the reply
> [@magnusbaeck](#):
>
> Your multiline configuration is wrong. The logic should be "unless the line begins with something like "1499871540|2007-07-12..." join with the previous line.
Using the above pattern ,the data gets loaded successfully.
> [@magnusbaeck](#):
>
> Once that's working, look into using a csv filter to parse the line and split on "|"
but i'm not able to split the data on "|".
The configuration file used for loading is below :
```
> file {
> path => "/opt/test5/practice_new/data.xml"
> start_position => "beginning"
> codec => multiline
> {
> pattern => "^1499871540|2017-07-1"
> negate => true
> what => "previous"
> }
> }
> }
> filter{
> csv {
> separator => "|"
> columns => ["id","timestamp","success","host","status","request","response"]
> }
> }
> output{
> stdout { codec => rubydebug }
> elasticsearch {
> index => "req_res"
> hosts => ["localhost:9200"]
> }
> stdout {}
> }
```
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 18, 2017, 1:53pm UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/7 "2017-07-18T13:53:37Z")
> but i'm not able to split the data on "|".
What's the problem? If there's a concern that "|" characters could exist inside the XML documents you could use a grok filter.
We'll save time if you show us the result of your `stdout { codec => rubydebug }` output.
---
### Author:  [@abhisek](https://discuss.elastic.co/u/abhisek)
#### Post date: [July 19, 2017, 6:21am UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/8 "2017-07-19T06:21:09Z")
> [@magnusbaeck](#):
>
> We'll save time if you show us the result of your stdout { codec =\> rubydebug } output.
```
> "path" => "/opt/test5/practice_new/data.xml",
> "@timestamp" => 2017-07-19T06:05:07.541Z,
> "@version" => "1",
> "host" => "monitor.htintra.net",
> "message" => "1499871540|2017-07-12 14:59:00.789398|1|abinitiosrv.htintra.net|read| <'soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:web=\"http://webservices.lookup.sdp.bharti.ibm.com\">\n<'soapenv:Header/>\n<'soapenv:Body>\n<'web:getLookUpServiceDetails>\n<'getLookUpService>\n<'serviceRequester>iOBD<'/serviceRequester>\n<'lineOfBusiness>mobility\n<'lookupAttribute>\n<'searchAttrValue>911425152231426<'/searchAttrValue>\n<'/lookupAttribute>\n<'/getLookUpService>\n<'/web:getLookUpServiceDetails>\n<'/soapenv:Body>\n<'/soapenv:Envelope> | <'soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\">\n<'soapenv:Body>\n<'ns:getLookUpServiceDetailsResponse xmlns:ns=\"http://webservices.lookup.sdp.bharti.ibm.com\">\n<'getLookUpServiceReturn>\n<'errorInfo>\n<'ErrorCode/>\n<'ErrorMessage/>\n<'/errorInfo>\n<'lookupResponseList>\n<'/soapenv:Envelope>",
> "tags" => [
> [0] "multiline",
> [1] "_csvparsefailure"
> ]
> }
```
---
### Author:  [@abhisek](https://discuss.elastic.co/u/abhisek)
#### Post date: [July 19, 2017, 6:32am UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/9 "2017-07-19T06:32:35Z")
I tried loading the data by removing the XML data in my file, with the same configuration mentioned above then it successfully split the data according to "|"
but when i add the xml data and try to run, then it gets loaded in kibana but the data does not split on "|".
Does the special characters in my XML data is the reason why it does not get split on "|" ?
> [@magnusbaeck](#):
>
> If there's a concern that "|" characters could exist inside the XML documents
There is a "|" character in my XML document which divide the data into request and response fields .
---
### Author:  [@abhisek](https://discuss.elastic.co/u/abhisek)
#### Post date: [July 20, 2017, 5:24am UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/10 "2017-07-20T05:24:32Z")
please reply ..
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 25, 2017, 8:29pm UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/11 "2017-07-25T20:29:17Z")
The csv filter can't cope with events with embedded newline characters (see issue below). You can use a grok filter instead or possibly a mutate filter and its split option.
>
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 22, 2017, 8:29pm UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362/12 "2017-08-22T20:29:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.