# \_dateparsefailure when parsing XML with logstash
**URL:** https://discuss.elastic.co/t/dateparsefailure-when-parsing-xml-with-logstash/148814
**Category:** Logstash
**Created:** [September 17, 2018, 10:48am UTC](https://discuss.elastic.co/t/dateparsefailure-when-parsing-xml-with-logstash/148814 "2018-09-17T10:48:43Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@drjackal](https://discuss.elastic.co/u/drjackal)
#### Post date: [September 17, 2018, 10:48am UTC](https://discuss.elastic.co/t/dateparsefailure-when-parsing-xml-with-logstash/148814/1 "2018-09-17T10:48:43Z")
Hi there!
I am trying to ingest my DMARC Reports via logstash. I manage to parse the XML, however I fail to get the correct timestamp for the events.
An XML example is:
\ 0.1 AMAZON-SES postmaster@amazonses.com ae2ae781-07d7-4562-8203-297b42968b11 1529366400 1529452800 example.com s r
none
none 100 0 190.190.190.190 1 none pass fail from.com example.com example.com pass example.com pass
I would like to use the end data; feedback\report\_metadata\date\_range\end as my timestamp. The value of this field is unix.
My pipeline looks like this:
input {
file {
path =\> "D:/DMARC/_.xml"
discover\_interval =\> 5
start\_position =\> "beginning"
close\_older =\> 60
codec =\> multiline {
pattern =\> "^\"
negate =\> "true"
what =\> "previous"
}
}
}
filter {
xml {
source =\> "message"
store\_xml =\> false
target =\> "feedback"
force\_array =\> false
xpath =\> [
"/feedback/report\_metadata/org\_name/text()", "dmarc\_reporter",
"/feedback/report\_metadata/email/text()", "dmarc\_reporter\_email",
"/feedback/report\_metadata/report\_id/text()", "dmarc\_repor\_id",
"/feedback/report\_metadata/date\_range/begin/text()", "dmarc\_time\_start",
"/feedback/report\_metadata/date\_range/end/text()", "timestamp",
"/feedback/policy\_published/domain/text()", "dmarc\_dns\_domain",
"/feedback/policy\_published/aspf/text()", "dmarc\_spf\_alignment",
"/feedback/policy\_published/p/text()", "dmarc\_policy",
"/feedback/policy\_published/sp/text()", "dmarc\_sp",
"/feedback/policy\_published/pct/text()", "dmarc\_pct",
"/feedback/policy\_published/adkim/text()", "dkim\_alignment",
"/feedback/record/row/source\_ip/text()", "dmarc\_source\_ip",
"/feedback/record/row/count/text()", "dmarc\_source\_ip\_count",
"/feedback/record/row/policy\_evaluated/disposition/text()", "dmarc\_disposition",
"/feedback/record/row/policy\_evaluated/dkim/text()", "dmarc\_dkim\_result",
"/feedback/record/row/policy\_evaluated/spf/text()", "dmarc\_spf\_result",
"/feedback/record/identifiers/envelope\_from/text()", "dmarc\_env\_from",
"/feedback/record/identifiers/header\_from/text()", "dmarc\_hdr\_from",
"/feedback/record/auth\_results/dkim/domain/text()", "dmarc\_dkim\_auth\_domain",
"/feedback/record/auth\_results/dkim/result/text()", "dmarc\_dkim\_auth\_result",
"/feedback/record/auth\_results/spf/domain/text()", "dmarc\_spf\_auth\_domain",
"/feedback/record/auth\_results/spf/result/text()", "dmarc\_spf\_auth\_result"
]
}
date {
match =\> ['timestamp','UNIX']
}
mutate {
gsub =\> [
"dkim\_alignment", "r", "Relaxed",
"dkim\_alignment", "s", "Strict",
"spf\_alignment", "r", "Relaxed",
"spf\_alignment", "s", "Strict"
]
}
}
output {
elasticsearch {
hosts =\> ["elkserver.local:9200"]
index =\> "logstash-dmarc-%{+xxxx.ww.dd}"
}
stdout { codec =\> rubydebug }
}
I am kind of stuck here and any help would be highly appreciated!
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 17, 2018, 4:54pm UTC](https://discuss.elastic.co/t/dateparsefailure-when-parsing-xml-with-logstash/148814/2 "2018-09-17T16:54:00Z")
Always post configuration file contents, log files, XML snippets etc as preformatted text. Otherwise it will get mangled. Please also feel free to use the preview pane one the right to proofread your posts and check that they're legible.
---
### Author:  [@drjackal](https://discuss.elastic.co/u/drjackal)
#### Post date: [September 18, 2018, 6:26am UTC](https://discuss.elastic.co/t/dateparsefailure-when-parsing-xml-with-logstash/148814/3 "2018-09-18T06:26:43Z")
Okay, let me try again..
My XML looks like this:
```
1.0
aol.net
dmarc-admin@alerts.aol.net
v2-1529909933-example.com
1529798400
1529884800
example.com
s
r
none
none
100
0
138.68.66.8
1
none
fail
fail
example.com
example.com
mfrom
softfail
```
My pipeline looks like this
```
input {
file {
path => "D:/DMARC/*.xml"
discover_interval => 5
start_position => "beginning"
close_older => 20
codec => multiline {
pattern => "^<\?feedback .*\>"
negate => "true"
what => "previous"
}
}
}
filter {
xml {
source => "message"
store_xml => false
target => "feedback"
force_array => false
xpath => [
"/feedback/report_metadata/org_name/text()", "dmarc_reporter",
"/feedback/report_metadata/email/text()", "dmarc_reporter_email",
"/feedback/report_metadata/report_id/text()", "dmarc_report_id",
"/feedback/report_metadata/date_range/begin/text()", "dmarc_time_start",
"/feedback/report_metadata/date_range/end/text()", "timestamp",
"/feedback/policy_published/domain/text()", "dmarc_dns_domain",
"/feedback/policy_published/aspf/text()", "dmarc_spf_alignment",
"/feedback/policy_published/p/text()", "dmarc_policy",
"/feedback/policy_published/sp/text()", "dmarc_sp",
"/feedback/policy_published/pct/text()", "dmarc_pct",
"/feedback/policy_published/adkim/text()", "dkim_alignment",
"/feedback/record/row/source_ip/text()", "dmarc_source_ip",
"/feedback/record/row/count/text()", "dmarc_source_ip_count",
"/feedback/record/row/policy_evaluated/disposition/text()", "dmarc_disposition",
"/feedback/record/row/policy_evaluated/dkim/text()", "dmarc_dkim_result",
"/feedback/record/row/policy_evaluated/spf/text()", "dmarc_spf_result",
"/feedback/record/identifiers/envelope_from/text()", "dmarc_env_from",
"/feedback/record/identifiers/header_from/text()", "dmarc_hdr_from",
"/feedback/record/auth_results/dkim/domain/text()", "dmarc_dkim_auth_domain",
"/feedback/record/auth_results/dkim/result/text()", "dmarc_dkim_auth_result",
"/feedback/record/auth_results/spf/domain/text()", "dmarc_spf_auth_domain",
"/feedback/record/auth_results/spf/result/text()", "dmarc_spf_auth_result"
]
}
mutate {
convert => ["timestamp", "integer"]
remove_field => ["message"]
}
date {
match => ["timestamp","UNIX"]
}
}
output {
elasticsearch {
hosts => ["debruelk2.de.eu.sew:9200"]
index => "logstash-dmarc-%{+xxxx.ww.dd}"
}
stdout { codec => rubydebug }
}
```
I have already changed the pipeline a bit but it is still not working...
Help would be highly appreciated.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 18, 2018, 6:33am UTC](https://discuss.elastic.co/t/dateparsefailure-when-parsing-xml-with-logstash/148814/4 "2018-09-18T06:33:03Z")
> ```
> pattern => "^<\?feedback .*\>"
>
> ```
Your XML doesn't contain any `` tags (only `` and ``). Apart from that things look pretty okay from what I can tell. What do you get from your stdout output after fixing the multiline codec?
---
### Author:  [@drjackal](https://discuss.elastic.co/u/drjackal)
#### Post date: [September 18, 2018, 10:32am UTC](https://discuss.elastic.co/t/dateparsefailure-when-parsing-xml-with-logstash/148814/5 "2018-09-18T10:32:32Z")
Hi Magnus,
In the end it had nothing to do with the XML Tags.
I don't know why but I hat to use this to set the correct timestamp:
```
date {
match => ["[dmarc_time_start][0]","UNIX" ]
}
```
instead of
```
date {
match => ["dmarc_time_start","UNIX"]
}
```
Looks like the XML text was parsed as an array..
Now I've got another error 🙂
- multiline\_codec\_max\_lines\_reached
Thanks,
Jack
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 16, 2018, 10:32am UTC](https://discuss.elastic.co/t/dateparsefailure-when-parsing-xml-with-logstash/148814/6 "2018-10-16T10:32:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.