# Date format match
**URL:** https://discuss.elastic.co/t/date-format-match/135191
**Category:** Logstash
**Created:** [June 8, 2018, 7:14pm UTC](https://discuss.elastic.co/t/date-format-match/135191 "2018-06-08T19:14:40Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Roque\_Moyano](https://discuss.elastic.co/u/Roque_Moyano)
#### Post date: [June 8, 2018, 7:14pm UTC](https://discuss.elastic.co/t/date-format-match/135191/1 "2018-06-08T19:14:40Z")
Hi there
I just read and saw a lot of examples but in can't solve it , I have my filter of this way:
filter {
grok {
match =\> { "message" =\>"[%{TIMESTAMP\_ISO8601:date\_log}] | [%{WORD:ENV}] | [%{WORD:APPLICATION}] | [%{WORD:TYPE}] | [%{GREEDYDATA:LOGMS}]" }
}
my log has the format:
[2018-06-08 11:20:23] | [TEST] | [WSRESTALG] | [CRITICAL] | [this is an example log message]
the error in logstash is:
Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"docker-2018.06.08", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x93ca06a], :response=\>{"index"=\>{"\_index"=\>"docker-2018.06.08", "\_type"=\>"doc", "\_id"=\>"4cRE4GMBWr8dcirVLSft", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse ", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Invalid format: "2018-06-08 11:20:23" is malformed at " 11:20:23""}}}}}
if I try with some example "dates" as [2016-09-19T18:19:00] the filter works fine , the problem is that my date field in the logs are in this format
2018-06-08 11:20:23
I have not found the pattern to match that , I have tried with
%{DATESTAMP:date\_log}
but it doesn't work neither. Any help would be appreciate.
thanks in advance.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 8, 2018, 7:39pm UTC](https://discuss.elastic.co/t/date-format-match/135191/2 "2018-06-08T19:39:32Z")
I do not know why you are getting that error, but what I would do is this. Note that you need to escape the | .
```auto
grok {
match => { "message" =>"\[%{TIMESTAMP_ISO8601:date_log}\] \| \[%{WORD:ENV}\] \| \[%{WORD:APPLICATION}\] \| \[%{WORD:TYPE}\] \| \[%{GREEDYDATA:LOGMS}\]" }
}
date {
match => ["date_log", "yyyy-MM-dd HH:mm:ss"]
timezone => "Europe/Moscow"
remove_field => "date_log"
}
```
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [June 11, 2018, 5:59am UTC](https://discuss.elastic.co/t/date-format-match/135191/3 "2018-06-11T05:59:23Z")
The error comes from Elasticsearch and is related to the mapping of a field (your log entry appears to be garbled after "failed to parse").
Just use a `stdout { codec => rubydebug }` output while you're developing your filter. Once that looks okay you can try to index it in ES.
---
### Author:  [@Roque\_Moyano](https://discuss.elastic.co/u/Roque_Moyano)
#### Post date: [June 11, 2018, 1:27pm UTC](https://discuss.elastic.co/t/date-format-match/135191/4 "2018-06-11T13:27:34Z")
thanks for answering , still with this error , this is my conf
```
input {
```
beats {
# The port to listen on for filebeat connections.
port =\> 5044
# The IP address to listen for filebeat connections.
host =\> "0.0.0.0"
}
}
filter {
grok {
match =\> { "message" =\>"[%{TIMESTAMP\_ISO8601:date\_log}] | [%{WORD:ENV}] | [%{WORD:APPLICATION}] | [%{WORD:TYPE}] | [%{GREEDYDATA:LOGMS}]" }
}
output {
elasticsearch {
hosts =\> ["11.224.212:9200"]
manage\_template =\> false
index =\> "docker-%{+YYYY.MM.dd}"
document\_type =\> "%{[@metadata][type]}"
}
}
output {
file
{
path=\> "/tmp/file.txt"
codec =\> line { format =\> "custom format: %{message}"}
}
}
this is the output of the debug:
ustom format: [2018-06-08 18:19:00] | [DEV] | [WSRESTALG] | [INFO] | [No active profile set, falling back to default profiles: default]
custom format: [2018-06-08 18:19:00] | [DEV] | [WSRESTALG] | [INFO] | [No active profile set, falling back to default profiles: default]
**custom format: [2018-06-08T18:19:00] | [DEV] | [WSRESTALG] | [INFO] | [No active profile set, falling back to default profiles: default]**
the last log works , the previous doesn't , just the Date field is the problem:
2018-06-11T10:09:36,671][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"docker-2018.06.11", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x1a1a2461], :response=\>{"index"=\>{"\_index"=\>"docker-2018.06.11", "\_type"=\>"doc", "\_id"=\>"YcT37mMBWr8dcirVVUUY", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse [date\_log]", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Invalid format: "2018-06-08 18:19:00" is malformed at " 18:19:00""}}}}}
in [http://grokconstructor.appspot.com/do/match#result](http://grokconstructor.appspot.com/do/match#result) , the filter works:
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [June 12, 2018, 6:02am UTC](https://discuss.elastic.co/t/date-format-match/135191/5 "2018-06-12T06:02:38Z")
> this is the output of the debug:
No, that's not what I asked for.
Your problem has nothing to do with your grok filter. It appears to be working fine. The problem is that the Elasticsearch mapping of your `date_log` field doesn't match the contents of the field. I suggest you use a date filter to parse `date_log` into `@timestamp` and then remove `date_log`. @Badger has already shown you how to do that.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 10, 2018, 6:02am UTC](https://discuss.elastic.co/t/date-format-match/135191/6 "2018-07-10T06:02:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.