# Date Filter
**URL:** https://discuss.elastic.co/t/date-filter/118128
**Category:** Logstash
**Created:** [February 2, 2018, 12:12am UTC](https://discuss.elastic.co/t/date-filter/118128 "2018-02-02T00:12:30Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@merrillbeth](https://discuss.elastic.co/u/merrillbeth)
#### Post date: [February 2, 2018, 12:12am UTC](https://discuss.elastic.co/t/date-filter/118128/1 "2018-02-02T00:12:30Z")
I can't seem to get this date filter to work:
"logstash.version"=\>"6.1.0"
Elasticsearch 5.6.3
filter{
grok {
patterns\_dir =\> ["/etc/logstash/patterns"]
match =\> ["message", "%{LOGTIMESTAMP:logTimestamp}"]
}
date {
locale =\> "en"
match =\> ["logTimestamp", "MMM D HH:mm:ss"]
target =\> "logTimestamp"
}
}
Log entry is like so:
{
"\_index": "logstash-2018.02.02",
"\_type": "doc",
"\_id": "AWFT0dubiPn6kkLXSbjY",
"\_score": 1,
"\_source": {
"logTimestamp": "Feb 1 23:52:34",
"@version": "1",
"message": "{"@timestamp":"2018-02-01T23:52:34.763Z","@metadata":
{"beat":"filebeat","type":"doc","version":"6.1.0","topic":"Capsule\_logs"},"source":"/data\_0/logs/company/sandbox-dal-9-data.company.com/postgresql343/postgresql343.log","offset":12112324,"message":"Feb 1 23:52:34 [bluemix-sandbox-dal-9-data.company.com](http://bluemix-sandbox-dal-9-data.company.com) postgresql343: haproxy\_status.23 | 2018/02/01 23:52:34 hastatus response time 12.295422ms; cmd time 12.26587ms; response code 503","tags":["postgresql"],"prospector":{"type":"log"},"beat":{"name":"syslog.internal","hostname":"syslog.internal","version":"6.1.0"}}",
"@timestamp": "2018-02-02T00:01:57.337Z",
"tags": [
"\_dateparsefailure"
]
},
"fields": {
"@timestamp": [
1517529717337
]
}
}
I want to use the timestamp for the logs found in the message. Why do I keep getting the \_dateparsefailure ? The grok pattern works and logTimestamp gets dumped into kibana as a string.
Thanks!
---
### Author:  [@aberrantCode](https://discuss.elastic.co/u/aberrantCode)
#### Post date: [February 2, 2018, 1:21am UTC](https://discuss.elastic.co/t/date-filter/118128/2 "2018-02-02T01:21:13Z")
Try "en-US" or "en\_US" for locale.
---
### Author:  [@merrillbeth](https://discuss.elastic.co/u/merrillbeth)
#### Post date: [February 2, 2018, 1:45am UTC](https://discuss.elastic.co/t/date-filter/118128/3 "2018-02-02T01:45:05Z")
> [@aberrantCode](#):
>
> en-US
That causes the date filter to ignore the logTimestamp and match the @timestamp attribute.
---
### Author:  [@aberrantCode](https://discuss.elastic.co/u/aberrantCode)
#### Post date: [February 2, 2018, 1:52am UTC](https://discuss.elastic.co/t/date-filter/118128/4 "2018-02-02T01:52:11Z")
Are you using a JDBC input filter? [Related Issue?](https://github.com/logstash-plugins/logstash-filter-date/issues/95)
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 2, 2018, 7:25am UTC](https://discuss.elastic.co/t/date-filter/118128/5 "2018-02-02T07:25:58Z")
When the date filter can't parse a string it'll log clues about what it's having problems with.
> match =\> ["logTimestamp", "MMM D HH:mm:ss"]
"D" is day-of-year. Use "d" instead. I think you'll also need to specify a second pattern with "dd" instead of "d".
---
### Author:  [@merrillbeth](https://discuss.elastic.co/u/merrillbeth)
#### Post date: [February 7, 2018, 8:18pm UTC](https://discuss.elastic.co/t/date-filter/118128/6 "2018-02-07T20:18:08Z")
Hi. I've tried this as well but am still having the issue where the match string now matches the Time attribute rather than the log timestamp. My current config looks like so:
input {
kafka {
bootstrap\_servers =\> "[kafka02.company.net:9093](http://kafka02.company.net:9093)"
topics =\> ["Capsule\_logs"]
}
}
filter{
grok {
patterns\_dir =\> ["/etc/logstash/patterns"]
match =\> ["message", "%{LOGTIMESTAMP:logTimestamp}"]
}
```
date {
timezone => "UTC"
match => ["logTimestamp", "MMM d HH:mm:ss"]
target => "logTimestamp"
}
}
```
output {
elasticsearch {
hosts =\> ["[https://user](https://user):password.deployment-logs.company.com:17825/"]
ssl =\> true
ssl\_certificate\_verification =\> true
}
}
I tried the locale parameter which caused a dateparse error. I realize once this is working I'll have to add an additional match parameter for dd.
Thanks!
---
### Author:  [@merrillbeth](https://discuss.elastic.co/u/merrillbeth)
#### Post date: [February 7, 2018, 8:42pm UTC](https://discuss.elastic.co/t/date-filter/118128/7 "2018-02-07T20:42:01Z")
Hey. That's a good thought. I've updated the GH issue.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 7, 2018, 8:55pm UTC](https://discuss.elastic.co/t/date-filter/118128/8 "2018-02-07T20:55:01Z")
> I've tried this as well but am still having the issue where the match string now matches the Time attribute rather than the log timestamp.
What do you mean? Please show examples instead of describing the results.
---
### Author:  [@merrillbeth](https://discuss.elastic.co/u/merrillbeth)
#### Post date: [February 12, 2018, 8:57pm UTC](https://discuss.elastic.co/t/date-filter/118128/9 "2018-02-12T20:57:04Z")
**filter:**
filter{
grok {
patterns\_dir =\> ["/etc/logstash/patterns"]
match =\> ["message", "%{LOGTIMESTAMP:logTimestamp}"]
}
date {
match =\> ["logTimestamp", "MMM dd HH:mm:ss"]
target =\> "logTimestamp"
}
}
**Pattern:**
LOGTIMESTAMP %{MONTH} +%{MONTHDAY} %{TIME}
**LogSample:**
{"@timestamp":"2018-02-12T20:53:59.319Z","@metadata":{"beat":"filebeat","type":"doc","version":"6.1.0","topic":"Capsule\_logs"},"message":"Feb 12 16:46:59 [server.company.com](http://server.company.com) mongodb315: mongos.24 | 2018-02-12T16:46:59.472+0000 I SHARDING [Balancer] distributed lock with ts: 5a81c503a6805c4336799218' unlocked.","tags":["mongodb"],"prospector":{"type":"log"},"beat":{"name":"syslog.internal","hostname":"syslog.internal","version":"6.1.0"},"source":"/data\_0/logs/compose/server.company.com/mongodb315/mongodb315.log","offset":1337412}
logTimestamp is matched:
"logTimestamp": "Feb 12 16:46:59"
But with a \_dateparsefailure. I don't understand why the date filter fails to match even though the patterns match.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 12, 2018, 9:01pm UTC](https://discuss.elastic.co/t/date-filter/118128/10 "2018-02-12T21:01:40Z")
If the date filter fails it'll log a message that points to the error.
---
### Author:  [@merrillbeth](https://discuss.elastic.co/u/merrillbeth)
#### Post date: [February 13, 2018, 12:18am UTC](https://discuss.elastic.co/t/date-filter/118128/11 "2018-02-13T00:18:56Z")
With Logstash logging set to debug. In the logs what I see is:
[2018-02-13T00:16:11,475][DEBUG][logstash.pipeline] output received {"event"=\>{"tags"=\>["\_dateparsefailure"], "message"=\>"{"@timestamp":"2018-02-13T00:16:07.784Z","@metadata":{"beat":"filebeat","type":"doc","version":"6.1.0","topic":"Capsule\_logs"},"source":"/data\_0/logs/company.com/postgresql463/postgresql463.log","offset":8754897,"message":"Feb 12 23:25:22 [company.com](http://company.com) postgresql463: postgres.24 | Updating the TTL for primary.","tags":["postgresql"],"prospector":{"type":"log"},"beat":{"name":"syslog.internal","hostname":"syslog.internal","version":"6.1.0"}}", "@version"=\>"1", "@timestamp"=\>2018-02-13T00:16:10.933Z, "logTimestamp"=\>"Feb 12 23:25:22"}}
The logTimestamp is being set but the event is being tagged with a dateparsefailure. I don't see anything specifically noting why this is happening, just that it is happening.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 13, 2018, 12:19am UTC](https://discuss.elastic.co/t/date-filter/118128/12 "2018-03-13T00:19:39Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.