# Logstash/ES date format issue **URL:** \ **Category:** Logstash\ **Created:** [November 5, 2015, 10:35pm UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933 "2015-11-05T22:35:21Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![xtingray](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xtingray/32/5350_2.png) [@xtingray](https://discuss.elastic.co/u/xtingray)\ **Post date:** [November 5, 2015, 10:35pm UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933/1 "2015-11-05T22:35:21Z")
I have been playing with logstash filters, receiving log records with this format: ``` 2015-11-05 17:37:01 This is a log message ``` Using this filter: ``` grok { match => { "message" => "%{DATE:date} %{TIME:time} %{GREEDYDATA:msg}" } } ``` The format of dates in ES is like this: yy-mm-dd (the year part lost the first 2 digits i.e. 15-11-05). If I try to run a query using format yyyy-mm-dd for dates I can't get any record. But using this filter: ``` grok { match => { "message" => "%{TIMESTAMP_ISO8601:date} %{GREEDYDATA:msg}" } } ``` ES supports queries using the format: yyyy-mm-dd (the year part didn't lose any digit) Is there any way to use DATE type for a logstash filter field without losing any digit in the year part? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 6, 2015, 6:50am UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933/2 "2015-11-06T06:50:33Z")
If you inspect the definition of the DATE pattern you'll note that it matches either mm-dd-yy or dd-mm-yy dates. Because you didn't anchor your expression to the beginning of the line by preceding it with `^` it just skipped the century and matched against "15-11-05", which happens to match either mm-dd-yy, dd-mm-yy, yy-mm-dd, or yy-dd-mm. > match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:date %{GREEDYDATA:msg}" } That expression is missing a brace after "date". > Is there any way to use DATE type for a logstash filter field without losing any digit in the year part? No, the DATE _pattern_ only matches two-digits years. You'll have to use something else in your grok expression. What's wrong with TIMESTAMP\_ISO8601? ---
**Author:** ![xtingray](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xtingray/32/5350_2.png) [@xtingray](https://discuss.elastic.co/u/xtingray)\ **Post date:** [November 6, 2015, 3:16pm UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933/3 "2015-11-06T15:16:36Z")
The missing brace was a typo. Sorry. About the TIMESTAMP\_ISO8601, I don't want to use it because it makes harder to define some specific queries that I need. In example: I need to create three independent filters: one for a days range, one for the hours and other for minutes. I need to resolve queries like this: Get all the records available between 2015-11-01 and 2015-11-30, but only those which happened between this range of hours 10 - 11 and only between minutes 0 and 15. If I can split date and time I have a chance to make those filters. But if everything is stored in just one variable of type TIMESTAMP\_ISO8601, how could I? I like the idea of using "^" to avoid losing the century part. I will give it a try. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 6, 2015, 3:28pm UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933/4 "2015-11-06T15:28:41Z")
> Get all the records available between 2015-11-01 and 2015-11-30, but only those which happened between this range of hours 10 - 11 and only between minutes 0 and 15. With a scripted field the uses Groovy this should be possible. See [Kibana 4 Beta 3: Now More Filtery | Elastic Blog](https://www.elastic.co/blog/kibana-4-beta-3-now-more-filtery). > I like the idea of using "^" to avoid losing the century part. As long as you keep using the DATE pattern it won't make a difference. ---
**Author:** ![xtingray](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xtingray/32/5350_2.png) [@xtingray](https://discuss.elastic.co/u/xtingray)\ **Post date:** [November 6, 2015, 3:32pm UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933/5 "2015-11-06T15:32:22Z")
Groovy is a great resource, but I will have to handle millions of records. Quoting from Zachary Tong: > The problem with using a script in your search is that it scales linearly to the number of documents that need to be evaluated. So if you are matching 1bn docs...it has to run that script 1bn times. And scripts are a fair amount slower than native functionality, since it has to boot up a Groovy interpreter for each execution. Groovy is not an option for me 😔 ---
**Author:** ![yahoo](https://avatars.discourse-cdn.com/v4/letter/y/898d66/32.png) [@yahoo](https://discuss.elastic.co/u/yahoo)\ **Post date:** [April 16, 2016, 7:57pm UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933/6 "2016-04-16T19:57:10Z")
Hi @xtingray Did you find a solution? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 21, 2016, 8:24am UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933/8 "2016-06-21T08:24:43Z")
@supermario9, please start a new thread for your unrelated problem. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:51am UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933/9 "2017-07-06T04:51:34Z")