# Date timestamp not matching

**URL:** <https://discuss.elastic.co/t/date-timestamp-not-matching/236277>\
**Category:** Logstash\
**Created:** [June 9, 2020, 6:58am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277 "2020-06-09T06:58:08Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [June 9, 2020, 6:58am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/1 "2020-06-09T06:58:08Z")

</div>

I am having trouble trying to get this timestamp to match, any ideas?

`[20200609 084347]`

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [June 9, 2020, 9:05am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/2 "2020-06-09T09:05:21Z")

</div>

That looks like "[yyyyMMdd HHmmss]" to me.

(Just a sidenote: In general your chances of having someone help you increase if "I am having trouble" is followed by "I have tried X and Y that did not work." so people have more details and know that you are putting in effort.)

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [June 9, 2020, 9:55am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/3 "2020-06-09T09:55:03Z")

</div>

It doesn't seem to work. I get:

`Jun 09 11:53:50 mon-01 logstash[24891]: [2020-06-09T11:53:50,421][WARN][logstash.outputs.elasticsearch][main][49e1af4fd928e2ce9f2190100e0f6d6671d7d8d7be6cdbf3f87f31b0add83914] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"var_log_2020.06.09", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x5ece5487>], :response=>{"index"=>{"_index"=>"var_log_2020.06.09", "_type"=>"_doc", "_id"=>"9uKAmHIBkgu1dtKtFSNM", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [log_timestamp] of type [date] in document with id '9uKAmHIBkgu1dtKtFSNM'. Preview of field's value: '20200609 115349'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse date field [20200609 115349] with format [strict_date_optional_time||epoch_millis]", "caused_by"=>{"type"=>"date_time_parse_exception", "reason"=>"Failed to parse with all enclosed parsers"}}}}}}`

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [June 9, 2020, 10:41am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/4 "2020-06-09T10:41:57Z")

</div>

What does your Logstash config look like? Did you try to process this field with a date filter with the correct pattern? If you send it to ES with its original format, it makes sense that ES rejects the event because [strict\_date\_optional\_time||epoch\_millis] is expected.

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [June 9, 2020, 11:18am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/5 "2020-06-09T11:18:39Z")

</div>

So from Logstash I am of course sending the data to Elasticsearch. Here is my grok filter pattern:

**patters/common**

`DATESTAMP_LOG %{YEAR}%{MONTHNUM}%{MONTHDAY} %{HOUR}%{MINUTE}%{SECOND}`

**conf.d/15-app-filters**

`match => { "message" => "\[%{DATESTAMP_LOG:log_timestamp}\] %{LOGLEVEL:loglevel} %{WORD:thread_name} %{NUMBER:application_runtime_ms}ms \(*%{WORD:user_name}\)* - %{GREEDYDATA:message}" }`

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [June 9, 2020, 11:30am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/6 "2020-06-09T11:30:49Z")

</div>

As I said: you need to apply the date filter.

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [June 9, 2020, 11:41am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/7 "2020-06-09T11:41:19Z")

</div>

> [@calanon](#):
>
> ``
> 
> **conf.d/15-app-filters**

Oh sorry my bad I forgot to paste this:

```
date {
    match => ["log_timestamp", "yyyyMMdd HHmmss"]
    target => "@timestamp"
    timezone => "Europe/Berlin"
    add_field => { "debug" => "timestampMatched" }
  }

```

---

<div class="post-metadata">

**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\
**Post date:** [June 9, 2020, 11:43am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/8 "2020-06-09T11:43:12Z")

</div>

you will also need to add that format to your elasticsearch mapping to address this error.

> [@calanon](#):
>
> reason"=\>"failed to parse field [log\_timestamp] of type [date] in document with id '9uKAmHIBkgu1dtKtFSNM'. Preview of field's value: '20200609 115349'", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"failed to parse date field [20200609 115349] with format [strict\_date\_optional\_time||epoch\_millis]",

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [June 9, 2020, 11:43am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/9 "2020-06-09T11:43:59Z")

</div>

This same format?

`yyyyMMdd HHmmss`

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [June 9, 2020, 11:48am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/10 "2020-06-09T11:48:57Z")

</div>

I added this:

```
"log_timestamp": {
  "type": "date",
  "index": true,
  "format": "yyMMdd HHmmss",
  "ignore_malformed": false,
  "doc_values": true,
  "store": false
},
```

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [June 9, 2020, 11:50am UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/11 "2020-06-09T11:50:22Z")

</div>

With this configuration @timestamp should contain the parsed date, but log\_timestamp still contains the original format. ES won't accept the field unless it uses one of the formats that have been configured in the mapping.

Edit: Ah, okay. You adjusted the mapping 🙂

Edit2: But you forgot 2 'y's

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [June 9, 2020, 12:02pm UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/12 "2020-06-09T12:02:36Z")

</div>

Oh yes thank you Jenni

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 7, 2020, 12:03pm UTC](https://discuss.elastic.co/t/date-timestamp-not-matching/236277/13 "2020-07-07T12:03:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.