# Date format parsing

**URL:** <https://discuss.elastic.co/t/date-format-parsing/125584>\
**Category:** Logstash\
**Created:** [March 26, 2018, 10:41am UTC](https://discuss.elastic.co/t/date-format-parsing/125584 "2018-03-26T10:41:20Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [March 26, 2018, 10:41am UTC](https://discuss.elastic.co/t/date-format-parsing/125584/1 "2018-03-26T10:41:20Z")

</div>

Hi,

The Original format of the date in the events is:

`Monday, March 5, 2018 11:58:24:874`

The date filter I used:

```
       date {
                  match => ["timestamp", "EEEE, MMMM d, yyyy HH:mm:ss:SSS"]
       }

```

I am getting an error:

**Could not index event to Elasticsearch**

**"reason"=\>"failed to parse [timestamp]"**

How can I solve this?

I thought to use ruby to reformat the date.

I did:

```
       ruby {
              code => "event.set('eventtimestamp', event.get('timestamp').time.strftime('%A, %B %-d, %Y %H:%M:%S:%L'))"
       }

```

But something is wrong here and I am getting an exception in the time() function.

Any idea?

Thanks  
Sharon.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 26, 2018, 11:35am UTC](https://discuss.elastic.co/t/date-format-parsing/125584/2 "2018-03-26T11:35:14Z")

</div>

> I am getting an error:
> 
> Could not index event to Elasticsearch
> 
> "reason"=\>"failed to parse [timestamp]"

Why keep the `timestamp` field in the first place? Your date filter will write the parsed result to the `@timestamp` field.

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [March 26, 2018, 12:04pm UTC](https://discuss.elastic.co/t/date-format-parsing/125584/3 "2018-03-26T12:04:13Z")

</div>

Is this what you mean?

**My logstash code:**

```
    ##########################################
    if [fields][type] == "crmauditcrmcsrsrv" {
    ##########################################        
       mutate {
                add_field => { "[@metadata][fields_type]" => "crmauditcrmcsrsrv" }
       }                            
       grok {
             break_on_match => true
             keep_empty_captures => false
             match => { 
                  message => [                                            
                        "%{AUDITTIME:timestamp}\sThreadID\:%{NUMBER:ThreadDetails}\s%{DATA:service}\sAUDIT:\s%{GREEDYDATA:auditinfo}\s%{HTTPMETHOD:method}\s%{DATA:methodDescription}\s%{DATA:transactionStack}\s\|%{DATA:idNumber}\|"
                 ]
             }
             patterns_dir => "/etc/logstash/patterns"
       }           
       if [auditinfo] =~ /.+/ {
           kv {
                source => "auditinfo"
                value_split => "="
                field_split => " "
           }
       }	
       date {
                  match => ["timestamp", "EEEE, MMMM d, yyyy HH:mm:ss:SSS"]
       }
    }

```

**The date pattern:**

`AUDITTIME %{DAY}, %{MONTH} %{MONTHDAY}, %{YEAR} %{HOUR}:%{MINUTE}:%{SECOND}:%{MILSECOND}`

**In the stdout of Logstash I see:** (Which is right)

```
     "@timestamp" => 2017-06-29T13:14:14.461Z,
   
      "timestamp" => "Thursday, June 29, 2017 16:14:14:461"

```

**But in the Logstash log I see:** (Events aren't indexed into Elasticsearch)

[2018-03-26T14:57:51,265][WARN][logstash.outputs.elasticsearch] `Could not index event to Elasticsearch`. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"logstash2017.06.29crmauditcrmcsrsrv", :\_type=\>"log", :\_routing=\>nil}, 2017-06-29T13:14:14.461Z vpelastic Thursday, June 29, 2017 16:14:14:461 ThreadID:92 com.clariertefy.cih.ertetservicedrfgegs.party.xbeans.ReeretrievePrtgergtayMeansXB.postExecute  
AUDIT: User=Admdedwdwsa1 Transaction=Asefferf1-0wdqd002d-00000000-00001ba5 SET CardBankAccount com.clarify.cbo.Field.pay\_means.id\_number |TJlI|], :response=\>{"index"=\>{"\_index"=\>"logstash2017.06.29crmauditcrmcsrsrv", "\_type"=\>"log", "\_id"=\>"AWJiK\_cfgfgdix8S55454z1F6", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse [timestamp]", `"caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"Thursday, June 29, 2017 16:14:14...\""`}}}}}

Thanks  
Sharon.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 26, 2018, 12:17pm UTC](https://discuss.elastic.co/t/date-format-parsing/125584/4 "2018-03-26T12:17:36Z")

</div>

As I said, do you really need to keep the `timestamp` field when you have `@timestamp`? If you don't need `timestamp` you can remove it and the error will disappear.

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [March 26, 2018, 12:19pm UTC](https://discuss.elastic.co/t/date-format-parsing/125584/5 "2018-03-26T12:19:30Z")

</div>

Now I understand. Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 23, 2018, 12:19pm UTC](https://discuss.elastic.co/t/date-format-parsing/125584/6 "2018-04-23T12:19:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.