Convert @timestamp date format

Hi! This is the sample logs which I am trying to store in elasticsearch index.

Mar 26 08:48:21 ip-192-168-0-94 sshd[18576]: Received disconnect from 115.238.245.2: 11:  [preauth]

I want to convert this date pattern to something like "2017-01-12T07:56:41+0000". This is the filter I am using to parse the date.

date {
match => [ "date", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
  }

I can see that in my elasticsearch index @timestamp field is stored like this- "2018-03-18T19:50:42.000Z". How do I convert this @timestamp format to "2017-01-12T07:56:41+0000". I've tried using mutate filter but it didn't work.

mutate {
    replace => [ "date", "yyyy-MM-dd'T'HH:mm:ssZ" ]
}

It just replaced the value. I am stucked here, any help would be appreciated. Thanks!

Are you saying you want @timestamp to be displayed in a different format? If so, why?

Yes, I want to change @timestamp in a different format because the time stored in other indexes are in "2017-01-12T07:56:41+0000" format and for the mapping purpose I want to store the same timestamp format across all of my indexes.
I have also thought about updating the 'date' field with the new format but couldn't find the way of doing so.

For the latter, this does what I think you want...

  mutate { add_field => { "datetime" => "%{@timestamp}" } }
  mutate { gsub => [ "datetime", ".000Z$", "+0000" ] }

This worked :slight_smile:
Thank you so much for the help.

I have one doubt when I used this mutate filter to convert timestamp format in my local windows machine it worked properly and gave me time with +0000 format but, the same filter I've tried on my actual server which is an ubuntu machine & the datetime on kibana is still showed in 2018-03-28T22:12:37.472Z format. What could be the possible reason? Please help!

@Badger Is the timezone on which my machine is running could be the reason why mutate is not replacing the value of datetime field?

Where are you getting fractional seconds from if you start with a date like "Mar 26 08:48:21"?

No matter. Change the gsub to

mutate { gsub => [ "datetime", ".[0-9]{3}Z$", "+0000" ] }

Appreciated the help. It worked :slight_smile:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.