Convert @timestamp date format

Elastic Stack Logstash
1

Hi! This is the sample logs which I am trying to store in elasticsearch index.

Mar 26 08:48:21 ip-192-168-0-94 sshd[18576]: Received disconnect from 115.238.245.2: 11:  [preauth]

I want to convert this date pattern to something like "2017-01-12T07:56:41+0000". This is the filter I am using to parse the date.

date {
match => [ "date", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
  }

I can see that in my elasticsearch index @timestamp field is stored like this- "2018-03-18T19:50:42.000Z". How do I convert this @timestamp format to "2017-01-12T07:56:41+0000". I've tried using mutate filter but it didn't work.

mutate {
    replace => [ "date", "yyyy-MM-dd'T'HH:mm:ssZ" ]
}

It just replaced the value. I am stucked here, any help would be appreciated. Thanks!

2

Are you saying you want @timestamp to be displayed in a different format? If so, why?

3

Yes, I want to change @timestamp in a different format because the time stored in other indexes are in "2017-01-12T07:56:41+0000" format and for the mapping purpose I want to store the same timestamp format across all of my indexes.
I have also thought about updating the 'date' field with the new format but couldn't find the way of doing so.

4

For the latter, this does what I think you want...

  mutate { add_field => { "datetime" => "%{@timestamp}" } }
  mutate { gsub => [ "datetime", ".000Z$", "+0000" ] }
1 Like
5

This worked :slight_smile:
Thank you so much for the help.

6

I have one doubt when I used this mutate filter to convert timestamp format in my local windows machine it worked properly and gave me time with +0000 format but, the same filter I've tried on my actual server which is an ubuntu machine & the datetime on kibana is still showed in 2018-03-28T22:12:37.472Z format. What could be the possible reason? Please help!

7

@Badger Is the timezone on which my machine is running could be the reason why mutate is not replacing the value of datetime field?

8

Where are you getting fractional seconds from if you start with a date like "Mar 26 08:48:21"?

No matter. Change the gsub to

mutate { gsub => [ "datetime", ".[0-9]{3}Z$", "+0000" ] }
9

Appreciated the help. It worked :slight_smile:

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.