Failure to convert epoch number in scientific format to date

my input json file (coming from mongoexport) is showing the time in scientific notation, for example:


When I try and convert this to date format using "UNIX_MX" I get the following

lastUpdate 1,575,882,773,525

and the system @timestamp is incorrect:

@timestamp Dec 9, 2019 @ 11:12:53.525 (the correct timestamp is Dec 14 2019 and works fine when I don't us the date filter)

Please suggest how I can fix this issue.

This is my input and filter code:

input {
	file   {
		path => "/usr/share/logstash/mydata/mongodb.json"  
		start_position => "beginning"
		sincedb_path => "/dev/null"
		codec => json

## Add your filters / logstash plugins configuration here

filter {

   date {
	   match => [ "lastUpdate" => "UNIX_MS" ]

   mutate {  remove_field => [ "_id", "path", "host", "userId", "lessonId", "tags", "classreportId", "data.lesson.image", "answers.checkAnswer"] }

I don't understand the issue. A date filter is used to modify @timestamp by default. If you want it to modify another field then use the target option. And specify the timezone too, since 1,575,882,773,525 is 2019/12/09 9:12:53.525 (9 AM, not 11 AM).

1 Like

Thank you so much for pointing this out to me. I had a basic misunderstanding of how this plugin works and this cleared it up and now it runs perfectly!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.