Converting TimeStamp to epoch in LogStash

Elastic Stack Logstash
1

Hi Team,
I am trying to get some records from SQLServer through logstash.
In the database the record is like this : 2002-09-03 04:00:00.000 ,
But it's saving in elastic like : "fileddate" : "2002-09-03T04:00:00.000Z" without applying any filters.
How can i convert this into EPOCH

2

@Jenni / @Christian_Dahlqvist / @warkolm Could you please help me on this

3

It looks like you've got a Logstash Timestamp object/ Elasticsearch date field there, right? Do you want to have the seconds since 1970/01/01 00:00:00 as an integer or float instead? Then you can use the mutate filter to convert the timestamp. Or did you want something else?

4

Hi @Jenni.
Yes it's a date field from SQL.
I want to have my date field into integer (basically EPOCH) format into my index.
How can i write the mutate filter.

5 
mutate {
  convert => { "fileddate" => "integer" }
}
1 Like
6

Thank you,
Integer is throwing an error, So i tried with long and it's working.
The code is

      convert => {
        "fileddate" => "long"
      }

Now i'm trying to add another field for epoch and copying the data from fileddate and converting. But it's not working can you please tell me, where i'm going wrong.

mutate {
      add_field => {
        "epochtime" => "%{fileddate}"
      }
      convert => {
        "epochtime" => "long"
      }
    }
7

add_field is a common option that is executed when a filter has been successful. Therefore it is executed after every other option of the mutate filter. So the execution order for your code above is the opposite of what you expected and convert is called before the field exists. You'll have to split that into two mutate filters.

1 Like
8

Now I'm Calling this Like you mentioned, still throwing error

filter {

    mutate {
      add_field => {
        "epochtime" => "%{fileddate}"
      }
    }

    mutate {
      convert => {
        "epochtime" => "long"
      }
    }
  }
}
9

"throwing error" is a pretty unspecific statement. But I'd guess that it is because by using %{…} you created epochtime as string, not as a Timestamp, so it cannot be converted this way. Use copy instead of add_field to create the field with the correct data type.

10

Please don't ping people that aren't already part of a topic.

1 Like
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.