Date field being converted to timestamps

Hello everyone,
Im new in this world.... I hope this question is not really dumb, but Ive being searching the net the whole day and couldnt find an answer.

The thing is simple, Im trying to import some data from a MySQL DB with Logstash.. but the thing is, I have a date field in MySQL, named "date", (formated as DATE), that it gets instantly converted to UTC, and is store as UTC, but I wanted to be stored as it is, that is yyyy-MM-dd
I have mapped the specific date format with kibana by:
PUT /people/default/_mapping
{
"properties": {
"date": {
"type": "date",
"format": "yyyy-MM-dd"
}
}
}

And I tried to use the filter on the logstash conf file:
------ 1st try -----
filter {
date {
match => ["date", "yyyy-MM-dd"]
}
}

---- 2nd try -----
filter {
date {
match => ["date", "yyyy-MM-dd"]
target => "@timestamp"
}
}

---- 3er try ----
filter {
date {
match => ["date", "yyyy-MM-dd"]
target => "date"
}
}

But alway get the UTC timestamp instead of the yyyy-MM-dd format.. also since I have defined the mapping for the date field, logstash throws an error saying: "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [date] of type [date]"

Thanks in advance for your help

If it is a Logstash::TimeStamp then it is going to be in UTC and it is going to look like 2019-05-07T15:41:42.010Z. You can convert it to a string, perhaps formatting it with ruby filter and strftime, but of course then it will not be a date, just a string.

Thanks for the reply. I though about change in it to string but couldnt do it, could you show me how?

Also, so, there is no way to simple get my date values from the mysql and insert them just the way they are into es?

Thanks a lot

event.get("@timestamp").to_f returns the number of seconds since the epoch (.to_f includes fractional seconds). You can format number of second since epoch as a date using strftime

    ruby {
        code => '
            t = Time.at(event.get("@timestamp").to_f)
            event.set("someField", t.strftime("%Y-%m-%d"))
        '
    }
2 Likes

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.