Field conversion - Transforming Unix Time into Date

[pedrodantas]

Hello!

I was wondering if there is a way to convert a field that is being received in Unix Time (long) into a fully readable Date in the Kibana environment.

My application is sending JSON logs to Kibana. Two of the log fields are numbers representing a UnixTimeStamp. In this case, the begin and end dates of a procedure.

{
  beginDate: 1669734768,
  endDate: 1669734780
}

This means that when I try to access these fields in a Table Visualization, the data is represented as it shows.

Is there a way to parse this UnixTimeStamp into a more user-friendy date model so that it is easier to read in the Visualization?
Something like "May 25th 2016, 22:39:30.000" or similar.

Thank you!

[aaron-nimocks]

Few ways to do this but I'd suggest having your ingestion pipeline with the data use a Date Processor on its way into Elastic. It will take that value and create a new field that is formatted and mapped properly.

Example

POST /_ingest/pipeline/_simulate
{
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "beginDate": 1669734768
      }
    }
  ],
  "pipeline": {
    "processors": [
      {
        "date": {
          "field": "beginDate",
          "formats": [
            "UNIX"
          ],
          "target_field": "beginDateFormatted"
        }
      }
    ]
  }
}

[pedrodantas]

Alright! I've tried something similar to your approach, creating a variable called "epoch" that maps the UnixTimeStamp to Date.

mutate {
  add_field => {"epoch" => "[@metadata][rabbitmq_properties][timestamp]"}
}
date {
  match => [ "[message][beginDate]","UNIX" ]
  target => "epoch"
}

The field is parsed correctly as can be seen in the log that was received here.

However, when the field is fetched in the Kibana Visualization it appears as a number.

Do you have any idea why this is happening?

Thanks again!

[aaron-nimocks]

What is your elastic data mapping for the field epoch? I don't think it's correct. Should be a date type.

[pedrodantas]

Yup, it is working now! Thank you for your help!