Kibana 4.1.1 unable to map timestamp which is in UNIX Epoch format

Elastic Stack Kibana
1

Hi Team,

I have recently upgraded my Kibana from 3 to 4.1.1, my nginx access logs has Epoch Time format like 1434589605.318 and kibana is not able to detect this time stamp field, hence I am not able to segregate the logs, however if I try the same access logs on kibana3 it is able to detect timestamp after I mention the filed in Time picker.

Please guide me on how to map this time stamp to index so that I can segregate the logs based on time stamp.

Thanks and Regards,
Naren.

2

What does the mapping in ES look like for that field?

3

There is no change in the way Kibana 4 treats timestamps, compared to Kibana 3. Both require that your time field is mapped as a "date" in Elasticsearch.

For reference, here is a sample mapping that works:

  "@timestamp" : {
    "type" : "date",
    "format" : "strict_date_optional_time||epoch_millis"
  },
4

Thanks for your reply, my kibana4 gets logs from one ES cluster where time format is in epoch format and it is mapped to field "ts", however nothing is being displayed in indices filed

1263×630 88.4 KB
could you please guide me on where should I map the ts field so that it will populate in Indices

Thanks and Regards,
Narendra.

5

Could you post your mappings from ES to make sure we're talking about the same thing?

6

Hi Tanya,

Thanks for your reply, below are my mappings.

Screen Shot 2015-09-11 at 5.47.26 pm.png1254×668 72.8 KB
Screen Shot 2015-09-11 at 5.48.03 pm.png1263×672 70 KB

7

So, I don't see anything with type "date" in there -- that's what you need.

Screen Shot 2015-09-11 at 6.44.41 AM.png2394×916 141 KB

8

The Field TS is the timestamp in epoch format, in kiabna 3 I am able to mention @ts in time picker and it is able to recognise it, however in kibana4 i am not able to map it,

Screen Shot 2015-09-12 at 9.20.38 pm.png1275×327 73.9 KB

Is there a way to mention it in kibana4 apart from modifying it in elastic search?

9

Hello!
I've been struggling with timestamps too. The only way i found to get it to work as i wanted to, was to create a new template for timestamp mapping.
curl -XPUT 'http://YOURELASTICSEARCHSERVER:9200/_template/indexname*/' -d @elastic_template.json

content of file "elastic_template.json"
{
"template": "indexname*",
"order": 1,
"mappings" : {
"index" : {
"properties" : {
"log_date" : {
"type" : "date",
"format": "yyyy-MM-dd HH:mm:ss,SSS||yyyy-MM-dd HH:mm:ss.SSS||yyyy/MM/dd HH:mm:ss||yyyy-MM-dd||MMM dd, yyyy HH:mm:ss a"
},
"appname" : {
"type" : "string",
"index": "not_analyzed"
},
"msg_content" : {
"type" : "string"
}
}
}
}
}
}

I guess this will work for epoch_millis as well. Most important thing: Ensure elasticsearch is saving your timestamp as date, check with:
curl -XGET 'http://YOURELASTICSEARCHSERVER:9200/indexname*/_mapping?pretty'

Hope it helps!