Kibana Discover not showing data for any time range

Elastic Stack Kibana
1

I have an index with timestamp field mapping as following:

"@timestamp": {
    "type": "date",
    "format": "EEE, dd MMM YYYY HH:mm:ss z"
}

@timestamp field in document shows "@timestamp" : "Wed, 26 Sep 2018 11:55:41 GMT".
When I create an index pattern for this index and and choose Time Filter field name as @timestamp, with default type as date, Kibana discover does not show any result. How can I get it to work?

2

Can you check to see how the date was ingested in Elasticsearch?
Easiest way to do is to go in Dev Tools / Console in Kibana and run that default query from there, it should show you docs in ES.

3

Hello @Marius_Dragomir,

What do you mean by default query?

I queried a document of index temp-http and the output of GET /temp-http/_doc/D0u8FWYB-0GCY8C8HHdq is:

{
  "_index" : "temp-http",
  "_type" : "_doc",
  "_id" : "D0u8FWYB-0GCY8C8HHdq",
  "_version" : 1,
  "_seq_no" : 39600,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "response_time" : "162",
    "@timestamp" : "Wed, 26 Sep 2018 11:55:41 GMT",
    "deviceId" : "<id>",
    "url" : "<url>",
    "response_code" : "200",
    "response_msg" : "OK",
    "request_type" : "GET",
    "response" : "",
    "request" : "",
    "uid" : "<uid>"
  }
}
4

Hello @Marius_Dragomir,
Please help.

5

I have another index http-logs with timestamp field mapping as following:

"@timestamp": {
      "type": "date",
      "format": "MMM dd, yyyy HH:mm:ss.SSS"
}

Kibana discover shows data for this index with @timestamp type as date for the index pattern.
It seems like there is issue with the timestamp format used for the index temp-http but I can't figure it out.

6

All the data seems from 2018, based on that timestamp. On the top right of DIscover or visualization page there is a time picker which by default is set to "Last 15 minutes", hence Kibana will show no results. Change that to something that will fit your data(something like "Last 5 years") and this way you will see your documents.

7

Hello @Marius_Dragomir

All the data isn't from 2018. Changing the search time to 5 years didn't work.

The issue started on 11th December when I upgraded Elasticsearch and Kibana to 7.4.0 from 6.8.3. Index temp-http has been present since the beginning with the timestamp field format as "EEE, dd MMM YYYY HH:mm:ss z". But after the upgrade, it doesn't seem to work in Kibana discover.

On the other hand, I created index http-logs after the upgrade with timestamp field format as "MMM dd, yyyy HH:mm:ss.SSS" and it works. Not sure what got messed up.

8

It's probably due to the breaking changes in ES 7.0 regarding the transition from Joda time to Java time: https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html

9

Yes, seems like it.

But how do I fix this because I can't re-index temp-http to a new index with timestamp format based on Java time due to format mismatch?

10

I followed this:

And I tried to create an ingest pipeline like following for re-indexing:

POST _ingest/pipeline/_simulate
{
  "pipeline" :{
  "description": "date pipeline ",
  "processors": [
    {
        "script": {
          "source": """
             SimpleDateFormat format = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss Z");
             ctx._source['@timestamp'] = format.format(format.parse(ctx._source['@timestamp']));
          """
        }
    }
  ]},
  "docs": [
    {
      "_index" : "temp-http",
      "_type" : "_doc",
      "_id" : "ROhMSGgBnWcQaR3FnxJ2",
      "_version" : 2,
      "found" : true,
      "_source" : {
        "response_time" : "175",
        "@timestamp" : "Sun, 13 Jan 2019 17:40:07 GMT"
      }
    }
  ]
}

But it gives "class java.lang.Integer cannot be cast to class java.lang.Long (java.lang.Integer and java.lang.Long are in module java.base of loader 'bootstrap')".
What should I change to make this work?

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.