Kibana not showing any results in Discover with custom timestamp

Elastic Stack Kibana
1

Version: 7.7.1

I've pushed some data into Elasticsearch using the Python client library. Here is the mapping for my data that I got by running GET rss-test/_mapping in the Dev Tools Console:

    {
      "rss-test" : {
        "mappings" : {
          "properties" : {
            "content" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "tags" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "timestamp" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "timestamp_dt" : {
              "type" : "date",
              "format" : "date_hour_minute_second"
            },
            "title" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "url" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            }
          }
        }
      }
    }

I've inserted one document and it can be seen using the search API. I've created an index pattern from the Kibana Management view as shown below:

index_patter
index_patter2408×1634 136 KB

However, on the Discover tab, I can't see any data for this specific index pattern. I've expanded the time filter to 5 years. Here's a screenshot of that:

Discover
Discover3746×1316 130 KB

When I create an index pattern without choosing the field timestamp_dt as a time filter, I can see the data.

2

Could you expand your time range in Discover a few hours into the future? It's possible this is a timezone issue - as you are using date_hour_minute_second as format, Elasticsearch is assuming UTC as far as I know - if the date you are ingesting is not actually in the UTC time zone, it's possible it ends up getting indexed with an offset. It's always recommended to clearly specify the time zone in dates you are ingesting (or using a timezone-independent format like timestamps).

3

Thanks for your response! You're absolutely right about the timezone issue. I'm currently in a UTC + 4 timezone and it wasn't showing the document as a result.

Do you have any specific recommendations for the date format?

4

If you can influence the incoming date format I recommend strict_date_optional_time (https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html), it's following the ISO date standard which includes the time zone, so there's no ambiguity at any level.

1 Like
5

I could probably modify the date in my pipeline. Thanks for the advice. I'll apply it and see how it goes!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.