# The indices which match this index pattern don't contain any time fields

**URL:** <https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403>\
**Category:** Kibana\
**Created:** [January 4, 2022, 5:38am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403 "2022-01-04T05:38:30Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 4, 2022, 5:38am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/1 "2022-01-04T05:38:30Z")

</div>

Hello,

I need assistance in creating an index pattern so I can search logs in Kibana. I have the following logstash configuration:

```auto
input {
  beats {
    port => 5044
    host => "0.0.0.0"
  }
}

filter {

  json {
    source => "message"
  }

  mutate {
    convert => {
      "startTime" => string
    }
  }
 
 date {
    match => ["startTime" , "yyyy-MM-dd'T'HH:mm:ss'.'SSS'Z'"]
    timezone => "UTC"
    target => "@timestamp"
 }

 mutate {
    remove_field => ["startTime", "@version", "tags", "message", "ecs", "agent", "input", "host"]
  }
}

output {
  elasticsearch {
    hosts => "${es_host}"
    user => "${es_user}"
    password => "${es_pwd}"
    index => "xxx-development-%{+YYYY.MM.dd}"
    ilm_enabled => true
    ilm_rollover_alias => "xxx-development"
    ilm_policy => "xxx-development"
  }
}

```

I have the following date field defined in the index template:

```auto
"properties": {
        "@timestamp": {
          "type": "date"
        },

```

An example log message read from filebeat before sent to Logstash looks like this:

```auto
{"startTime":"2021-12-02T05:56:04.696Z","level":"FATAL","serviceName":"ABC","pid":3674,"logId":"App Unhandled Rejection","data":"blah" ,"ServicePid":3674}}}

```

Essentially the startTime field is being sent as @timestamp as per above logstash config. I have previously created index patterns before for other templates using the same method but now it does not work.

When I got to Kibana-\>Index Patterns-\>Create index pattern, I can see matching sources for xxx-development-\*, however on step 2 it picks up no time fields.

What can I do?

---

<div class="post-metadata">

**Author:** ![dosant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dosant/32/64489_2.png) [@dosant](https://discuss.elastic.co/u/dosant)\
**Post date:** [January 4, 2022, 1:22pm UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/2 "2022-01-04T13:22:41Z")

</div>

Hi @damienhaynes,

I'd start from checking indices mapping in Elasticsearch,  
you can do it from Dev tools in kibana:

`GET /xxx-development-*/_mapping`

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 5:35am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/3 "2022-01-05T05:35:06Z")

</div>

Hi @dosant,

thanks for the reply. Here is an example of what gets returned for one of the index logs:

 ![index_log](https://us1.discourse-cdn.com/elastic/original/3X/f/a/faaef0d227b2e8c9cf2b37464640f8cbf7d8fd27.jpeg)

the bulk of the fields are undeath data but is too big to post here.

Is there something specific I should look for, it lists the @timestamp field as type 'date', so that part looks good.

Cheers,  
Damien

---

<div class="post-metadata">

**Author:** ![can.ozdemir](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@can.ozdemir](https://discuss.elastic.co/u/can.ozdemir)\
**Post date:** [January 5, 2022, 6:30am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/4 "2022-01-05T06:30:20Z")

</div>

Hello there,

Can you see data being indexed into @timestamp field?

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 6:43am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/5 "2022-01-05T06:43:17Z")

</div>

Hi @can.ozdemir, how can I do this? Is there a specific GET request I can make on an index log?

I'm not sure how to query what is in a log without Kibana.

---

<div class="post-metadata">

**Author:** ![can.ozdemir](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@can.ozdemir](https://discuss.elastic.co/u/can.ozdemir)\
**Post date:** [January 5, 2022, 6:46am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/6 "2022-01-05T06:46:12Z")

</div>

You can do;

GET /xxx-development-\*/\_search

and look at the @timestamp field if its populated.

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 6:48am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/7 "2022-01-05T06:48:09Z")

</div>

Here is what the \_mapping returns for an empty index log:

```auto
{
  "XXX-development-2022.01.05-000004" : {
    "mappings" : {
      "dynamic" : "true",
      "_source" : {
        "includes" : [],
        "excludes" : []
      },
      "dynamic_date_formats" : [
        "strict_date_optional_time",
        "yyyy/MM/dd HH:mm:ss Z||yyyy/MM/dd Z"
      ],
      "dynamic_templates" : [
        {
          "message_field" : {
            "path_match" : "message",
            "match_mapping_type" : "string",
            "mapping" : {
              "norms" : false,
              "type" : "text"
            }
          }
        },
        {
          "string_fields" : {
            "match" : "*",
            "match_mapping_type" : "string",
            "mapping" : {
              "fields" : {
                "keyword" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              },
              "norms" : false,
              "type" : "text"
            }
          }
        }
      ],
      "date_detection" : true,
      "numeric_detection" : false,
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "@version" : {
          "type" : "keyword"
        },
        "geoip" : {
          "dynamic" : "true",
          "properties" : {
            "ip" : {
              "type" : "ip"
            },
            "latitude" : {
              "type" : "half_float"
            },
            "location" : {
              "type" : "geo_point"
            },
            "longitude" : {
              "type" : "half_float"
            }
          }
        }
      }
    }
  },

```

This is quite a bit different than my working staging/production one (could be because I have tried to do some manual edits to the index template in an attempt to get it to work) e.g:

```auto
"XXX-staging-2022.01.01-000337" : {
    "mappings" : {
      "dynamic_templates" : [
        {
          "message_field" : {
            "path_match" : "message",
            "match_mapping_type" : "string",
            "mapping" : {
              "norms" : false,
              "type" : "text"
            }
          }
        },
        {
          "string_fields" : {
            "match" : "*",
            "match_mapping_type" : "string",
            "mapping" : {
              "fields" : {
                "keyword" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              },
              "norms" : false,
              "type" : "text"
            }
          }
        }
      ],
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "@version" : {
          "type" : "keyword"
        },
        "geoip" : {
          "dynamic" : "true",
          "properties" : {
            "ip" : {
              "type" : "ip"
            },
            "latitude" : {
              "type" : "half_float"
            },
            "location" : {
              "type" : "geo_point"
            },
            "longitude" : {
              "type" : "half_float"
            }
          }
        }
      }
    }
  },

```

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 6:50am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/8 "2022-01-05T06:50:10Z")

</div>

> [@can.ozdemir](#):
>
> GET /xxx-development-\*/\_search

Here is an example of an entry:

```auto
    {
        "_index" : "xxx-development-2022.01.04-000001",
        "_type" : "_doc",
        "_id" : "-yqOIn4B1SKzgg2f1LA3",
        "_score" : 1.0,
        "_source" : {
          "data" : {
            "CurrentTS" : 1641257288608,
            "LagMS" : 490,
            "StartTS" : 1641257288118,
            "PreviousTS" : 0,
            "Counter" : 0
          },
          "serviceType" : "VENUE",
          "logId" : "[LAG] Exceeds lag time",
          "pid" : 1375,
          "serviceName" : "blah",
          "log" : {
            "file" : {
              "path" : "/home/ubuntu/xxx/Logs/VENUE_blah.trace.2022-01-04-00.log"
            }
          },
          "@timestamp" : "2022-01-04T00:48:08.609Z",
          "level" : "DEBUG"
        }
      },

```

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 6:53am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/9 "2022-01-05T06:53:05Z")

</div>

Is that a timestamp format it can recognise?

---

<div class="post-metadata">

**Author:** ![can.ozdemir](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@can.ozdemir](https://discuss.elastic.co/u/can.ozdemir)\
**Post date:** [January 5, 2022, 7:59am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/10 "2022-01-05T07:59:30Z")

</div>

Yes thats a legit format, I have tried what you are experiencing on a local cluster. I used file input instead with same mappings and everything, I can confirm kibana lets me choose @timestamp for my index pattern. I cannot figure out whats wrong here 😕

Maybe your index template overrides the index mappings, can you make sure you can do a range aggregation using the @timestamp field.

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 8:08am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/11 "2022-01-05T08:08:45Z")

</div>

> [@can.ozdemir](#):
>
> Maybe your index template overrides the index mappings, can you make sure you can do a range aggregation using the @timestamp field.

I can try, do you mind giving me an example command (sorry never done that before so not sure how).

---

<div class="post-metadata">

**Author:** ![can.ozdemir](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@can.ozdemir](https://discuss.elastic.co/u/can.ozdemir)\
**Post date:** [January 5, 2022, 8:11am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/12 "2022-01-05T08:11:42Z")

</div>

you can try this, It should return documents within 2 days.

```auto
GET /xxx-development*/_search
{
  "query": {
    "range": {
      "@timestamp": {
        "gte": "now-2d",
        "lte": "now"
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 8:15am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/13 "2022-01-05T08:15:46Z")

</div>

> [@can.ozdemir](#):
>
> ```auto
> GET /xxx-development*/_search
> {
> "query": {
> "range": {
> "@timestamp": {
> "gte": "now-2d",
> "lte": "now"
> }
> }
> }
> }
> 
> ```

Thanks, that returned results as expected e.g.

```auto
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 4,
    "successful" : 4,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : 1.0,
    "hits" : [
      {
....

```

Although I would expect more than 10 hits, not sure what happen to the rest.

---

<div class="post-metadata">

**Author:** ![can.ozdemir](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@can.ozdemir](https://discuss.elastic.co/u/can.ozdemir)\
**Post date:** [January 5, 2022, 8:19am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/14 "2022-01-05T08:19:41Z")

</div>

That total you see is total shards used to get this response, you have more than 10K results you can see it under:

hits.total.value part of the response.

This can confirm your @timestamp is legit and can be used as a date. I cannot point out why kibana refuses to pick it up as date on index pattern creation. have you tried creating it like xxx-development\* instead of xxx-development-\* maybe ilm rollover alias confuses kibana.

note: If you meant you see 10 results on kibana, thats the default size of query responses you can change it by adding size: before the query json object.

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 8:25am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/15 "2022-01-05T08:25:31Z")

</div>

I just tried `xxx-development*` (picks up alias as well), but still, `The indices which match this index pattern don't contain any time fields.` on step 2 :(.

A few things that I have not mentioned:

- I previously had an index with this name but I removed it and started again (i.e. I cleaned up the index logs, index template and policy).
- the reason why I deleted everything is that I was getting conflicting fields that I could not refresh to fix. Inspecting the index pattern it was reporting conflicts but on index logs that were cleaned up months ago!

Note: if I try to continue after Step2 I just get blank page so I don't do that anymore until I can fix the timestamp issue.

---

<div class="post-metadata">

**Author:** ![can.ozdemir](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@can.ozdemir](https://discuss.elastic.co/u/can.ozdemir)\
**Post date:** [January 5, 2022, 8:32am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/16 "2022-01-05T08:32:29Z")

</div>

what do you mean by;

> if I try to continue after Step2 I just get blank page

even if you don't choose a time field, kibanas discover should show you the documents.

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 9:19am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/17 "2022-01-05T09:19:30Z")

</div>

So when I say proceed past Step2, this is what I see (nothing):

 ![empty_age](https://us1.discourse-cdn.com/elastic/original/3X/b/d/bd59a11065c0e00a80b40790e9a9b9e99fb82a61.jpeg)

If I try to reload or select it from the index pattern page, same thing.

If I bring up the dev console (f12), I see:

 ![bad_response](https://us1.discourse-cdn.com/elastic/original/3X/e/f/ef20adc184425ac501f8a981fec73acc3a3fab4e.jpeg)

Something is completely screwed up, not sure what to check next. Any suggestions for diagnostics?

---

<div class="post-metadata">

**Author:** ![can.ozdemir](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@can.ozdemir](https://discuss.elastic.co/u/can.ozdemir)\
**Post date:** [January 5, 2022, 9:31am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/18 "2022-01-05T09:31:31Z")

</div>

Hey there,

can you post the result for;

GET \_cat/templates/your-template-name

---

<div class="post-metadata">

**Author:** ![damienhaynes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damienhaynes/32/99782_2.png) [@damienhaynes](https://discuss.elastic.co/u/damienhaynes)\
**Post date:** [January 5, 2022, 10:27pm UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/19 "2022-01-05T22:27:55Z")

</div>

> [@can.ozdemir](#):
>
> GET \_cat/templates/your-template-name

This is what I get:

```auto
xxx-development [xxx-development-*] 0 20211223

```

---

<div class="post-metadata">

**Author:** ![can.ozdemir](https://avatars.discourse-cdn.com/v4/letter/c/898d66/32.png) [@can.ozdemir](https://discuss.elastic.co/u/can.ozdemir)\
**Post date:** [January 6, 2022, 5:50am UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403/20 "2022-01-06T05:50:13Z")

</div>

I cannot think of a way to diagnose this issue any further, I am sorry. but the kibana error you got when creating the index patterns is concerning maybe try to solve that problem first.

[Next page](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/293403.md?page=2)