Fields with date type not showing up in time-field drop down in Kibana

Elastic Stack Kibana
1

Is there any specific format for time-field in Kibana?

I have an index containing some fields with name matching regex .time. or .*_ts (for e.g. create_time, feed_ts) having their type as date and format as dateOptionalTime. When I configure an index pattern for this in Kibana by checking Index contains time-based events, I don't get any options for time-field names. Is there a format required for field-names that can be used for time-fields or is there a specific format or type required for the fields?

I am using ElasticSearch 1.7.4 and Kibana 4

2

Please have a look at the following URL for supported date formats in elasticsearch 1.7:
https://www.elastic.co/guide/en/elasticsearch/reference/1.7/mapping-date-format.html

The page describes recognized formats for date field. You will also need to make sure your mapping is correct.

Your field will need to have date type:
https://www.elastic.co/guide/en/elasticsearch/reference/1.7/mapping-core-types.html#date

You can read more about how to set/update mappings here:
https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-put-mapping.html

let me know if this is helpful.

1 Like
3

I already have this mapping for the data in my index, but kibana still doesn't show the date types in the time-field drop down, here's what I get when I GET the _mapping

{
impression-data: {
  mappings: {
    _default_: {
      dynamic_templates: [
        {
          timestamp: {
            mapping: {
              format: dateOptionalTime,
              type: date,
              doc_values: true
            }match: .*time.*|.*_ts,
            match_pattern: regex
          }
        }{
          default_strings: {
            mapping: {
              index: not_analyzed,
              type: string,
              doc_values: true
            }match_mapping_type: string
          }
        }{
          default_values: {
            mapping: {
              type: {
                dynamic_type
              }doc_values: true
            }match: *match_mapping_type: *
          }
        }
      ]numeric_detection: true_all: {
        enabled: false
      }properties: {
        
      }
    }default_type: {
      dynamic_templates: [
        {
          timestamp: {
            mapping: {
              format: dateOptionalTime,
              type: date,
              doc_values: true
            }match: .*time.*|.*_ts,
            match_pattern: regex
          }
        }{
          default_strings: {
            mapping: {
              index: not_analyzed,
              type: string,
              doc_values: true
            }match_mapping_type: string
          }
        }{
          default_values: {
            mapping: {
              type: {
                dynamic_type
              }doc_values: true
            }match: *match_mapping_type: *
          }
        }
      ]numeric_detection: true_all: {
        enabled: false
      }properties: {
        author: {
          type: string,
          index: not_analyzed,
          doc_values: true
        }comment_count: {
          type: double,
          doc_values: true
        }comment_raw_count: {
          type: double,
          doc_values: true
        }create_time: {
          type: date,
          doc_values: true,
          format: dateOptionalTime
        }ddb_ts: {
          type: date,
          doc_values: true,
          format: dateOptionalTime
        }diversity_score: {
          type: double,
          doc_values: true
        }feed_ts: {
          type: date,
          doc_values: true,
          format: dateOptionalTime
        }gfs_action: {
          type: string,
          index: not_analyzed,
          doc_values: true
        }gfs_ts: {
          type: date,
          doc_values: true,
          format: dateOptionalTime
        }ic_id: {
          type: string,
          index: not_analyzed,
          doc_values: true
        }ml_score: {
          type: double,
          doc_values: true
        }n_image: {
          type: double,
          doc_values: true
        }n_urls: {
          type: double,
          doc_values: true
        }ranking_reason: {
          type: string,
          index: not_analyzed,
          doc_values: true
        }smile_count: {
          type: double,
          doc_values: true
        }smile_raw_count: {
          type: double,
          doc_values: true
        }story_age: {
          type: double,
          doc_values: true
        }story_age_ms: {
          type: long,
          doc_values: true
        }story_id: {
          type: string,
          index: not_analyzed,
          doc_values: true
        }target_page: {
          type: string,
          index: not_analyzed,
          doc_values: true
        }user_id: {
          type: string,
          index: not_analyzed,
          doc_values: true
        }vend_position: {
          type: long,
          doc_values: true
        }
      }
    }
  }
}
}
4

What does the response look like if you issue the following request to Elasticsearch:

<elasticsearch-host>/<index-name>/_mapping/field/*?ignore_unavailable=false&allow_no_indices=false&include_defaults=true
5

Full response pasted here : http://pasted.co/b4b0a8f3
Here's a bit for one of the date fields

                feed_ts: {
                    full_name: feed_ts
                    mapping: {
                        feed_ts: {
                            type: date
                            index_name: feed_ts
                            boost: 1
                            index: not_analyzed
                            store: false
                            doc_values: true
                            term_vector: no
                            norms: {
                                enabled: false
                            }
                            index_options: docs
                            index_analyzer: _date/16
                            search_analyzer: _date/max
                            postings_format: default
                            doc_values_format: default
                            similarity: default
                            fielddata: { }
                            ignore_malformed: false
                            coerce: true
                            precision_step: 16
                            format: dateOptionalTime
                            null_value: null
                            include_in_all: false
                            numeric_resolution: milliseconds
                            locale: 
                        }
                    }
                }
6

I'm not seeing any obvious reasons why the fields wouldn't show up. Are there any errors in the browser console? If not, debugging the javascript might be your best bet.

7

Okay thanks, I tried removing and adding the index with a different mapping so that only a single field feed_ts has type date, this seems to work as that (feed_ts) shows up in the Time-field name while creating the index pattern

8

Got to know from one of my teammates, there's a bug in the UI for the version that we use, so the solution was while creating the index, check and uncheck checkbox "Use event times to create index names" then the drop down shows all the date fields

9

Ah weird! Do you know if there's an existing issue for this on the Github repo? If not would you mind filing a bug report? If it's not already fixed in newer versions, I'd like to be able to squash an annoying bug like this.