Date not correctly detected/mapped

Elastic Stack Kibana
1

Hi all,
first of all, I must say i am new in this elastic world. I hace a setup with rsyslog-logstash-Kibana (and also Grafana)

the problem I could not be able to solve is this.

my application is sending a Json message in my logs. I was able to cut out the not-json part of the log and correctly parse the Json with grok.

now the problem is that I have 2 data fields sent by application in the epoch millisecond format, but kibana is dynamically interpreting that as a number. I'd prefer not to disable dynamic indexing. So I've tried to use a mapping template:

PUT _template/all_indices
{
  "template": "*",
  "mappings": {
    "_default_": {
      "dynamic_templates": [
        {
          "full_name": {
            "path_match":   "*Date",
            "mapping": {
              "type":       "date",
              "format": "epoch_millis"
            }
          }
        }
      ]
    }
  }
}

(my data fields contain "date" in the name)
now if I run a get_all I correctly see my datafield as "date" but, when I go to discover the field is still considered as a number (#icon on the left of the page with logs). What Am I missing?

thanks to all of you

2

Hi @gius78,

did you update the mapping after you already created your index-pattern in Kibana? The default date-field needs to be explicitly set for the index-pattern, otherwise Kibana will not pick up on it.

Can you try the following?

  1. recreate the index pattern to configure your index pattern with a default time field

Go to 'Management > Index patterns' and delete the index pattern. Then create a new one (this can use the same name as the one you just related). The wizard will ask you to select the default time-field. Select your date field. A default time field is useful because Kibana will take it automatically into account and filter on it properly on the Discover, dashboard and Visualize pages.

  1. refresh the index pattern to get Kibana detect the new mapping.

Go to 'Management > Index patterns' and press the refresh button (top right). This will make Kibana detect the new mapping. I don't think it will reconfigure the index-pattern with a default-date, so the time-picker date-range won't be automatically applied. So my hunch is you probably want to stick with (1), as this is what most people actually need.

Thx,

1 Like
3

OMG I was so close! I lost an entire day and I was only a click from the solution! I have just refreshed and now is OK!

thanks so much!

p.s.: do you think it's a good way the template I created?

4

yes, I think the template is good. Note that this is applied to all mappings (*), so if you would have mappings with fields that end with *Date, but are not date-fields, those will get updated too.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.