Map keyword field in kibana 6.3.0


(DEEPAK SINGLA) #1

Hi
In latest verson of kibana 6.3.0, there are separate fields. One is without "keyword" and another is with "keyword". I want to map both fields and showed a single one.


(David Pilato) #2

You need to adapt your mapping and remove most likely the keyword one if you don't use it.


(DEEPAK SINGLA) #3

Hi @dadoonet

Could you please provide me an example.


(DEEPAK SINGLA) #4

Hi @dadoonet

Could you please provide me an example?


(David Pilato) #5

In your template there's no keyword field so probably this field is generated by your application or anything which is calling elasticsearch.
Check your ingestion layer.

If you don't find it, could you share the output of:

GET Input-*/_search?size=1

(DEEPAK SINGLA) #6

Hi @dadoonet


(David Pilato) #7

Could you run:

GET /Input-*/_search?size=1
{
    "query": {
        "exists" : { "field" : "@version.keyword" }
    }
}

(jaya"><lol) #8

This post was flagged by the community and is temporarily hidden.


(DEEPAK SINGLA) #9
  "took": 3,
  "timed_out": false,
  "_shards": {
    "total": 10,
    "successful": 10,
    "skipped": 0,
    "failed": 0
  },

(David Pilato) #10

What is the output of:

GET Input-2018.10.12/_mapping

(DEEPAK SINGLA) #11

Hi @dadoonet

{
  "Input-2018.10.13": {
    "mappings": {
      "doc": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
         

(David Pilato) #12

What is the output of:

GET /_template

(DEEPAK SINGLA) #13

Hi @dadoonet

"logstash": {
    "order": 0,
    "index_patterns": [
      "Input-%{+YYYY.MM.dd}"
    ],
    

(David Pilato) #14

Can you share the full output of the previous request? Here you copied only one part I think. At least this is not consistent with what you pasted so far.


(DEEPAK SINGLA) #15

Hi @dadoonet

Because of text limit, full information is not coming.

{
  "kibana_index_template:.kibana": {
    "order": 0,
    "index_patterns": [
      ".kibana"
    ],
    "settings": {
      "index": {
        "number_of_shards": "1",
        "auto_expand_replicas": "0-1"
      }
    },
    

(David Pilato) #16

Then share it as a gist on gist.github.com and paste the link here.


(DEEPAK SINGLA) #17

Hi @dadoonet

https://gist.github.com/hello/6e1803ea3a692ac45a7e0d6175e91fb4

(David Pilato) #18

I think I understand now.

In the logstash template you have:

    "index_patterns": [
      "event-%{+YYYY.MM.dd}"
    ],

Where I believe it should be:

    "index_patterns": [
      "event-*"
    ],

So the logstash template is not applied to your index when the index is created the first time which explains why you have all default elasticsearch behavior.

Also, I don't understand why you have been saying that the index name is Input-2018.10.13 if actually the template is applied on event-*. That does not make sense to me.


(DEEPAK SINGLA) #19

Hi @dadoonet

I want to use index pattern event-%{+YYYY.MM.dd}, but it is not matching with kibana index pattrn.

That is why I am using event-*.
Please let me know right index pattern.


(David Pilato) #20

No I meant your elasticsearch index template is incorrect. Fix it.