Map keyword field in kibana 6.3.0

Elastic Stack Elasticsearch
1

Hi
In latest verson of kibana 6.3.0, there are separate fields. One is without "keyword" and another is with "keyword". I want to map both fields and showed a single one.

image
image.png1574×784 58.1 KB

1 Like
2

You need to adapt your mapping and remove most likely the keyword one if you don't use it.

3

Hi @dadoonet

Could you please provide me an example.

4

Hi @dadoonet

Could you please provide me an example?

5

In your template there's no keyword field so probably this field is generated by your application or anything which is calling elasticsearch.
Check your ingestion layer.

If you don't find it, could you share the output of:

GET Input-*/_search?size=1
6

Hi @dadoonet

7

Could you run:

GET /Input-*/_search?size=1
{
    "query": {
        "exists" : { "field" : "@version.keyword" }
    }
}
9 
  "took": 3,
  "timed_out": false,
  "_shards": {
    "total": 10,
    "successful": 10,
    "skipped": 0,
    "failed": 0
  },
10

What is the output of:

GET Input-2018.10.12/_mapping
11

Hi @dadoonet

{
  "Input-2018.10.13": {
    "mappings": {
      "doc": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
12

What is the output of:

GET /_template
13

Hi @dadoonet

"logstash": {
    "order": 0,
    "index_patterns": [
      "Input-%{+YYYY.MM.dd}"
    ],
14

Can you share the full output of the previous request? Here you copied only one part I think. At least this is not consistent with what you pasted so far.

15

Hi @dadoonet

Because of text limit, full information is not coming.

{
  "kibana_index_template:.kibana": {
    "order": 0,
    "index_patterns": [
      ".kibana"
    ],
    "settings": {
      "index": {
        "number_of_shards": "1",
        "auto_expand_replicas": "0-1"
      }
    },
16

Then share it as a gist on gist.github.com and paste the link here.

17

Hi @dadoonet

https://gist.github.com/hello/6e1803ea3a692ac45a7e0d6175e91fb4
18

I think I understand now.

In the logstash template you have:

    "index_patterns": [
      "event-%{+YYYY.MM.dd}"
    ],

Where I believe it should be:

    "index_patterns": [
      "event-*"
    ],

So the logstash template is not applied to your index when the index is created the first time which explains why you have all default elasticsearch behavior.

Also, I don't understand why you have been saying that the index name is Input-2018.10.13 if actually the template is applied on event-*. That does not make sense to me.

19

Hi @dadoonet

I want to use index pattern event-%{+YYYY.MM.dd}, but it is not matching with kibana index pattrn.

image
image.png1629×631 51.9 KB

That is why I am using event-*.
Please let me know right index pattern.

20

No I meant your elasticsearch index template is incorrect. Fix it.

21

Hi @dadoonet

I am using below index in logstash.conf

output {
    elasticsearch { 
	    index => "event-%{+YYYY.MM.dd}" 
	    hosts => ["localhost"]
        template => "c:/elasticsearc/template.json"		
	}

Do I also need to change it?

Actually I want to make the index in the form of "event-YYYY.MM.dd"

Thanks
Deepak