Fresh install kibana Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [log, doc]


#1

Hi,

[Background]
I upgrade ES from 5.6 to 6.1.2 and the kibana version is 6.1.2 (All are docker with OSS distributed version).
For some reason, i would like to use a fresh new kibana installation, so i delete '.kibana' index, then relaunch kibana to let it create its own index from scratch.

[Problem]

  1. I cannot "Create index pattern" on kibana. After click "create index pattern" button it return 400 error at "api/saved_objects/index-pattern".

  2. After "problem 2", i try to create the ".kibana" index by my self. And I got error:

PUT /.kibana
    {
      "error": {
        "root_cause": [
          {
            "type": "remote_transport_exception",
            "reason": "[es-master-1][10.0.0.39:9300][indices:admin/create]"
          }
        ],
        "type": "illegal_argument_exception",
        "reason": "Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [log, doc]"
      },
      "status": 400
    }

I found there is a template "kibana_index_template:.kibana" as below:

 "kibana_index_template:.kibana": {
    "order": 0,
    "index_patterns": [
      ".kibana"
    ],
    "settings": {
      "index": {
        "number_of_shards": "1"
      }
    },
    "mappings": {
      "doc": {
        "dynamic": "strict",
        "properties": {
          "type": {
            "type": "keyword"
          },
          "updated_at": {
            "type": "date"
          },
          "config": {
            "dynamic": true,
            "properties": {
              "buildNum": {
                "type": "keyword"
              }

... IGNORED ...

If I try to delete the template "kibana_index_template:.kibana", I will success to create ".kibana" index via PUT /.kibana.

[Question]
I know there is breaking changes regard to ES 6.0 and the Removal of Mapping Types.
It seems the template created by kibana automatically is not change to the new mapping yet?

Thank you!


(Bhavya R M) #2

Hi,

You are correct: just deleting .kibana doesn't solve this problem.
You need to use the upgrade assistant in 5.6 - reindex .kibana and then upgrade to 6.x.

Thanks,
Bhavya


#3

Hi,

Thanks for your reply.

I might not understand the problem.

Can't I assume only install the new kibana installation on 6.1? I can discard all previous data before 5.6, just want a new one.

If I have already upgrade ES and kibana to 6.1 with deleted .kibana and "kibana_index_template:.kibana" template'. How to upgrade by upgrade assistant?

BTW. I notice Upgrade Assistant only existed in X-Pack, but I'm using OSS docker version.

Thank you!


(Bhavya R M) #4

Hi,

We removed types from elasticsearch in 6.0. https://www.elastic.co/blog/kibana-6-removal-of-mapping-types. So if you are planning to bring over data from 5.x to 6.x - you need to upgrade to 5.6 before - get the x-pack basic (which is free) - which will add upgrade assistant to your management view:

Use it to upgrade and then migrate to 6.x. You can also manually migrate index patterns without using the upgrade assistant: https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade.html

Hope this helps.

Thanks,
Bhavya


#5

Hi,

What if I have upgrade my cluster from 5.6 to 6.1 (it seems hard to downgrade to 5.6 right now).
Does the "reindex" by manual still work for me? (perform reindex on ES 6.1).


(Bhavya R M) #6

Not really sure. Am checking in with our ops team and will keep you posted.

Cheers,
Bhavya


#7

Thank you for your kindly help! :slight_smile:


(Bhavya R M) #8

Yep it is possible to migrate in 6.x manually.
https://www.elastic.co/guide/en/kibana/6.x/migrating-6.0-index.html

Good luck,
Cheers
Bhavya


#9

Hi,
Please allow me to confirm some things with you.

  • The document you mention is describing how to migrate Kibana index to 6.0.
  • The situation i encounter is i have DELETE OLD KIBANA index, and wanna a new installation kibana 6.0.

Right now, i have no kibana index to reindex anymore. It seem it's not the solution what we are finding.

Thank you!


(Bhavya R M) #10

Hi,

.kibana in that example could be any index. Even if you have deleted .kibana index and if you bring over rest of your other indices without reindexing - the auto-recreated .kibana will fail the mapping.

If you have no data in 5.6.7(5.x) and all you want to do is to start from 6.x - then you can just start with a fresh install of elasticsearch and Kibana in 6.0. But if you have data in 5.6 - reindexing them is necessary for your ES/Kibana to come up.

Does that help? Or should I ask somebody else to look over this?

Thanks,
Bhavya


(Mark Walkom) #11

What does the mapping of the .kibana index look like now?


#12

Hi,

The .kibana index cannot create due to the error: Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [log, doc] in my first post.

The template kibana_index_template:.kibana of .kibana as below:

{
  "kibana_index_template:.kibana": {
    "order": 0,
    "index_patterns": [
      ".kibana"
    ],
    "settings": {
      "index": {
        "number_of_shards": "1"
      }
    },
    "mappings": {
      "doc": {
        "dynamic": "strict",
        "properties": {
          "type": {
            "type": "keyword"
          },
          "updated_at": {
            "type": "date"
          },
          "config": {
            "dynamic": true,
            "properties": {
              "buildNum": {
                "type": "keyword"
              }
            }
          },
          "index-pattern": {
            "properties": {
              "fieldFormatMap": {
                "type": "text"
              },
              "fields": {
                "type": "text"
              },
              "intervalName": {
                "type": "keyword"
              },
              "notExpandable": {
                "type": "boolean"
              },
              "sourceFilters": {
                "type": "text"
              },
              "timeFieldName": {
                "type": "keyword"
              },
              "title": {
                "type": "text"
              }
            }
          },
          "visualization": {
            "properties": {
              "description": {
                "type": "text"
              },
              "kibanaSavedObjectMeta": {
                "properties": {
                  "searchSourceJSON": {
                    "type": "text"
                  }
                }
              },
              "savedSearchId": {
                "type": "keyword"
              },
              "title": {
                "type": "text"
              },
              "uiStateJSON": {
                "type": "text"
              },
              "version": {
                "type": "integer"
              },
              "visState": {
                "type": "text"
              }
            }
          },
          "search": {
            "properties": {
              "columns": {
                "type": "keyword"
              },
              "description": {
                "type": "text"
              },
              "hits": {
                "type": "integer"
              },
              "kibanaSavedObjectMeta": {
                "properties": {
                  "searchSourceJSON": {
                    "type": "text"
                  }
                }
              },
              "sort": {
                "type": "keyword"
              },
              "title": {
                "type": "text"
              },
              "version": {
                "type": "integer"
              }
            }
          },
          "dashboard": {
            "properties": {
              "description": {
                "type": "text"
              },
              "hits": {
                "type": "integer"
              },
              "kibanaSavedObjectMeta": {
                "properties": {
                  "searchSourceJSON": {
                    "type": "text"
                  }
                }
              },
              "optionsJSON": {
                "type": "text"
              },
              "panelsJSON": {
                "type": "text"
              },
              "refreshInterval": {
                "properties": {
                  "display": {
                    "type": "keyword"
                  },
                  "pause": {
                    "type": "boolean"
                  },
                  "section": {
                    "type": "integer"
                  },
                  "value": {
                    "type": "integer"
                  }
                }
              },
              "timeFrom": {
                "type": "keyword"
              },
              "timeRestore": {
                "type": "boolean"
              },
              "timeTo": {
                "type": "keyword"
              },
              "title": {
                "type": "text"
              },
              "uiStateJSON": {
                "type": "text"
              },
              "version": {
                "type": "integer"
              }
            }
          },
          "url": {
            "properties": {
              "accessCount": {
                "type": "long"
              },
              "accessDate": {
                "type": "date"
              },
              "createDate": {
                "type": "date"
              },
              "url": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 2048
                  }
                }
              }
            }
          },
          "server": {
            "properties": {
              "uuid": {
                "type": "keyword"
              }
            }
          },
          "timelion-sheet": {
            "properties": {
              "description": {
                "type": "text"
              },
              "hits": {
                "type": "integer"
              },
              "kibanaSavedObjectMeta": {
                "properties": {
                  "searchSourceJSON": {
                    "type": "text"
                  }
                }
              },
              "timelion_chart_height": {
                "type": "integer"
              },
              "timelion_columns": {
                "type": "integer"
              },
              "timelion_interval": {
                "type": "keyword"
              },
              "timelion_other_interval": {
                "type": "keyword"
              },
              "timelion_rows": {
                "type": "integer"
              },
              "timelion_sheet": {
                "type": "text"
              },
              "title": {
                "type": "text"
              },
              "version": {
                "type": "integer"
              }
            }
          }
        }
      }
    },
    "aliases": {

    }
  }
}

(Mark Walkom) #13

What's the actual mapping of the index that is in Elasticsearch now look like? It should give hints as to what's happening.


#14

I've deleted the .kibana index, so there is no index mapping left.

GET /.kibana
{
  "error": {
    "root_cause": [
      {
        "type": "index_not_found_exception",
        "reason": "no such index",
        "index_uuid": "_na_",
        "resource.type": "index_or_alias",
        "resource.id": ".kibana",
        "index": ".kibana"
      }
    ],
    "type": "index_not_found_exception",
    "reason": "no such index",
    "index_uuid": "_na_",
    "resource.type": "index_or_alias",
    "resource.id": ".kibana",
    "index": ".kibana"
  },
  "status": 404
}

(Mark Walkom) #15

Check _cat/indices, If there is no .kibana then restart Kibana and it should create the index and the correct mapping.

If you run into the same error then check the mapping and then post it here so we can take a closer look.


#16

Hi,

I checked _cat/indices and there is no .kibana index in ES.
I restart kibana and try to create my first index pattern.

There is a HTTP 400 error in browser,


(Mark Walkom) #17

Does _cat/indices show a .kibana? What does that mapping look like?
What do the Kibana and Elasticsearch logs show?


#18

curl -XGET http://localhost:9200/_cat/indices | grep '\.kibana' not returned any result, therefore the .kibana index has not been created.

ES error

es-master-0 es-master [2018-01-30T07:52:20,577][WARN ][o.e.d.a.a.i.t.p.PutIndexTemplateRequest] Deprecated field [template] used, replaced by [index_patterns]

Kibana error

{"type":"response","@timestamp":"2018-01-30T07:54:00Z","tags":[],"pid":1,"method":"post","statusCode":400,"req":{"url":"/api/saved_objects/index-pattern?overwrite=false","method":"post","headers":{"host":"localhost:8001","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36","content-length":"74","accept":"application/json, text/plain, */*","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,zh-TW;q=0.8,zh;q=0.7","cache-control":"no-cache","content-type":"application/json;charset=UTF-8","kbn-version":"6.1.2","origin":"http://localhost:8001","pragma":"no-cache","referer":"http://localhost:8001/api/v1/proxy/namespaces/default/services/kibana:web/app/kibana","x-forwarded-for":"127.0.0.1, 125.227.160.187","x-forwarded-uri":"/api/v1/proxy/namespaces/default/services/kibana:web/api/saved_objects/index-pattern"},"remoteAddress":"10.128.0.6","userAgent":"10.128.0.6","referer":"http://localhost:8001/api/v1/proxy/namespaces/default/services/kibana:web/app/kibana"},"res":{"statusCode":400,"responseTime":697,"contentLength":9},"message":"POST /api/saved_objects/index-pattern?overwrite=false 400 697ms - 9.0B"}
{"type":"ops","@timestamp":"2018-01-30T07:54:00Z","tags":[],"pid":1,"os":{"load":[0.77685546875,0.96240234375,0.99560546875],"mem":{"total":27402412032,"free":6320754688},"uptime":534753},"proc":{"uptime":2352.283,"mem":{"rss":95854592,"heapTotal":66719744,"heapUsed":57941368,"external":437709},"delay":0.3062770366668701},"load":{"requests":{"5601":{"total":1,"disconnects":0,"statusCodes":{"400":1}}},"concurrents":{"5601":1},"responseTimes":{"5601":{"avg":697,"max":697}},"sockets":{"http":{"total":0},"https":{"total":0}}},"message":"memory: 55.3MB uptime: 0:39:12 load: [0.78 0.96 1.00] delay: 0.306"}

It seems the template created by kibana is not apply change to the new mapping yet like the guess at my first post?

Thank you!


(Mark Walkom) #19

I'm a little lost, cause if there is no .kibana and you start Kibana, then it'll create that index automatically. It should mention that in the logs.

That is a warning, not an error.

What is this?


#20

Yes, I'm confused too.
I understand Kibana should create the .kibana index when it starts.
This is the weirdest part, no .kibana index had been created and no error message at logging.

I past more detail log for your information.

kibana startup log:
Kibana startup log (pastbin)

to check if existed .kibana

# Check 1
curl http://elasticsearch-internal:9200/.kibana
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":".kibana","index_uuid":"_na_","index":".kibana"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":".kibana","index_uuid":"_na_","index":".kibana"},"status":404}

# Check 2
curl http://elasticsearch-internal:9200/_cat/indices | grep '\.kibana'
(no result)

BTW, i use reverse proxy and configured server path, so you will see http://localhost:8001/api/v1/proxy/namespaces/default/services/kibana:web/app/kibana.

My docker run with options and environment variables:

  • --verbose
  • SERVER_BASEPATH="/api/v1/proxy/namespaces/default/services/kibana:web/app/kibana"
  • KIBANA_INDEX=".kibana"

Docker Image: docker.elastic.co/kibana/kibana-oss:6.1.2

Thanks!