Kibana server failed to start

Elastic Stack Kibana
1

Could you please help check below error?
Thank you in advance.

LOG

{"type":"log","@timestamp":"2019-10-10T08:10:06Z","tags":["info","migrations"],"pid":9,"message":"Creating index .kibana_1."}
{"type":"log","@timestamp":"2019-10-10T08:10:06Z","tags":**info","Sentinl","init_indices"],"pid":9,"message":"creating .kibana index ..."}
{"type":"log","@timestamp":"2019-10-10T08:10:07Z","tags":["info","migrations"],"pid":9,"message":"Pointing alias .kibana to .kibana_1."}
{"type":"log","@timestamp":"2019-10-10T08:10:07Z","tags":["info","Sentinl","init_indices"],"pid":9,"message":"checking watcher_alarms-2019.10.10 index ..."}
{"type":"log","@timestamp":"2019-10-10T08:10:07Z","tags":["info","Sentinl","init_indices"],"pid":9,"message":"creating watcher_alarms-2019.10.10 index ..."}
{"type":"error","@timestamp":"2019-10-10T08:10:07Z","tags":["fatal","root"],"pid":9,"level":"fatal","error":{"message":"[invalid_alias_name_exception] Invalid alias name [.kibana], an index exists with the same name as the alias, with { index_uuid="LU9AvG3BQ0av3eawyiPf6w" & index=".kibana" }","name":"Error","stack":"[invalid_alias_name_exception] Invalid alias name [.kibana], an index exists with the same name as the alias, with { index_uuid="LU9AvG3BQ0av3eawyiPf6w" & index=".kibana" } :: {"path":"/_aliases","query":{},"body":"{\"actions\":[{\"add\":{\"index\":\".kibana_1\",\"alias\":\".kibana\"}}]}","statusCode":400,"response":"{\"error\":{\"root_cause\":[{\"type\":\"invalid_alias_name_exception\",\"reason\":\"Invalid alias name [.kibana], an index exists with the same name as the alias\",\"index_uuid\":\"LU9AvG3BQ0av3eawyiPf6w\",\"index\":\".kibana\"}],\"type\":\"invalid_alias_name_exception\",\"reason\":\"Invalid alias name [.kibana], an index exists with the same name as the alias\",\"index_uuid\":\"LU9AvG3BQ0av3eawyiPf6w\",\"index\":\".kibana\"},\"status\":400}"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector. (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:165:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4949:19)\n at emitNone (events.js:111:20)\n at IncomingMessage.emit (events.js:208:7)\n at endReadableNT (_stream_readable.js:1064:12)\n at _combinedTickCallback (internal/process/next_tick.js:139:11)\n at process._tickCallback (internal/process/next_tick.js:181:9)"},"message":"[invalid_alias_name_exception] Invalid alias name [.kibana], an index exists with the same name as the alias, with { index_uuid="LU9AvG3BQ0av3eawyiPf6w" & index=".kibana" }"}

FATAL [invalid_alias_name_exception] Invalid alias name [.kibana], an index exists with the same name as the alias, with { index_uuid="LU9AvG3BQ0av3eawyiPf6w" & index=".kibana" } :: {"path":"/_aliases","query":{},"body":"{"actions":[{"add":{"index":".kibana_1","alias":".kibana"}}]}","statusCode":400,"response":"{"error":{"root_cause":[{"type":"invalid_alias_name_exception","reason":"Invalid alias name [.kibana], an index exists with the same name as the alias","index_uuid":"LU9AvG3BQ0av3eawyiPf6w","index":".kibana"}],"type":"invalid_alias_name_exception","reason":"Invalid alias name [.kibana], an index exists with the same name as the alias","index_uuid":"LU9AvG3BQ0av3eawyiPf6w","index":".kibana"},"status":400}"}

2

What version of Kibana are you running and where did you download it from? If you're upgrading, what version are you upgrading from?

Can you also provide the following results from elasticsearch:

3

Hi,
Thanks for your response.

Kibana version:
bash-4.2$ kibana --version
6.5.4

Elasticsearch version:
"version" : {
"number" : "6.5.4",

There is no upgrade.

Docker environment

curl -XGET https://10.254.13.99:9200/_cat/aliases/.kibana*

//no alias. it may be caused by pointing alias to index unsuccessfully.

curl -XGET https://10.254.13.99:9200/_cat/indices/.kibana*

green open .kibana_1 9YgQDYZ6QF23i5CC6PN0KQ 1 0 0 0 261b 261b
yellow open .kibana_viewer BNOMWT2cTDyFmw-R4EAIAA 5 1 360 0 643kb 643kb
yellow open .kibana LU9AvG3BQ0av3eawyiPf6w 5 1 376 0 879.3kb 879.3kb

Reproduce step

  1. helm install es
  2. helm install kibana
  3. kubectl logs kibana-pod, we will see above error.

Migration would like porints .kibana alias to .kibana_1 index, but an index exists with the same name as the alias.
Sentinl created .kibana index before migration point .kibana alias to .kibana_1 index.

LOG
["info","migrations"],"pid":9,"message":"Creating index .kibana_1."}
{"type":"log","@timestamp":"2019-10-10T08:10:06Z","tags":**info","Sentinl","init_indices"],"pid":9,"message":"creating .kibana index ..."}
{"type":"log","@timestamp":"2019-10-10T08:10:07Z","tags":["info","migrations"],"pid":9,"message":"Pointing alias .kibana to .kibana_1."}

Should migratioin check the alias before pointing to index?

4

Couple questions:

Are you using these helm charts? https://github.com/elastic/helm-charts/
Do you have something that also created a Kibana index?
Do you have a Kibana plugin you're using?
Where did .kibana_viewer from?

This is all pretty strange. If you're installing 6.5.4 fresh, than you shouldn't have a .kibana. Additionally, if you were coming from a previous version where you did have a .kibana, the migration would create .kibana_1, which was a copy, then .kibana_2 which was the migrations. Then we create the alias and atomically delete .kibana.

5

we made change in helm chart.

Sentinl plugin creates .kibana index.
pls see below log
{"type":"log","@timestamp":"2019-10-10T08:10:06Z","tags":**info","Sentinl","init_indices"],"pid":9,"message":"creating .kibana index ..."}

my question is .kibana index shoud be created by any plugin?

i will have check where .kibana_view comes from.

6

No, that's not going to work. A plugin shouldn't be creating the index, that's going to create all sorts of issues. We provide an API for plugins to persist data through Elasticsearch, or they can use our import/export API.

7

Thanks for your information.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.