Kibana shows blank sites in Discover, Management, ... after deleting/creation of default index pattern

[Manuel16 (Manuel)]

Hi all,

currently we have a problem with Kibana: After deleting the default index pattern in Kibana, we are not able to create this default pattern again. Our default pattern is called "packets-", if we try to create it again our Kibana shows only "blank" sites in Discover, Visualize, Dashboards, Management and Dev-Tools. If we try to create the index named "packets-2016-" or "packets-2017-*" it will works fine. Our migration from Kibana 4.6 to 5.0 is already a few months ago and worked without problems.

With the default index pattern "packets-*" we receive only "blank" sites with the following error message in browser debugger:

Error: Request to Elasticsearch failed: "Bad Request"
KbnError@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:63:26224
errors.RequestFailure@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:63:27413
exports.default/</<@http://127.0.0.1:5601/bundles/kibana.bundle.js?v=14438:26:14736
processQueue@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:38:23621
scheduleProcessQueue/<@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:38:23888
$RootScopeProvider/this.$get</Scope.prototype.$eval@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:39:4607
$RootScopeProvider/this.$get</Scope.prototype.$digest@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:39:2343
$RootScopeProvider/this.$get</Scope.prototype.$apply@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:39:5026
done@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:37:25016
completeRequest@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:37:28702
createHttpBackend/</xhr.onload@http://127.0.0.1:5601/bundles/commons.bundle.js?v=14438:37:29634
1 commons.bundle.js:38:11454
consoleLog/< http://127.0.0.1:5601/bundles/commons.bundle.js:38:11454
$ExceptionHandlerProvider/this.$get</< http://127.0.0.1:5601/bundles/commons.bundle.js:37:17828
processQueue http://127.0.0.1:5601/bundles/commons.bundle.js:38:23742
scheduleProcessQueue/< http://127.0.0.1:5601/bundles/commons.bundle.js:38:23888
$RootScopeProvider/this.$get</Scope.prototype.$eval http://127.0.0.1:5601/bundles/commons.bundle.js:39:4607
$RootScopeProvider/this.$get</Scope.prototype.$digest http://127.0.0.1:5601/bundles/commons.bundle.js:39:2343
$RootScopeProvider/this.$get</Scope.prototype.$apply http://127.0.0.1:5601/bundles/commons.bundle.js:39:5026
done http://127.0.0.1:5601/bundles/commons.bundle.js:37:25016
completeRequest http://127.0.0.1:5601/bundles/commons.bundle.js:37:28702
createHttpBackend/</xhr.onload http://127.0.0.1:5601/bundles/commons.bundle.js:37:29634

It seems that this problem is related to this two Issues:

• Empty dashboard after migrating from 4.6 to 5
• Allow to redefine the default index when not found

Some additional information:

• Kibana version: 5.0.0
• Elasticsearch version: 5.0.0
• Server OS version: Ubuntu 16.04.1 LTS x64
• Browser version: Chrome Version 54.0.2840.100 (64-bit)
• Browser OS version: Mac OS
• Original install method (e.g. download page, yum, from source, etc.): kibana-5.0.0-linux-x86_64.tar.gz

Seems to be a new bug... any idea how to rebuild our default index pattern in Kibana? We tried to delete the whole ".kibana" index manually but without success.

Many thanks!

Regards,
Manuel

[tylersmalley]

Hi Manuel,

I went through the entire upgrade process from ES 2.4.2 to 5.0.0 in an attempt to reproduce this issue. I was able to successfully delete the default index and recreate it.

You mentioned a couple times that your default index pattern is called "packets-". Can you verify you meant "packets-*"

Looking over the issues you linked, is it possible that the default index is in fact setup and pointing to an index pattern which does not exist?

What are the contents of the config within the .kibana index? You can access this via http://localhost:9200/.kibana/config/_search?pretty

[Manuel16 (Manuel)]

Hi Tyler,

thanks for your response. Sorry for the confusion, our default index pattern is called packets-*. The content of .kibana index shows:

{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [
{
"_index" : ".kibana",
"_type" : "config",
"_id" : "5.0.0",
"_score" : 1.0,
"_source" : {
"buildNum" : 14438,
"defaultIndex" : "packets-*"
}
}
]
}
}

You are right, it looks like that the the index pattern does not exist. But the index / data exists. I can also create index pattern like packets-2016-* or packets-2017-* and it works. I can also access the index pattern directly in Elastic via Curl: GET /packets-*/_search?pretty:

{
"took": 145,
"timed_out": false,
"_shards": {
"total": 85,
"successful": 85,
"failed": 0
},
"hits": {
"total": 200422756,
"max_score": 1,
"hits": [
{
...

We have currently more than 5000 fields in that index. Is there a limitation in Kibana on the maximum amount of fields?

Regards,
Manuel

[tylersmalley]

Can you take a look at Console in your browser to see if there are any errors? If you need help finding it, just let me know which browser you are using.

[Manuel16 (Manuel)]

Yes, this is the error:

[tylersmalley]

I am trying to figure out what that the purpose of that request is, as fetching of data is done with _mget of _msearch requests and this is a single POST. Can you also confirm that Kibana is not pointing to a Tribe node?

Can you find the failing request under the network tab and view the source of the request?

Here is what that looks like:

[Manuel16 (Manuel)]

Kibana isn't pointing to a trible node but I could fixed the problem. It seems to be that there was a problem with to many fields? I saved a part of the data from elastic, after that I was able to create the default pattern again. Finally I reindexed the saved data and it works. So, for now I'm not able to reprocedure the error/problem.