That appears when clicking on the "No" button you can see in the screen (but it's appearing in other more important parts of the UI).
I checked DevTools in Chrome and I saw the request is failing with this message:
{
"message":"Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [doc, message]: [remote_transport_exception] [WARM_MASTER_MACHINES_NAME][MACHINES_IP:9300][indices:admin/create]",
"statusCode":400,
"error":"Bad Request"
}
I saw many people have asked about this and there are answers mentioning:
Do you have a .kibana index? No, we don't, I checked with _cat.
Do you have xpack enabled? I shouldn't have it as we don't use username+password authentication with ES. But out of excess of caution I threw xpack.security.enabled: false in kibana.yml with no luck. I also see some logs related to xpack , should I be concerned about them?
@bhavyarm In this post you mention one needs to reindex the data for this problem to get solved, though in my case we had ES 2, and we reindexed everything into ES 6.2.4. Sometime later we upgraded to 6.4.1 without reindexing all data as that didn't seem to be necessary. May that be an issue?
If you were already running on 6.2.4, then moving to 6.4.1 shouldn't have been an issue. The error you're getting should exist in all 6.x versions, since Elasticsearch 6.x only allows a single type for any index.
Everything Kibana writes uses doc as the type value, I'm not sure why you're getting a message type as well now. Kibana did, and I think still does, have some auto-migration stuff, and that's probably where the error is coming from.
Did you in the past, or do you right now, have any third party Kibana plugins you're using? Maybe that's where the documents with the message type are coming from?
@Joe_Fleming We don't have any 3rd party plugin for Kibana. This is actually a fresh install we are doing for testing what Kibana does. We are installing it on a different server and making it point to our ES 6.4.1 cluster.
Is there anything I should check for in Kibana's logs? Nothing catches my eye in there.
Or is there a way to debug that auto-migration utility for Kibana?
I read somewhere that Kibana should create a .kibana index, but Kibana is not creating that one either.
@Joe_Fleming We found some information in the logs about plugins that are being loaded, though not sure if any of them is 3rd party, as we have not attempted to install anything 3rd party on this machine. We are, though, using a Chef recipe to install Kibana, and now I'm wondering if this recipe installs by default something that can be causing issues.
We found some information in the logs about plugins that are being loaded, though not sure if any of them is 3rd party
All apps and even large parts of Kibana are actually plugins, so that's normal. If you didn't manually install plugins, then you don't have any 3rd party ones.
That said, the discovery-ec2 and repository-s3 are unfamiliar to me. I guess the easiest way to check if you have plugins installed is just to look in the plugins path in the root of Kibana. If the directory is empty, then you don't have any 3rd party plugins.
I read somewhere that Kibana should create a .kibana index, but Kibana is not creating that one either.
It will create it, but I believe it waits until you save something in Kibana. If you create an index pattern, for example, you'll have a .kibana index. I believe that regardless of the index existing, the index template is created when you start Kibana, and that would define what types exist in the index.
Something is trying to write a document to the .kibana index with "_type": "message", that's what is causing the error you see. But if it's not some other plugin, I'm not sure what would be doing that.
It's possible, but I think less likely, that the opposite is happening... that something else is creating the .kibana index and adding a document with "_type": "message", so when Kibana tries to write to the index, doc becomes the second type. But, if you don't have any plugins installed, I can't think why that would happen.
I've been looking into this issue with dwilches this week. I checked for 3rd party plugins, and confirmed we don't have any in kibana. It's a fresh install and not much is setup within it.
I am able to repro this from our es client directly (index pattern, master_node name and ip were removed):
$ curl -XPOST -H 'Content-Type: application/json' 'http://localhost:9200/.kibana/index-pattern/INDEXPATTERN-*' -d'{"title":"INDEXPATTERN-*","timeFieldName":"@timestamp","notExpandable":true}'
{"error":{"root_cause":[{"type":"remote_transport_exception","reason":"[MASTER_NODE][IP:9300][indices:admin/create]"}],"type":"illegal_argument_exception","reason":"Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [doc, message]"},"status":400}
It looks like it's using the template 'kibana_index_template:.kibana' within es according to the master logs, which is only set to use the doc type. I do see the message type used elsewhere though, both in our base template:
And the indexes that match the pattern I'm adding have "_type": "message" set. Could it be trying to add one of those in to the kibana index it's creating? The .kibana index doesn't exist yet, so it seems unlikely something outside of es is creating a document there.
Ah, I suspect it's the base_template you have. Since you're using index_patterns: ['*'], that's going to match the .kibana index. I'm no Elasticsearch expert, but I suspect that template causes that type to exist on the .kibana index, so then when you attempt to use Kibana, and the index is created and Kibana tries to write to it using the doc type, it fails because it already has a message type thanks to that template.
Looking in our docs about index templates, there's a section about multiple templates applying. It mentions that the types in the mappings are merged deeply, but doesn't talk about the requirement to only have a single type.
If you remove your base_template, does Kibana work? I suspect it would, but you'd need to verify that.
If I'm right about this, you have 2 options, at least as far as I can think up.
Option 1 - Be more exclusive with your "includes" pattern (using some prefix, like a company abbreviation or something on all your index names, elastic-* for example). This way it won't match any of our .-prefixed indices and you don't run into the multiple types issue.
Option 2 - Update the template, and the data you're indexing in Elasticsearch, to also use the doc type, instead of the message type. Again, since everything uses the same type, you don't run into the multiple types issue. You should be able to use the re-index api to update existing data with the new doc type if you need to.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
