How to move away from default document type mapping "doc"

Elastic Stack Elasticsearch
1

I discovered that the default template in 6.4.2 is currently setting the "doc" document_type via mapping.

The result of

GET /customer-environment-2018.11.16/_mapping

is

{
  "customer-environment-2018.11.16": {
    "mappings": {
      "doc": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "auth": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
		  ....
}

Originally, I needed to update the number of shards created by default. The answer to changing the number of shards used on index creation from the default of 5 is to use dynamic templates. https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-templates.html In order to create my dynamic template, I needed to copy the existing default mapping created by elasticsearch, then create my template from that with the shards adjusted to my need. Which I have setup and is working just fine.

However, concern comes up when digging further into the mappings type. Since, copying the default template to create the needed amount of shards, I am now explicitly mapping a document type of "doc". According to documentation cited in other questions in the forum, and all the research I can find related to mappings, is that mappings is being completely phased out by version 7.x/8.x.

My questions are:
-Do I wait until those versions release, have elasticsearch create a default index, pull that new template/mapping, then update my existing template to match?
-I've seen multiple solutions that don't seem to work with my situation. I am dynamically creating the index name from filebeat custom fields, then appending the date. It appears anything other than the "doc" document type also breaks the results that I get. I think this is where the issue arises, and where my understanding is lacking. When creating a template without mapping the document type to "doc", all of the fields are missing from the logs. I essentially end up with a datetime. What is the proper way to adjust the number of shards without breaking things from the default template? Is there something else tied to "doc" somewhere that I can update to completely remove that mapping?

I'd like to get ahead of this change so that it's not something I'm fighting when it's time to update. And, hopefully have a better understanding of what I have here, and the pieces of it that I'm not completely familiar with.

2

Anyone have an idea of how to approach this?

3

Bump

4

The removal of multiple mapping types is a long and drawn out process that will take all the way until version 9 to complete. It is happening in so many steps to avoid any one step causing too much heartache during the upgrade. To clarify, mappings are not being removed: it's vital that each index has a mapping. Support for multiple mapping types within a single index (in new indices) was removed in 6.x, and most of the remaining steps in the process are to do with removing the now-unnecessary type name in all the APIs etc. in which it appears.

In 6.4.2 the main thing to do is to prefer to use _doc as the type name instead of doc; in 7.0 _doc will effectively be the only permitted type name, and it will start to become optional.

The precise details of the changes in 7.0 are still being discussed, but one of the goals is to avoid doing anything that requires applications to be changed simultaneously with the Elasticsearch upgrade. The changes needed will be communicated using things like the deprecation log to catch any places that applications are using Elasticsearch 6.x in ways that won't work in 7.x.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.