Elasticsearch 7 mappping on Elasticsearch 6

Elastic Stack Elasticsearch
1

Hello,

is it possible to use Elasticsearch 7 mapping (with no types) on any version of Elasticsearch 6?
For instance this mapping:

{
  "mappings":{
    "properties":{
      "serviceName":{
        "type":"keyword",
        "ignore_above":256
      },
      "operationName":{
        "type":"keyword",
        "ignore_above":256
      }
    }
  }
}

Another question: Will the mapping with _doc type work on any Elasticsearch 8 version? If it would work it would allow us to use new mapping for ES 8 and wait until old indices with _doc TTLout.

{
  "mappings":{
    "_doc": {
      "properties":{
        "serviceName":{
          "type":"keyword",
          "ignore_above":256
        },
        "operationName":{
          "type":"keyword",
          "ignore_above":256
        }
      }
    }
  }
}
2

Welcome!

I think that in 6.x you can use the include_type_name=false option to have a similar behavior as you would find with 7.x.

Have a look at https://www.elastic.co/guide/en/elasticsearch/reference/6.8/removal-of-types.html.

3

Thanks for quick reponse. Creating the template with include_type_name=false worked well.

However now there is problem with writing the data. We use bulk API to store data. Is there a way to use typeless API with ES 6.8 and bulk API?

Removal of mapping types | Elasticsearch Guide [master] | Elastic says

Typeless document APIs such as bulk and update are only available as of 7.0, and will not work with 6.8 nodes. This also holds true for the typeless versions of queries that perform document lookups, such as terms .

4

I honestly don't know.

Can't you upgrade your cluster to 7.4?

5

Can you ask around to find out?

I am still interested to know whether the following mapping will work with Elasticsearch 8.

{
  "mappings":{
    "_doc": {
      "properties":{
        "serviceName":{
          "type":"keyword",
          "ignore_above":256
        },
        "operationName":{
          "type":"keyword",
          "ignore_above":256
        }
      }
    }
  }
}

We are working on OSS project Jaeger which uses Elasticsearch as backend storage. Our goal is to provide wide compatibility with Elasticsearch versions.

6

In my personal project, I had to implement multiple ways to support different major versions.

I am still interested to know whether the following mapping will work with Elasticsearch 8.

Probably not. It will need to be something like:

{
  "mappings":{
      "properties":{
        "serviceName":{
          "type":"keyword",
          "ignore_above":256
        },
        "operationName":{
          "type":"keyword",
          "ignore_above":256
        }
      }
    }
}
7

Thanks for the clarification @dadoonet!

I didn't understand well how the Elasticsearch upgrades work in general. Please double check if I got it right now.

Elasticsearch node defines:

    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"

The wire compatibility defines the minimal version this node can communicate with. The index compatibility defines the minimal version this node can read data from. In this case this node (ES7) can read indices created by 6.x and can be deployed in a mixed cluster with ES 6.8 nodes.

ES6 can read and write to indices created by ES5 and ES7 can write and read from indices created by ES6. Therefore users can upgrade from ES6 to ES7 without any migration/reindex. If they are using ES5 and want to upgrade to ES7 they have to go through ES6 and wait until old indices TTLout or explicitly reindex.

In our case we want to provide compatibility with ES 5,6,7 so we have to use different mappings for different versions, however users can upgrade ES version without any additional work as described above.

  • ES 5, 6 use typed indices:
  "mappings":{
    "dynamic_templates":[
      {
        "span_tags_map":{
          "mapping":{
            "type":"keyword",
            "ignore_above":256
          },
          "path_match":"tag.*"
        }
      },
    "properties":{
      "serviceName":{
        "type":"keyword",
        "ignore_above":256
      },
      "operationName":{
        "type":"keyword",
        "ignore_above":256
      }
    }
  }
  • S 7 use typeless mappings:
{
  "mappings":{
    "properties":{
      "serviceName":{
        "type":"keyword",
        "ignore_above":256
      },
      "operationName":{
        "type":"keyword",
        "ignore_above":256
      }
    }
  }
}
8

The problem is the API.

Even though you can start a 7.x node on 6.x data path, you can not send 6.x API requests to a 7.x node.

So in short, if you have a cluster running a 7.x version, you need to comply with the 7.x API.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.