Elasticsearch Mappings from version 5 to version 7

Elastic Stack Elasticsearch
1

Hello Team

I am using Elasticsearch version 5.6.3 , cluster (A).

I am planning to move to Elasticsearch version 7.8.0 using separate cluster (B) for it.

I am using _reindex api to move my index data from A to B cluster and version.

Before using _reindex , I am creating Indexes as per Cluster A mapping and Settings.

Could you please help me how should i remove TYPE NAME from
curl -XGET "localhost:9200/myindex/_mapping" . output of Elasticsearch version 5.6.3.

As my index creation based on existing mappings with type name is failing due to changes in version 7.8.0

I am looking for some option which should help me do it dynamically and my script could work ...

Could you please help me ...

2

Use _doc instead of the original type.

Please do, 5.x has been EOL for a number of years. I would suggest going to 7.14 at least though, 7.8 will be EOL at the end of this year.

3

Hi Mark

I understand regarding TYPE changes.

I am asking , How to remove OLD TYPE Name from ES version 5.6.3 while moving to ES Version 7.8.0.

I need to move 100s of indices from OLD version to new version. I have to create mappings .. while creating mappings .. it gives me error message.

4

It'd help if you shared your mapping that is erroring, as well as the error.

5

Below is the statement of mapping and settings derived from existing Index of version 5.6.3

curl -X PUT "localhost:9200/myindex?pretty" -H 'Content-Type: application/json' -d'
 {
 "settings": {
  "index": {
    "codec": "best_compression",
    "routing": {
      "allocation": {
        "require": {
          "box_type": "warm"
        }
      }
    },
    "number_of_shards": "5",
    "blocks": {
      "read_only_allow_delete": "true"
    }
  }
} ,
 "mappings": {
  "mytype": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "actconn": {
        "type": "text",
...
....

ERROR while creating same in Elasticsearch version 7.8.0 ==>

This is expected as in ES v5.6.3 we have mapping defined as"mytype" , while in ES 7.8.0 its removed.

My question is how should i remove "mytype" while getting STATEMENT generated from existing indices of v5.6.3 as /_mapping includes TYPE NAME for v5.6.3

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Root mapping definition has unsupported parameters:  [mytype : {properties={date={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tt={type=text, fields={keyword={ignore_above=256, type=keyword}}}, request={type=text, fields={keyword={ignore_above=256, type=keyword}}}, actconn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, ps={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tw={type=text, fields={keyword={ignore_above=256, type=keyword}}}, srv_conn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, f_end={type=text, fields={keyword={ignore_above=256, type=keyword}}}, pid={type=text, fields={keyword={ignore_above=256, type=keyword}}}, hostname={type=text, fields={keyword={ignore_above=256, type=keyword}}}, beconn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, host={type=text, fields={keyword={ignore_above=256, type=keyword}}}, req_headers={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tag={type=text, fields={keyword={ignore_above=256, type=keyword}}}, b_server={type=text, fields={keyword={ignore_above=256, type=keyword}}}, feconn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, privateip={type=text, fields={keyword={ignore_above=256, type=keyword}}}, srv_queue={type=text, fields={keyword={ignore_above=256, type=keyword}}}, res_cookie={type=text, fields={keyword={ignore_above=256, type=keyword}}}, req_cookie={type=text, fields={keyword={ignore_above=256, type=keyword}}}, c_ip={type=text, fields={keyword={ignore_above=256, type=keyword}}}, b_end={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tc={type=text, fields={keyword={ignore_above=256, type=keyword}}}, backend_queue={type=text, fields={keyword={ignore_above=256, type=keyword}}}, retries={type=text, fields={keyword={ignore_above=256, type=keyword}}}, @timestamp={type=date}, logName={type=text, fields={keyword={ignore_above=256, type=keyword}}}, bytes={type=text, fields={keyword={ignore_above=256, type=keyword}}}, status code={type=text, fields={keyword={ignore_above=256, type=keyword}}}, time={type=text, fields={keyword={ignore_above=256, type=keyword}}}, t_state={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tq={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tr={type=text, fields={keyword={ignore_above=256, type=keyword}}}}}]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [mytype : {properties={date={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tt={type=text, fields={keyword={ignore_above=256, type=keyword}}}, request={type=text, fields={keyword={ignore_above=256, type=keyword}}}, actconn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, ps={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tw={type=text, fields={keyword={ignore_above=256, type=keyword}}}, srv_conn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, f_end={type=text, fields={keyword={ignore_above=256, type=keyword}}}, pid={type=text, fields={keyword={ignore_above=256, type=keyword}}}, hostname={type=text, fields={keyword={ignore_above=256, type=keyword}}}, beconn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, host={type=text, fields={keyword={ignore_above=256, type=keyword}}}, req_headers={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tag={type=text, fields={keyword={ignore_above=256, type=keyword}}}, b_server={type=text, fields={keyword={ignore_above=256, type=keyword}}}, feconn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, privateip={type=text, fields={keyword={ignore_above=256, type=keyword}}}, srv_queue={type=text, fields={keyword={ignore_above=256, type=keyword}}}, res_cookie={type=text, fields={keyword={ignore_above=256, type=keyword}}}, req_cookie={type=text, fields={keyword={ignore_above=256, type=keyword}}}, c_ip={type=text, fields={keyword={ignore_above=256, type=keyword}}}, b_end={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tc={type=text, fields={keyword={ignore_above=256, type=keyword}}}, backend_queue={type=text, fields={keyword={ignore_above=256, type=keyword}}}, retries={type=text, fields={keyword={ignore_above=256, type=keyword}}}, @timestamp={type=date}, logName={type=text, fields={keyword={ignore_above=256, type=keyword}}}, bytes={type=text, fields={keyword={ignore_above=256, type=keyword}}}, status code={type=text, fields={keyword={ignore_above=256, type=keyword}}}, time={type=text, fields={keyword={ignore_above=256, type=keyword}}}, t_state={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tq={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tr={type=text, fields={keyword={ignore_above=256, type=keyword}}}}}]",
    "caused_by" : {
      "type" : "mapper_parsing_exception",
      "reason" : "Root mapping definition has unsupported parameters:  [mytype : {properties={date={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tt={type=text, fields={keyword={ignore_above=256, type=keyword}}}, request={type=text, fields={keyword={ignore_above=256, type=keyword}}}, actconn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, ps={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tw={type=text, fields={keyword={ignore_above=256, type=keyword}}}, srv_conn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, f_end={type=text, fields={keyword={ignore_above=256, type=keyword}}}, pid={type=text, fields={keyword={ignore_above=256, type=keyword}}}, hostname={type=text, fields={keyword={ignore_above=256, type=keyword}}}, beconn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, host={type=text, fields={keyword={ignore_above=256, type=keyword}}}, req_headers={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tag={type=text, fields={keyword={ignore_above=256, type=keyword}}}, b_server={type=text, fields={keyword={ignore_above=256, type=keyword}}}, feconn={type=text, fields={keyword={ignore_above=256, type=keyword}}}, privateip={type=text, fields={keyword={ignore_above=256, type=keyword}}}, srv_queue={type=text, fields={keyword={ignore_above=256, type=keyword}}}, res_cookie={type=text, fields={keyword={ignore_above=256, type=keyword}}}, req_cookie={type=text, fields={keyword={ignore_above=256, type=keyword}}}, c_ip={type=text, fields={keyword={ignore_above=256, type=keyword}}}, b_end={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tc={type=text, fields={keyword={ignore_above=256, type=keyword}}}, backend_queue={type=text, fields={keyword={ignore_above=256, type=keyword}}}, retries={type=text, fields={keyword={ignore_above=256, type=keyword}}}, @timestamp={type=date}, logName={type=text, fields={keyword={ignore_above=256, type=keyword}}}, bytes={type=text, fields={keyword={ignore_above=256, type=keyword}}}, status code={type=text, fields={keyword={ignore_above=256, type=keyword}}}, time={type=text, fields={keyword={ignore_above=256, type=keyword}}}, t_state={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tq={type=text, fields={keyword={ignore_above=256, type=keyword}}}, tr={type=text, fields={keyword={ignore_above=256, type=keyword}}}}}]"
    }
  },
  "status" : 400
}
6

That need to be;

 "mappings": {
  "_doc": {
7

Yes. I understand that ..

How to make it ... as I am having multiple indices ... and multiple types created ...

It would take long long time to remove all types manually and make it _doc

It would be better to not to include "TYPE NAME " while capturing existing mappings ... from version 5.6.3

8

There's nothing in Elasticsearch that will do that.
You'd need to download the mappings and use a scripting tool to do it.

9

Could you please help me by suggesting tool to ge through it.

I was searching same ... but was not able to get it , hence i raised in here the issue...

10

If you download the mappings and save them as json, you could use sed to do it on the CLI.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.