Rejecting mapping update

Elastic Stack Beats
1

Hi,

I have upgraded my elasticsearch to 6.3.2. After an upgrade i am trying to reindex my data. But i am getting below error.

  "cause": {
    "type": "illegal_argument_exception",
    "reason": "Rejecting mapping update to [logstash-YYYY.DD.MM-reindexed] as the final mapping would have more than 1 type: [kube-logs, logs]"
  },
2

What is your Filebeat version?

3

Below is my Filebeat Configuration

filebeat.config.inputs:
enabled: true
path: "/etc/filebeat-additional/"
filebeat.inputs:

  • type: log
    paths: "/var/log/containers/*.log"
    tags: ["kube-logs","syslog"]
    symlinks: true
    json.message_key: log
    json.keys_under_root: true
    json.add_error_key: true
    multiline.pattern: '^\s'
    multiline.match: after
    output.logstash:
    hosts: ["${LOGSTASH_HOSTS:'logstash:5043'}"]
    timeout: 15
    logging:
    level: ${LOG_LEVEL:warning}
    processors:
  • drop_fields:
    fields: ["host"]
4

My filebeat version is 6.3.2

5

I am using below command for reindexing

POST /_reindex
{

"source": {

"index": "logstash-YYYY.MM.DD"

},

"dest": {

"index": "logstash-YYYY.MM.DD-reindexed"

}

}

6

What were the versions of Elasticsearch and Filebeat before you updated?

7

Before updating the version were 5.2.2

8

You have two types: kube-logs and logs. You need to create separate indices for each type.

9

Hi @kvch

My filebeat version is sending only one type. After upgrade I have restored the indexes. please could let me how to separate two indices?

Regards

10

Can anyone help me on the same for reindexing

11

So right now you have a 5.x index that contains two different types, and you want to reindex every type into a different new index.

I understand that the way of achieving this is to use the type filtering in the reindex API as explained here: Reindex API | Elasticsearch Guide [8.11] | Elastic

Something like:

POST _reindex
{
  "source": {
    "index": "logstash-YYYY.MM.DD",
    "type": "logs",
  },
  "dest": {
    "index": "logstash-YYYY.MM.DD-logs-reindexed"
  }
}

and

POST _reindex
{
  "source": {
    "index": "logstash-YYYY.MM.DD",
    "type": "kube-logs",
  },
  "dest": {
    "index": "logstash-YYYY.MM.DD-kube-logs-reindexed"
  }
}

You can also map all your documents to the same index, by removing the type information, as explained here:

Otherwise, you will get more authoritative answers in elasticsearch

1 Like
12

From elastic search 6 in each index we can have only one type.so create two index in new index.

13

I guess the below link has helped me a ton.

I tried creating the index with different type as it was mapped in my index. But was ending up in confusions.
But below reindex api helped me to have single t type.

POST _reindex
{
"source": {
"index": "old"
},
"dest": {
"index": "new"
},
"script": {
"source": """
ctx._source.type = ctx._type;
ctx._type = 'doc';
""",
"lang": "painless"
}
}

Thanks a ton again

14

True. But data of same date with multiple created confusion.

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.