Reindex from remote cluster and index names with special characters

Elastic Stack Elasticsearch
1

Hello all, we're reindexing from a remote cluster to a new one, the issue I'm having is that the source index name has special characters and escaping them doesn't seem to work.

Here is the command I'm running:

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://old-cluster:9200"
    },
    "index": "winlogbeat-%{[@metadata][version]}-2018.10"
  },
  "dest": {
    "index": "winlogbeat-7.3.1-2018.10"
  }
}

With the output being:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Malformed escape pair at index 12: /winlogbeat-%{[@metadata][version]}-2018.10/_search"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Malformed escape pair at index 12: /winlogbeat-%{[@metadata][version]}-2018.10/_search",
    "caused_by": {
      "type": "u_r_i_syntax_exception",
      "reason": "Malformed escape pair at index 12: /winlogbeat-%{[@metadata][version]}-2018.10/_search"
    }
  },
  "status": 400
}

I can try escaping some of the special characters but it I just get the "Unrecognized character escape" error when I do.

Can anyone help?

Thanks.

2

By using an online URL encoder/decoder I was able to convert it to the format Elasticsearch would accept so:

winlogbeat-%{[@metadata][version]}-2018.10

converts to:

winlogbeat-%25%7B%5B%40metadata%5D%5Bversion%5D%7D-2018.10

However, when I run the reindex from remote operation I get the following output:

.....................................

"failures": [
    {
      "index": "winlogbeat-7.3.1-2018.10",
      "type": "_doc",
      "id": "5ur7d2YBim9jPLH0rFGq",
      "cause": {
        "type": "mapper_parsing_exception",
        "reason": "object mapping for [host] tried to parse field [host] as object, but found a concrete value"
      },
      "status": 400
    },

....................................................................

The source index was created with an older version of Winlogbeat and from looking at other posts I can see that it might be a ECS compatibility problem - what are my options in that case?

Thanks.

3

Bump - anyone?

4

Looking at the documentation you can change the name of a field during reindexing:

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-change-name

So to change 'host' to 'host.name' the following seems to have worked for me:

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://10.166.95.57:9200"
    },
    "index": "winlogbeat-%25%7B%5B%40metadata%5D%5Bversion%5D%7D-2018.10"
  },
  "dest": {
    "index": "winlogbeat-7.3.1-2018.10"
  },
  "script": {
    "source": "ctx._source['host.name'] = ctx._source.remove(\"host\")"
    }
}
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.