Can't reindex from Remote [Solved]

Elastic Stack Elasticsearch
1

Hi,
I'm trying to use the reindex api from remote.
I add to my locale elasticsearch.yml the ip and the port from remote source in reindex.remote.whitelist
Then i retrieve the mapping of my index from remote and I have been created a new index in locale with that mapping.

Then I try this query from kibana in locale to reindex from remote:

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://xx.xx.x.xxx:9200",
      "username": "xxxx",
      "password": "xxxx"
    },
    "index": "news_crawler",
    "query": {
      "term": {
        "_type": "news"
      }
    }
  },
  "dest": {
    "index": "news_crawler",
    "type": "news"
  }
}

That it returns to me an error:

{
  "took": 26609,
  "timed_out": false,
  "total": 10784,
  "updated": 0,
  "created": 0,
  "deleted": 0,
  "batches": 1,
  "version_conflicts": 0,
  "noops": 0,
  "retries": {
    "bulk": 0,
    "search": 0
  },
  "throttled_millis": 0,
  "requests_per_second": -1,
  "throttled_until_millis": 0,
  "failures": [
    {
      "index": "news_crawler",
      "type": "news",
      "id": "1vOL2GUBUaaJStn3MZEg",
      "cause": {
        "type": "illegal_argument_exception",
        "reason": "Rejecting mapping update to [news_crawler] as the final mapping would have more than 1 type: [news, _doc]"
      },
      "status": 400
    },
    {
      "index": "news_crawler",
      "type": "news",
      "id": "sfOK2GUBUaaJStn3pZGI",
      "cause": {
        "type": "illegal_argument_exception",
        "reason": "Rejecting mapping update to [news_crawler] as the final mapping would have more than 1 type: [news, _doc]"
      },
      "status": 400
    },
    {
      "index": "news_crawler",
      "type": "news",
      "id": "wfOK2GUBUaaJStn34pEq",
      "cause": {
        "type": "illegal_argument_exception",
        "reason": "Rejecting mapping update to [news_crawler] as the final mapping would have more than 1 type: [news, _doc]"
      },
      "status": 400
    }, ....

But my index in remote doesn't have more than one doc_type. In fact if I query my remote kibana in this way:

GET news_crawler/_search
{
  "size": 0,
  "aggs": {
    "myagg": {
      "terms": {
        "field": "_type",
        "size": 10
      }
    }
  }
}

It returns to me that I have only one doc_type in index:

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 4,
    "successful": 4,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 10784,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "myagg": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "news",
          "doc_count": 10784
        }
      ]
    }
  }
}

If in locale i query GET news_crawler/mapping it result that I have declared only one doc_type - news . Could someone helps me?

2

Hi Geppo,

So the index that you are newly creating includes the new default type _doc, in your case, you can map the news type to _doc by adding a script to the reindex request like this:

POST /_reindex
{
  "source": {
    "remote": {
      "host": "http://xx.xx.x.xxx:9200",
      "username": "xxxx",
      "password": "xxxx"
    },
    "index": "news_crawler",
    "query": {
      "term": {
        "_type": "news"
      }
    }
  },
  "dest": {
    "index": "news_crawler",
    "type": "news"
  },
  "script": {
    "source": "ctx._type = \"_doc\"",
    "lang": "painless"
  }
}
3

Hi dakrone, It was my error. My co-worker has created a new template, without talking to me, that matches "*" in index name and create a _doc datatype. I have disabled it and now it works fine

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.