# Duplicate field during reindex

**URL:** <https://discuss.elastic.co/t/duplicate-field-during-reindex/193534>\
**Category:** Elasticsearch\
**Created:** [August 2, 2019, 12:33pm UTC](https://discuss.elastic.co/t/duplicate-field-during-reindex/193534 "2019-08-02T12:33:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![GabrielM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabrielm/32/47827_2.png) [@GabrielM](https://discuss.elastic.co/u/GabrielM)\
**Post date:** [August 2, 2019, 12:33pm UTC](https://discuss.elastic.co/t/duplicate-field-during-reindex/193534/1 "2019-08-02T12:33:04Z")

</div>

Hello ! I am doing a reindex from a 1.3.4 cluster to a 7.1.1 cluster but I have an error during the reindex :

```
curl -X POST "$ES/_reindex?wait_for_completion=true" -H 'Content-Type: application/json' -d'
{                
  "source": {
    "remote": {
      "host": "http://42.42.42.42:9200"
    },
    "index": "xxxxxx-2017-09",
    "size": 1000
  },
  "dest": {
    "index": "xxxxxx-2017-09"
  },
  "script": {
    "lang": "painless",
    "source": "ctx._id=null"
  }
}
'

{
  "error": {
    "root_cause": [
      {
        "type": "exception",
        "reason": "Error parsing the response, remote is likely not an Elasticsearch instance"
      }
    ],
    "type": "exception",
    "reason": "Error parsing the response, remote is likely not an Elasticsearch instance",
    "caused_by": {
      "type": "x_content_parse_exception",
      "reason": "[1:157528] [search_response] failed to parse field [hits]",
      "caused_by": {
        "type": "x_content_parse_exception",
        "reason": "[1:157528] [hits] failed to parse field [hits]",
        "caused_by": {
          "type": "x_content_parse_exception",
          "reason": "[1:157528] [hit] failed to parse field [_source]",
          "caused_by": {
            "type": "parsing_exception",
            "reason": "[hit] failed to parse [_source]",
            "line": 1,
            "col": 157528,
            "caused_by": {
              "type": "json_parse_exception",
              "reason": "Duplicate field 'text'\n at [Source: org.apache.http.nio.entity.ContentInputStream@3f28eba4; line: 1, column: 157700]",
              "suppressed": [
                {
                  "type": "illegal_state_exception",
                  "reason": "Failed to close the XContentBuilder",
                  "caused_by": {
                    "type": "i_o_exception",
                    "reason": "Unclosed object or array found"
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "status": 500
}

```

Apparently the problem comes from a duplicated field.

Do you know how can I get those documents with duplicated field and maybe how to delete them please ?

Thank you and have a good day !

PS: I did the same reindex operation for other indices on the same cluster and everything goes fine ☹

---

<div class="post-metadata">

**Author:** ![mihalikv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mihalikv/32/26793_2.png) [@mihalikv](https://discuss.elastic.co/u/mihalikv)\
**Post date:** [August 2, 2019, 12:38pm UTC](https://discuss.elastic.co/t/duplicate-field-during-reindex/193534/2 "2019-08-02T12:38:32Z")

</div>

Hi, I search little bit. Maybe manually define \_source.include ?

> [@Reindex 5.4.0 to 7.1.1](https://discuss.elastic.co/t/reindex-5-4-0-to-7-1-1/186512/4):
>
> I figured it out. I had to remove the \_type, but that still threw the exception. So i also had to be explicit about what i wanted to include. POST \_reindex?wait\_for\_completion=false { "source": { "remote": { "host": "http://training-es5.videotel-stage.com:9200" }, "index": "cts-training", "\_source": { "include": ["key", "origin", "originKey", "generationDate", "who", "when", "where", "what", "verified"] }, "query": { "bool": {"must\_not":…

---

<div class="post-metadata">

**Author:** ![GabrielM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabrielm/32/47827_2.png) [@GabrielM](https://discuss.elastic.co/u/GabrielM)\
**Post date:** [August 2, 2019, 1:30pm UTC](https://discuss.elastic.co/t/duplicate-field-during-reindex/193534/4 "2019-08-02T13:30:15Z")

</div>

Hi, thank you for your quick response !

I did this request (with the include) :

```
curl -X POST "$ES/_reindex?wait_for_completion=true" -H 'Content-Type: application/json' -d'
{                
  "source": {
    "remote": {
      "host": "http://42.42.42.42:9200"
    },
    "index": "xxxxxxxxxxxx-2017-09",
    "size": 1000,
    "_source": {
      "include": [
        "xxxxxxxxxxxx",
        "text",
        "xxxxxxxxxxxx"
      ]
    }
  },
  "dest": {
    "index": "xxxxxxxxxxxx-2017-09"
  },
  "script": {
    "source": "ctx._id=null;"
  }
}
'

```

And no error !

But I can't query the content of the index anymore.

The index is not empty :

```
health status index docs.count store.size
green open xxxxxxxxxxxx-2017-09 2424000 4.5gb

```

But returns zero doc :

```
curl -X GET -H 'Content-Type: application/json' "$ES/xxxxxxxxxxxx-2017-09/_search"

{"took":4,"timed_out":false,"_shards":{"total":12,"successful":12,"skipped":0,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]}}

```

Do you know why please ?

---

<div class="post-metadata">

**Author:** ![GabrielM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabrielm/32/47827_2.png) [@GabrielM](https://discuss.elastic.co/u/GabrielM)\
**Post date:** [August 5, 2019, 7:09am UTC](https://discuss.elastic.co/t/duplicate-field-during-reindex/193534/5 "2019-08-05T07:09:30Z")

</div>

Nvm, it works. I tried again and after a while the data is here. Don't know how.

Thank you very much !

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 2, 2019, 7:09am UTC](https://discuss.elastic.co/t/duplicate-field-during-reindex/193534/6 "2019-09-02T07:09:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.