How to remove duplicate fields within doc?


Trying to migrate from 5.X to 6.X but have issues when reading ES data that was stored before the migration. The error I'm getting back is:

              {<<"reason">>,<<"Failed to parse content to map">>}]]},
           {<<"reason">>,<<"all shards failed">>},
                {<<"reason">>,<<"Failed to parse content to map">>},
                   <<"Duplicate field 'X_ID'\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@28e05854; line: 1, column: 2061]">>}]}]}]]}]},

I've resolved an issue in source code where duplicate fields (MY_ID) were being entered. But now that I have documents with duplicate fields, is there a way to remove those fields? Otherwise, I cannot read any documents that were added on 5.X.

Hi, you could try setting an _source.include parameter when reading data from ES 5.x.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.