# Reindex in a cluster does not get all documents **URL:** \ **Category:** Elasticsearch\ **Created:** [June 19, 2019, 2:29pm UTC](https://discuss.elastic.co/t/reindex-in-a-cluster-does-not-get-all-documents/186492 "2019-06-19T14:29:16Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![M.alsioufi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.alsioufi/32/52269_2.png) [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)\ **Post date:** [June 19, 2019, 2:29pm UTC](https://discuss.elastic.co/t/reindex-in-a-cluster-does-not-get-all-documents/186492/1 "2019-06-19T14:29:16Z")
Elastic Stack Version: 6.5 I am trying to reindex my index "old\_index" to another index "new\_index" on the same cluster. ``` POST _reindex { "source": { "index": "old_index", "_source": { "include": [ "@timestamp", "@version", "field1", "field2", "field3" ] } }, "script": { "inline": """ctx._source.new_field1= ctx._source.field1;ctx._source.remove("field1");""" }, "dest": { "index": "new_index" } } ``` I have a cluster of 3 nodes, on "old\_index" I have +2M documents, however, on "new\_index" I am only getting around 500K. I have run > GET \_tasks?actions=\*reindex&detailed It shows reindexing is done! I am not sure where is all my data! Any help will be so appreciated! ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [June 19, 2019, 2:55pm UTC](https://discuss.elastic.co/t/reindex-in-a-cluster-does-not-get-all-documents/186492/2 "2019-06-19T14:55:05Z")
can you share the output of `GET _cat/indices/old_index,new_index?v`please? Can you also share the output of the indexing task or the output of the above command as you have not started it in the background? ---
**Author:** ![M.alsioufi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.alsioufi/32/52269_2.png) [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)\ **Post date:** [June 20, 2019, 8:12am UTC](https://discuss.elastic.co/t/reindex-in-a-cluster-does-not-get-all-documents/186492/3 "2019-06-20T08:12:00Z")
Running > GET \_tasks?actions=\*reindex&detailed While running the re-indexing code ``` { "nodes" : { "aOYQyOfCQiyOGT9HaQr8nQ" : { "name" : "SOME-NODE-NAME", "transport_address" : "some-ip:9300", "host" : "some-ip", "ip" : "some-ip:9300", "roles" : [ "data" ], "attributes" : { "ml.machine_memory" : "17096605696", "xpack.installed" : "true", "ml.max_open_jobs" : "20", "ml.enabled" : "true" }, "tasks" : { "aOYQyOfCQiyOGT9HaQr8nQ:91604987" : { "node" : "aOYQyOfCQiyOGT9HaQr8nQ", "id" : 91604987, "type" : "transport", "action" : "indices:data/write/reindex", "status" : { "total" : 2703761, "updated" : 0, "created" : 5000, "deleted" : 0, "batches" : 6, "version_conflicts" : 0, "noops" : 0, "retries" : { "bulk" : 0, "search" : 0 }, "throttled_millis" : 0, "requests_per_second" : -1.0, "throttled_until_millis" : 0 }, "description" : """reindex from [old_index] updated with Script{type=inline, lang='painless', idOrCode='ctx._source.new_field1= ctx._source.field1;ctx._source.remove("field1");', options={}, params={}} to [new_index]""", "start_time_in_millis" : 1561016910227, "running_time_in_nanos" : 2826725842, "cancellable" : true, "headers" : { } } } } } } ``` `some-ip` is the ip of the node that I am running reindexing on (the one hosting Kibana as I am running this on Kibana dev tools) Running the same request without having re-indexing in background, I got: ``` { "nodes" : { } } ``` Running: > GET \_cat/indices/old\_index,new\_index I got: ``` green open old_index kTzZiAggRMG0y5K4Xt7xzQ 5 2 2703795 528414 13.3gb 4.3gb green open new_index q123gbafShOSA4Vh45gC3g 5 2 504996 0 871.4mb 291.5mb ``` ---
**Author:** ![M.alsioufi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.alsioufi/32/52269_2.png) [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)\ **Post date:** [June 27, 2019, 1:22pm UTC](https://discuss.elastic.co/t/reindex-in-a-cluster-does-not-get-all-documents/186492/4 "2019-06-27T13:22:05Z")
I am still having this issue, any help is appreciated! ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [July 1, 2019, 9:05am UTC](https://discuss.elastic.co/t/reindex-in-a-cluster-does-not-get-all-documents/186492/5 "2019-07-01T09:05:43Z")
when the reindex is finished, how does the output of the task API look like? The output above shows that 5k documents have been processed, but I guess you have to process more as 500k have been indexed. Also, when you just specify a reindex without any script, does this copy all data? ---
**Author:** ![M.alsioufi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.alsioufi/32/52269_2.png) [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)\ **Post date:** [July 2, 2019, 9:48am UTC](https://discuss.elastic.co/t/reindex-in-a-cluster-does-not-get-all-documents/186492/6 "2019-07-02T09:48:52Z")
I have checked the logs and I found that there were a problem in the mappings that caused the re-indexing to stop at some point, so I fixed this issue by fixing the mappings of the index. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 30, 2019, 9:48am UTC](https://discuss.elastic.co/t/reindex-in-a-cluster-does-not-get-all-documents/186492/7 "2019-07-30T09:48:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.