# Elasticsearch 5.1.1 remote reindex process aborts without any error **URL:** \ **Category:** Elasticsearch\ **Created:** [January 19, 2017, 9:24pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222 "2017-01-19T21:24:21Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![hailima](https://avatars.discourse-cdn.com/v4/letter/h/f19dbf/32.png) [@hailima](https://discuss.elastic.co/u/hailima)\ **Post date:** [January 19, 2017, 9:24pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/1 "2017-01-19T21:24:21Z")
Hi All, I migrate my index from ES 1.4.4 to 5.1.1 using "\_reindex" API. The reindex process always aborts without any errors before finishing all records. Here are details: 1. Create a new index with settings in 5.1.1 ES server: ```auto (PUT) http://hostname5_1_1:9200/nexttextindex { "settings": { "index": { "number_of_replicas": "0", "number_of_shards": "5", "refresh_interval": "-1" } } } ``` 1. ```auto (POST) http://hostname5_1_1:9200/_reindex?wait_for_completion=false { "source": { "remote": { "host": "http://hostname1_4_4.com:9200" }, "index": "nexttextindex", "size": 10 }, "dest": { "index": "nexttextindex" } } ``` 1. Use task API to monitor the reindex process: ```auto (GET) http://hotname5_1_1:9200/_tasks?detailed=true&actions=*reindex { "nodes": { "H_cHXZN9SnqbdBNaew5wew": { "name": "hostname5_1_1", "transport_address": "10.169.167.203:9300", "host": "10.169.167.203", "ip": "10.169.167.203:9300", "roles": [ "master", "data", "ingest" ], "tasks": { "H_cHXZN9SnqbdBNaew5wew:198701": { "node": "H_cHXZN9SnqbdBNaew5wew", "id": 198701, "type": "transport", "action": "indices:data/write/reindex", "status": { "total": 5653044, "updated": 0, "created": 43350, "deleted": 0, "batches": 6, "version_conflicts": 0, "noops": 0, "retries": { "bulk": 0, "search": 0 }, "throttled_millis": 0, "requests_per_second": 0, "throttled_until_millis": 0 }, "description": "", "start_time_in_millis": 1484860595578, "running_time_in_nanos": 11292914938, "cancellable": true } } } } } ``` 1. As you can see from 3), everything looks good. But, After a couple of hours, the task is done and only about 60K records got indexed in ES 5.1.1 server instead of 5653044. Repeated the process a few times, it always aborted without any errors. Appreciate if any help! ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [January 19, 2017, 9:45pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/2 "2017-01-19T21:45:47Z")
You should be able to get the status of those reindex tasks with `GET /_tasks/` where the `` is whatever id was returned when you started. If you didn't store them then you should be able to look around with something like `GET .tasks/_search`. Those should contain the failure reason. Or, if it thinks it finished successfully, it should show you that. ---
**Author:** ![hailima](https://avatars.discourse-cdn.com/v4/letter/h/f19dbf/32.png) [@hailima](https://discuss.elastic.co/u/hailima)\ **Post date:** [January 19, 2017, 10:01pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/3 "2017-01-19T22:01:06Z")
Thanks for quick reply, Nick I used "GET /\_tasks/", But, it's not working. Here details: GET .. 9200/\_tasks/198701 { "error": { "root\_cause": [ { "type": "illegal\_argument\_exception", "reason": "malformed task id 198701" } ], "type": "illegal\_argument\_exception", "reason": "malformed task id 198701" }, "status": 400 } ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [January 19, 2017, 10:14pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/4 "2017-01-19T22:14:57Z")
You'll need the part before the `:` in the task id as well. If you don't have it then try the search I sent. You should wrap your code blocks in ``` so they are readable. ---
**Author:** ![hailima](https://avatars.discourse-cdn.com/v4/letter/h/f19dbf/32.png) [@hailima](https://discuss.elastic.co/u/hailima)\ **Post date:** [January 19, 2017, 10:15pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/5 "2017-01-19T22:15:19Z")
Ok, command "..9200/.tasks/\_search" is good one which gives me error info: 1. Error: ````auto "type": "process_cluster_event_timeout_exception", "reason": "failed to process cluster event (put-mapping) within 30s" }, "status": 503``` Any way to change the timeout (30s) to longer time? I have many different doc types inside the index and under each doc type, i have up to more than 1k docs. 2) If any error occurs during the reindexing process, is there any way to ignore the error and continue the process ? I have a few M records, it's time-consuming if restarting it again. It's Ok if I lose some data in the reindex process ```` ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [January 19, 2017, 10:30pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/6 "2017-01-19T22:30:39Z")
> [@hailima](#): > > Any way to change the timeout (30s) to longer time? Probably but I don't know if off hand. I think your are better off manually creating the mapping before running reindex. > [@hailima](#): > > If any error occurs during the reindexing process, is there any way to ignore the error and continue the process ? No. The isn't a place to store the errors so we never implemented this. If you want some protection against this maybe try reindexing in chunks? ---
**Author:** ![hailima](https://avatars.discourse-cdn.com/v4/letter/h/f19dbf/32.png) [@hailima](https://discuss.elastic.co/u/hailima)\ **Post date:** [January 19, 2017, 10:30pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/7 "2017-01-19T22:30:40Z")
tried followings with tasks API /\_tasks/210411 /\_tasks/210411:1 no of them is working, But, does not brother me since .tasks/\_search is good. The doc for this part is not clear. Where can I find out the descriptions for the status like 404 or 503? Thanks ---
**Author:** ![hailima](https://avatars.discourse-cdn.com/v4/letter/h/f19dbf/32.png) [@hailima](https://discuss.elastic.co/u/hailima)\ **Post date:** [January 19, 2017, 10:42pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/8 "2017-01-19T22:42:33Z")
We don't need to store the errors and just continue to reindex next records without aborting due to last failure ... Any setting to make it happen? What do you mean by "reindexing in chunks"? we are using "size" for batching, right? ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [January 19, 2017, 10:59pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/9 "2017-01-19T22:59:51Z")
> [@hailima](#): > > We don't need to store the errors and just continue to reindex next records without aborting due to last failure ... Any setting to make it happen? There isn't. > [@hailima](#): > > What do you mean by "reindexing in chunks"? Use a query to limit what you are reindexing to certain days or namespaces or something. Whatever natural division your data has. Then do it again and again until you migrate all the data. ---
**Author:** ![hailima](https://avatars.discourse-cdn.com/v4/letter/h/f19dbf/32.png) [@hailima](https://discuss.elastic.co/u/hailima)\ **Post date:** [January 20, 2017, 12:24am UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/10 "2017-01-20T00:24:04Z")
thanks and it's very helpful! Is there anyway to clean up task errors under the command .tasks/\_search? ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [January 20, 2017, 2:00am UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/11 "2017-01-20T02:00:28Z")
You can and should delete them when you are done with them. You can use delete-by-query or delete each one when you know you are done with it. ---
**Author:** ![hailima](https://avatars.discourse-cdn.com/v4/letter/h/f19dbf/32.png) [@hailima](https://discuss.elastic.co/u/hailima)\ **Post date:** [January 23, 2017, 10:46pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/12 "2017-01-23T22:46:48Z")
ok and thanks, If i don't delete them, are they expired automatically? if yes, when are they expired? ---
**Author:** ![hailima](https://avatars.discourse-cdn.com/v4/letter/h/f19dbf/32.png) [@hailima](https://discuss.elastic.co/u/hailima)\ **Post date:** [January 24, 2017, 5:00pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/13 "2017-01-24T17:00:28Z")
Hi Nik, The task cancel API is not working for ES 5.1.1 Here are ones I tried based reindex doc (POST) /\_tasks/1241809:1/\_cancel (POST) /\_tasks/1241809/\_cancel any idea ? thanks ---
**Author:** ![flow\_state](https://avatars.discourse-cdn.com/v4/letter/f/2bfe46/32.png) [@flow\_state](https://discuss.elastic.co/u/flow_state)\ **Post date:** [January 24, 2017, 7:36pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/14 "2017-01-24T19:36:33Z")
We are currently over a month behind on a migration because we have to babysit each and every reindex because of silent failures. We've resorted to setting the logger._root_ to "Debug". Good luck! ---
**Author:** ![hailima](https://avatars.discourse-cdn.com/v4/letter/h/f19dbf/32.png) [@hailima](https://discuss.elastic.co/u/hailima)\ **Post date:** [January 24, 2017, 11:21pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/15 "2017-01-24T23:21:05Z")
> [@nik9000](#): > > wrap your code I wrapped the codes inside `xxxxx` . It's not working as expected ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 21, 2017, 11:21pm UTC](https://discuss.elastic.co/t/elasticsearch-5-1-1-remote-reindex-process-aborts-without-any-error/72222/16 "2017-02-21T23:21:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.