# Migrating 500 millions of Document from one cluster to another cluster

**URL:** <https://discuss.elastic.co/t/migrating-500-millions-of-document-from-one-cluster-to-another-cluster/221006>\
**Category:** Elasticsearch\
**Created:** [February 26, 2020, 10:17am UTC](https://discuss.elastic.co/t/migrating-500-millions-of-document-from-one-cluster-to-another-cluster/221006 "2020-02-26T10:17:04Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Thilak](https://avatars.discourse-cdn.com/v4/letter/t/4af34b/32.png) [@Thilak](https://discuss.elastic.co/u/Thilak)\
**Post date:** [February 26, 2020, 10:17am UTC](https://discuss.elastic.co/t/migrating-500-millions-of-document-from-one-cluster-to-another-cluster/221006/1 "2020-02-26T10:17:04Z")

</div>

Hello  
Currently, we are running 3 Node Cluster having over 83 indices and 2893 shards with a total of over 500 millions of Documents. We need to migrate the entire data to a different cluster. During the migration, we need to do a clean up some of the property value and remove some of the fields based on some condition. We thought of migrating using Logstash. Need clarification from Elastic Search Team is Logstash is the best way to migrate 5 TB of data from one server to different server.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 26, 2020, 11:13am UTC](https://discuss.elastic.co/t/migrating-500-millions-of-document-from-one-cluster-to-another-cluster/221006/2 "2020-02-26T11:13:46Z")

</div>

It is. You can also think about reindex from remote feature which runs in elasticsearch only.  
If you can do your cleanup with an ingest pipeline, I'd probably use that.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 25, 2020, 11:13am UTC](https://discuss.elastic.co/t/migrating-500-millions-of-document-from-one-cluster-to-another-cluster/221006/3 "2020-03-25T11:13:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.