# Reindexing with logstash

**URL:** <https://discuss.elastic.co/t/reindexing-with-logstash/66464>\
**Category:** Logstash\
**Created:** [November 17, 2016, 10:15pm UTC](https://discuss.elastic.co/t/reindexing-with-logstash/66464 "2016-11-17T22:15:18Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![nin](https://avatars.discourse-cdn.com/v4/letter/n/77aa72/32.png) [@nin](https://discuss.elastic.co/u/nin)\
**Post date:** [November 17, 2016, 10:15pm UTC](https://discuss.elastic.co/t/reindexing-with-logstash/66464/1 "2016-11-17T22:15:18Z")

</div>

I have a an old elastic cluster running 1.7.5 that is underperforming.  
I've setup a new cluster on 2.4.6 and want to use logstash to reindex data from old to new.  
In the process I am consolidating daily indexes to monthlies.

What is the behavior around the logstash elastic input? If i give it an index to feed data to the new elasticsearch does it keep looping over the index and keep feeding duplicate docs over and over again? Does it know once the index specified in the logstash input has completed?  
If not what is the best way to track this?  
Or does adding document\_id =\> "%{[@metadata][\_id]}" prevent this if it's the case?

Thanks.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 22, 2016, 8:59pm UTC](https://discuss.elastic.co/t/reindexing-with-logstash/66464/2 "2016-11-22T20:59:44Z")

</div>

> Does it know once the index specified in the logstash input has completed?

No, it doesn't track this between Logstash restarts.

> Or does adding document\_id =\> "%{[@metadata][\_id]}" prevent this if it's the case?

I'm not sure it's sufficient, but it should definitely help. I don't recall if the elasticsearch output requires additional options to fully support upsert operations.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 20, 2016, 8:59pm UTC](https://discuss.elastic.co/t/reindexing-with-logstash/66464/3 "2016-12-20T20:59:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.