# Ingestion have been stoped at 70%, would my ingested data will get lost when I reingest?

**URL:** <https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099>\
**Category:** Elasticsearch\
**Created:** [April 14, 2021, 9:58am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099 "2021-04-14T09:58:46Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![sanjaysh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sanjaysh/32/79183_2.png) [@sanjaysh](https://discuss.elastic.co/u/sanjaysh)\
**Post date:** [April 14, 2021, 9:58am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/1 "2021-04-14T09:58:46Z")

</div>

I am ingesting data on EC and I have almost ingested my 70% of data but the ingestion process have suddenly got stopped, and now I have to ingest again with the remaining data so would the remaining data will lost or would it be appended with the ingested data.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 14, 2021, 10:05am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/2 "2021-04-14T10:05:34Z")

</div>

Welcome to our community! 😃

Unless the ingestion process tracks the data - which is what Filebeat or Logstash should do - then you may need to reprocess the lot.

---

<div class="post-metadata">

**Author:** ![sanjaysh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sanjaysh/32/79183_2.png) [@sanjaysh](https://discuss.elastic.co/u/sanjaysh)\
**Post date:** [April 14, 2021, 10:30am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/3 "2021-04-14T10:30:51Z")

</div>

I did not understand  
I mean the same index with remaining data 3GB out of 20GB data if I ingest to the same index so will I lost 9GB ingested data to Elastic Cloud

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 14, 2021, 10:37am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/4 "2021-04-14T10:37:16Z")

</div>

It should still be there.

---

<div class="post-metadata">

**Author:** ![sanjaysh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sanjaysh/32/79183_2.png) [@sanjaysh](https://discuss.elastic.co/u/sanjaysh)\
**Post date:** [April 14, 2021, 10:47am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/5 "2021-04-14T10:47:11Z")

</div>

i am concerning because this happend in production so if i get surety so i will work on like that

---

<div class="post-metadata">

**Author:** ![sanjaysh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sanjaysh/32/79183_2.png) [@sanjaysh](https://discuss.elastic.co/u/sanjaysh)\
**Post date:** [April 14, 2021, 10:48am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/6 "2021-04-14T10:48:15Z")

</div>

so you mean my old ingested 9GB data i will not lose if i ingest with remaining data to the same index?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 14, 2021, 10:48am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/7 "2021-04-14T10:48:46Z")

</div>

That really depends on the details of how you are ingesting the data.

---

<div class="post-metadata">

**Author:** ![sanjaysh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sanjaysh/32/79183_2.png) [@sanjaysh](https://discuss.elastic.co/u/sanjaysh)\
**Post date:** [April 14, 2021, 10:50am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/8 "2021-04-14T10:50:21Z")

</div>

like what detail?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 14, 2021, 10:51am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/9 "2021-04-14T10:51:21Z")

</div>

You haven't really provided much, so I guess anything at this stage would be helpful 😉

---

<div class="post-metadata">

**Author:** ![sanjaysh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sanjaysh/32/79183_2.png) [@sanjaysh](https://discuss.elastic.co/u/sanjaysh)\
**Post date:** [April 14, 2021, 10:51am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/10 "2021-04-14T10:51:34Z")

</div>

no changes on code only data are being changed

---

<div class="post-metadata">

**Author:** ![sanjaysh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sanjaysh/32/79183_2.png) [@sanjaysh](https://discuss.elastic.co/u/sanjaysh)\
**Post date:** [April 14, 2021, 11:05am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/11 "2021-04-14T11:05:05Z")

</div>

yeah but i am finding surety  
for this question  
mr./mrs. : ""warkolm""

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 14, 2021, 11:35am UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/12 "2021-04-14T11:35:02Z")

</div>

How are you ingesting data? Filebeat? Logstash?

---

<div class="post-metadata">

**Author:** ![sanjaysh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sanjaysh/32/79183_2.png) [@sanjaysh](https://discuss.elastic.co/u/sanjaysh)\
**Post date:** [April 14, 2021, 12:27pm UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/13 "2021-04-14T12:27:27Z")

</div>

using CSV file

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 14, 2021, 12:38pm UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/14 "2021-04-14T12:38:55Z")

</div>

How are you reading the file? Custom script?

The reasoning am asking is that both Filebeat and Logstash can keep track of what has been ingested and resume when restarted or when the cluster is back.

---

<div class="post-metadata">

**Author:** ![sanjaysh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sanjaysh/32/79183_2.png) [@sanjaysh](https://discuss.elastic.co/u/sanjaysh)\
**Post date:** [April 14, 2021, 12:49pm UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/15 "2021-04-14T12:49:21Z")

</div>

by using script I am ingesting and reading as well can you help me to know If i ingest same kind of data to the same index so data will append or not and will i lost old data on the same index

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 12, 2021, 12:49pm UTC](https://discuss.elastic.co/t/ingestion-have-been-stoped-at-70-would-my-ingested-data-will-get-lost-when-i-reingest/270099/16 "2021-05-12T12:49:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
