Ingestion have been stoped at 70%, would my ingested data will get lost when I reingest?

I am ingesting data on EC and I have almost ingested my 70% of data but the ingestion process have suddenly got stopped, and now I have to ingest again with the remaining data so would the remaining data will lost or would it be appended with the ingested data.

Welcome to our community! :smiley:

Unless the ingestion process tracks the data - which is what Filebeat or Logstash should do - then you may need to reprocess the lot.

I did not understand
I mean the same index with remaining data 3GB out of 20GB data if I ingest to the same index so will I lost 9GB ingested data to Elastic Cloud

It should still be there.

i am concerning because this happend in production so if i get surety so i will work on like that

so you mean my old ingested 9GB data i will not lose if i ingest with remaining data to the same index?

That really depends on the details of how you are ingesting the data.

like what detail?

You haven't really provided much, so I guess anything at this stage would be helpful :wink:

no changes on code only data are being changed

yeah but i am finding surety
for this question
mr./mrs. : ""warkolm""

How are you ingesting data? Filebeat? Logstash?

using CSV file

How are you reading the file? Custom script?

The reasoning am asking is that both Filebeat and Logstash can keep track of what has been ingested and resume when restarted or when the cluster is back.

by using script I am ingesting and reading as well can you help me to know If i ingest same kind of data to the same index so data will append or not and will i lost old data on the same index

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.