# Indexed documents disappearing randomly within a short time after indexing **URL:** \ **Category:** Elasticsearch\ **Created:** [December 19, 2022, 1:05pm UTC](https://discuss.elastic.co/t/indexed-documents-disappearing-randomly-within-a-short-time-after-indexing/321560 "2022-12-19T13:05:06Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![bob22](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob22/32/114924_2.png) [@bob22](https://discuss.elastic.co/u/bob22)\ **Post date:** [December 19, 2022, 1:05pm UTC](https://discuss.elastic.co/t/indexed-documents-disappearing-randomly-within-a-short-time-after-indexing/321560/1 "2022-12-19T13:05:06Z")
Hello, I'm having a weird issue with elasticsearch when indexed documents would disappear from the index within a short time after indexing. My application is indexing a couple of thousands documents (not a huge batch) and I can see in kibana the amount of newly indexed documents, but after a short time, this number drops and as if those documents were never indexed. There is enough disk space, there is no ID collision, other documents are being indexed to the same index and do not disappear... I've never seen this strange behaviour. To briefly explain what I do: I have an index of 'articles', then an app is translating those articles. When translated, the original article will receive a new id, a new property (Language = "de", or "es", etc..), some additional 'tags' and be stored in the same index where the original article came from. If the app has translated 5K articles, then I expect to see them in the index... I see them briefly, then they start disappearing in batches, until the number eventually stops at 160 or 230, etc.. not the 5K, which was already ingested. What is going on? I'll be grateful for any hints Thanks in advance! ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 19, 2022, 1:21pm UTC](https://discuss.elastic.co/t/indexed-documents-disappearing-randomly-within-a-short-time-after-indexing/321560/2 "2022-12-19T13:21:47Z")
> [@bob22](#): > > there is no ID collision How are your IDs constructed? If you keep track of the ID of the documents you have inserted that disappear and try to GET them by ID, what do you get? ---
**Author:** ![bob22](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob22/32/114924_2.png) [@bob22](https://discuss.elastic.co/u/bob22)\ **Post date:** [December 19, 2022, 1:25pm UTC](https://discuss.elastic.co/t/indexed-documents-disappearing-randomly-within-a-short-time-after-indexing/321560/3 "2022-12-19T13:25:06Z")
The original ID looks like this "C5EA78A22B04F30CD0E14F54D85EC0AF", The new ID is an MD5 hash of the original ID + "\_{Language}", - becomes completely different, yet I can check if it's been processed or not.. When I try to get the translated article by ID (the new ID), I can get it, only until it disappears, then it's as if it was never indexed... doesn't exist.. strange ---
**Author:** ![bob22](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob22/32/114924_2.png) [@bob22](https://discuss.elastic.co/u/bob22)\ **Post date:** [December 19, 2022, 1:33pm UTC](https://discuss.elastic.co/t/indexed-documents-disappearing-randomly-within-a-short-time-after-indexing/321560/4 "2022-12-19T13:33:30Z")
I am running a test now, where the translated articles will not be stored in the same index the original articles came from, but in a new index - "translated-articles" (for example), and looks like the articles don't disappear from the new index... I would love to understand what's going on.. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 19, 2022, 1:44pm UTC](https://discuss.elastic.co/t/indexed-documents-disappearing-randomly-within-a-short-time-after-indexing/321560/5 "2022-12-19T13:44:39Z")
Elasticsearch does not delete documents automatically and there is no TTL, so I suspect you have some process either deleting or overwriting the documents in question. ---
**Author:** ![bob22](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob22/32/114924_2.png) [@bob22](https://discuss.elastic.co/u/bob22)\ **Post date:** [December 19, 2022, 1:51pm UTC](https://discuss.elastic.co/t/indexed-documents-disappearing-randomly-within-a-short-time-after-indexing/321560/6 "2022-12-19T13:51:48Z")
aha.. ok I will try to confirm and find out which other process could possibly interact with it.. I do have multiple 'micro apps' each doing it's part (pre-processing/topical scoring, that translating app, etc..) It could be the reason indeed ---
**Author:** ![bob22](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob22/32/114924_2.png) [@bob22](https://discuss.elastic.co/u/bob22)\ **Post date:** [December 20, 2022, 9:29am UTC](https://discuss.elastic.co/t/indexed-documents-disappearing-randomly-within-a-short-time-after-indexing/321560/7 "2022-12-20T09:29:51Z")
Thank you very much for your help! Your reply has helped me switch the focus and identify the problem. It was indeed a process that deleted the newly translated articles. A de-duplication system was going after one property that was being cloned in the translated articles 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 17, 2023, 9:30am UTC](https://discuss.elastic.co/t/indexed-documents-disappearing-randomly-within-a-short-time-after-indexing/321560/8 "2023-01-17T09:30:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.