# Missing indexed documents

**URL:** <https://discuss.elastic.co/t/missing-indexed-documents/10575>\
**Category:** Elasticsearch\
**Created:** [January 31, 2013, 6:52pm UTC](https://discuss.elastic.co/t/missing-indexed-documents/10575 "2013-01-31T18:52:33Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mkocic](https://avatars.discourse-cdn.com/v4/letter/m/898d66/32.png) [@mkocic](https://discuss.elastic.co/u/mkocic)\
**Post date:** [January 31, 2013, 6:52pm UTC](https://discuss.elastic.co/t/missing-indexed-documents/10575/1 "2013-01-31T18:52:33Z")

</div>

Hi,

I am using elasticsearch 0.20.2 on 8GB RAM system (EC2 with Arch Linux). I notice something weird happen. One day i used regular search to find some documents and i got them, but two days later those documents disappeared. I try everything to find them but they just missing. And my code (using pycassa) do not using deleting.

let me explain you my case:

1. i have three nodes (three EC2 instances) with 5 shareds and 1 replica.
2. i have index with \>9 000 000 indexed documents.
3. each document is lower than 20KB.
4. new documents are constantly added into index (with 60 sec pause on each 10240 inserted documents).

First i was thinking that id is not unique but this is not cause - checked.  
After that i was suspicious on limits for file descriptors and i set them to 65000 (soft and hard). elasticsearch is started with these limitations - checked. But when i do lsof -a -p \<es\_pid\> | wc -l i see only 2090 open files and also i see that only opened are those /0/indices/index/\* but why other numbers are not opened (e.g. /1/indices/index/\*).

Here is cluster/index health state:

{"cluster\_name":"index\_cluster","status":"yellow","timed\_out":false,"number\_of\_nodes":3,"number\_of\_data\_nodes":3,"active\_primary\_shards":5,"active\_shards":11,"relocating\_shards":0,"initializing\_shards":2,"unassigned\_shards":7}

This is very dangerous i really don't know how documents could disappeared.  
Can someone help be with this.

Thank you,  
Milan

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:53am UTC](https://discuss.elastic.co/t/missing-indexed-documents/10575/2 "2017-07-06T02:53:43Z")

</div>