# Indexed documents not showing up in search results **URL:** \ **Category:** Elasticsearch\ **Created:** [October 20, 2014, 1:52pm UTC](https://discuss.elastic.co/t/indexed-documents-not-showing-up-in-search-results/20341 "2014-10-20T13:52:46Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Pieter\_Agenbag](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pieter_agenbag/32/4562_2.png) [@Pieter\_Agenbag](https://discuss.elastic.co/u/Pieter_Agenbag)\ **Post date:** [October 20, 2014, 1:52pm UTC](https://discuss.elastic.co/t/indexed-documents-not-showing-up-in-search-results/20341/1 "2014-10-20T13:52:46Z")
Good afternoon. The subject might be a little misleading as to the true nature of my problem - which I'll try to explain here in as much detail as possible. First of all , I am rather new to Elasticsearch. Secondly , this problem has happened more than once (after dumping all indexes and starting over). Ok ,here goes: I have a single elasticsearch node running on a dedicated Dell PowerEdge R720 with dual 6 core cpus and 96GB ram. (of which 32GB is assigned to the HEAP) The machine is connected back to back with a 10GB fiber to another Dell R620. My processing happens on the R620 which then uses the bulk api to index (currently testing at 8000 documents per second) into the ES on the R720. The documents are all of the same type and indexed into daily "partitions" .. events-2014-10-18 , events-2014-10-19 , events-2014-10-20 etc. I then have a Kibana dashboard on top of that. All of this works perfectly for several days and then seemingly stops. I noticed that my Kibana dashboard (defaulting to the last hours data) stopped plotting. Further investigation showed that my application is still processing and indexing documents, but a search on todays index , with descending timestamp ordering shows the last document as 3:59 this morning. POST /events-2014-10-20/event/\_search { "size":20 ,"from": 0 ,"query": { "filtered": { "query": { "match\_all": {} } } } ,"sort": [ { "timestamp": { "order": "desc" } } ] } .... result.... { "took": 22836, "timed\_out": false, "\_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 385153635, "max\_score": null, "hits": [ { "\_index": "events-2014-10-20", "\_type": "event", "\_id": "HYOy53Q3TM-TwBz6SLRB5w", "\_score": null, "\_source": { "timestamp": "2014-10-20 03:59:14", .. etc Now, when I do a bulk index , the API tells me that the documents were created and specifies the autogenerated ids. So , taking one of those IDs and querying it directly , I DO get the document back. In the correct index , with the correct timestamp at 10:12 (from earlier when I was investigating) GET /events-2014-10-20/event/kY6MaTgCThizVGgb3iSGrw { "\_index": "events-2014-10-20", "\_type": "event", "\_id": "kY6MaTgCThizVGgb3iSGrw", "\_version": 1, "found": true, "\_source": { "timestamp": "2014-10-20 10:12:35", * * * ## Some system stats : The machine's cpu spikes every now and then (as I issue bulk indexes) , but drops down to idle again afterwards. There's plenty left on the heap and the disks is only 33% used. curl -XGET 'localhost:9200/\_cat/health?v' epoch timestamp cluster status node.total node.data shards pri relo init unassign 1413812685 15:44:45 elasticsearch yellow 1 1 85 85 0 0 15 curl -XGET 'localhost:9200/\_cat/count?v' epoch timestamp count 1413812740 15:45:40 4171812520 curl -XGET '10.15.3.19:9200/\_cat/indices/events\*?v' health index pri rep docs.count docs.deleted store.size pri.store.size green events-2014-10-19 5 0 702422696 0 230.5gb 230.5gb green events-2014-10-17 5 0 702341395 0 230.4gb 230.4gb green events-2014-10-18 5 0 702352975 0 230.3gb 230.3gb green events-2014-10-15 5 0 703229636 0 230.8gb 230.8gb green events-2014-10-16 5 0 701547992 0 230.2gb 230.2gb green events-2014-10-14 5 0 252394923 0 83.1gb 83.1gb yellow events-2014-10-20 5 1 407331075 0 127.9gb 127.9gb curl -XGET 'localhost:9200/\_cat/pending\_tasks?v' insertOrder timeInQueue priority source curl -XGET '10.15.3.19:9200/\_cat/shards/events-2014-10-20?v' index shard prirep state docs store ip node events-2014-10-20 4 p STARTED 81862132 27.8gb 10.15.3.19 X-Ray events-2014-10-20 4 r UNASSIGNED events-2014-10-20 0 p STARTED 81869999 25.4gb 10.15.3.19 X-Ray events-2014-10-20 0 r UNASSIGNED events-2014-10-20 3 p STARTED 81885822 25.5gb 10.15.3.19 X-Ray events-2014-10-20 3 r UNASSIGNED events-2014-10-20 1 p STARTED 81868103 25.5gb 10.15.3.19 X-Ray events-2014-10-20 1 r UNASSIGNED events-2014-10-20 2 p STARTED 81871297 26.5gb 10.15.3.19 X-Ray events-2014-10-20 2 r UNASSIGNED I can obviously not put this system into production and my deadline is fast approaching - so , any help will be greatly appreciated. Regards. Pieter -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/a0895574-c383-4a31-b4cb-620d0210e459%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a0895574-c383-4a31-b4cb-620d0210e459%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pieter\_Agenbag](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pieter_agenbag/32/4562_2.png) [@Pieter\_Agenbag](https://discuss.elastic.co/u/Pieter_Agenbag)\ **Post date:** [October 20, 2014, 2:01pm UTC](https://discuss.elastic.co/t/indexed-documents-not-showing-up-in-search-results/20341/2 "2014-10-20T14:01:00Z")
The elasticsearch log file was not showing any activity during the time when this was happening , but when I restarted the elasticsearch service - I got some warnings (see attached log file) , but then seem to proceed normally and has not complained any more . Note: My process is still indexing new documents. On Monday, October 20, 2014 3:52:47 PM UTC+2, Pieter Agenbag wrote: > Good afternoon. > > The subject might be a little misleading as to the true nature of my > problem - which I'll try to explain here in as much detail as possible. > > First of all , I am rather new to Elasticsearch. > > Secondly , this problem has happened more than once (after dumping all > indexes and starting over). > > Ok ,here goes: > > I have a single elasticsearch node running on a dedicated Dell PowerEdge > R720 with dual 6 core cpus and 96GB ram. (of which 32GB is assigned to the > HEAP) > The machine is connected back to back with a 10GB fiber to another Dell > R620. > > My processing happens on the R620 which then uses the bulk api to index > (currently testing at 8000 documents per second) into the ES on the R720. > The documents are all of the same type and indexed into daily "partitions" > .. events-2014-10-18 , events-2014-10-19 , events-2014-10-20 etc. > I then have a Kibana dashboard on top of that. > > All of this works perfectly for several days and then seemingly stops. I > noticed that my Kibana dashboard (defaulting to the last hours data) > stopped plotting. Further investigation showed that my application is still > processing and indexing documents, > but a search on todays index , with descending timestamp ordering shows > the last document as 3:59 this morning. > > POST /events-2014-10-20/event/\_search > { > "size":20 > ,"from": 0 > ,"query": > { > "filtered": > { > "query": > { > "match\_all": {} > } > } > } > ,"sort": > [ > { > "timestamp": > { > "order": "desc" > } > } > ] > } > > .... result.... > { > "took": 22836, > "timed\_out": false, > "\_shards": { > "total": 5, > "successful": 5, > "failed": 0 > }, > "hits": { > "total": 385153635, > "max\_score": null, > "hits": [ > { > "\_index": "events-2014-10-20", > "\_type": "event", > "\_id": "HYOy53Q3TM-TwBz6SLRB5w", > "\_score": null, > "\_source": { > "timestamp": "2014-10-20 03:59:14", > .. etc > > Now, when I do a bulk index , the API tells me that the documents were > created and specifies the autogenerated ids. > So , taking one of those IDs and querying it directly , I DO get the > document back. > In the correct index , with the correct timestamp at 10:12 (from earlier > when I was investigating) > > GET /events-2014-10-20/event/kY6MaTgCThizVGgb3iSGrw > > { > "\_index": "events-2014-10-20", > "\_type": "event", > "\_id": "kY6MaTgCThizVGgb3iSGrw", > "\_version": 1, > "found": true, > "\_source": { > "timestamp": "2014-10-20 10:12:35", > > * * * > > ## Some system stats : > > The machine's cpu spikes every now and then (as I issue bulk indexes) , > but drops down to idle again afterwards. > There's plenty left on the heap and the disks is only 33% used. > > curl -XGET 'localhost:9200/\_cat/health?v' > epoch timestamp cluster status node.total node.data shards pri > relo init unassign > 1413812685 15:44:45 elasticsearch yellow 1 1 85 85 > 0 0 15 > > curl -XGET 'localhost:9200/\_cat/count?v' > epoch timestamp count > 1413812740 15:45:40 4171812520 > > curl -XGET '10.15.3.19:9200/\_cat/indices/events\*?v' > health index pri rep docs.count docs.deleted store.size > pri.store.size > green events-2014-10-19 5 0 702422696 0 230.5gb > 230.5gb > green events-2014-10-17 5 0 702341395 0 230.4gb > 230.4gb > green events-2014-10-18 5 0 702352975 0 230.3gb > 230.3gb > green events-2014-10-15 5 0 703229636 0 230.8gb > 230.8gb > green events-2014-10-16 5 0 701547992 0 230.2gb > 230.2gb > green events-2014-10-14 5 0 252394923 0 83.1gb > 83.1gb > yellow events-2014-10-20 5 1 407331075 0 127.9gb > 127.9gb > > curl -XGET 'localhost:9200/\_cat/pending\_tasks?v' > insertOrder timeInQueue priority source > > curl -XGET '10.15.3.19:9200/\_cat/shards/events-2014-10-20?v' > index shard prirep state docs store ip node > events-2014-10-20 4 p STARTED 81862132 27.8gb 10.15.3.19 X-Ray > events-2014-10-20 4 r UNASSIGNED > events-2014-10-20 0 p STARTED 81869999 25.4gb 10.15.3.19 X-Ray > events-2014-10-20 0 r UNASSIGNED > events-2014-10-20 3 p STARTED 81885822 25.5gb 10.15.3.19 X-Ray > events-2014-10-20 3 r UNASSIGNED > events-2014-10-20 1 p STARTED 81868103 25.5gb 10.15.3.19 X-Ray > events-2014-10-20 1 r UNASSIGNED > events-2014-10-20 2 p STARTED 81871297 26.5gb 10.15.3.19 X-Ray > events-2014-10-20 2 r UNASSIGNED > > I can obviously not put this system into production and my deadline is > fast approaching - so , any help will be greatly appreciated. > > Regards. > Pieter -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e5ce48f5-8b1d-4a64-a0b7-ef7d6f45dc2f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e5ce48f5-8b1d-4a64-a0b7-ef7d6f45dc2f%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:55am UTC](https://discuss.elastic.co/t/indexed-documents-not-showing-up-in-search-results/20341/3 "2017-07-06T00:55:02Z")