# Elasticsearch not indexing all files **URL:** \ **Category:** Elasticsearch\ **Created:** [September 11, 2013, 9:46am UTC](https://discuss.elastic.co/t/elasticsearch-not-indexing-all-files/13551 "2013-09-11T09:46:47Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![KS1](https://avatars.discourse-cdn.com/v4/letter/k/77aa72/32.png) [@KS1](https://discuss.elastic.co/u/KS1)\ **Post date:** [September 11, 2013, 9:46am UTC](https://discuss.elastic.co/t/elasticsearch-not-indexing-all-files/13551/1 "2013-09-11T09:46:47Z")
Hi, I am trying to index all my files stored in MongoDB using Elasticsearch. But only 180842 files are indexed whereas I have 1637870 files in my DB. I tried to delete an create an index and everytime it stops after indexing 180842 files. Any idea why not all documents are indexed? I checked Elasticsearch log files and there is no error. But I found the below lines in my log file. (1) [2013-09-11 02:20:57,539][INFO][river.mongodb] [Arsenic] [mongodb][mongodb] Add attachment: 522bef23649dd3bb06a61fd8 (2) [2013-09-11 02:20:57,539][INFO][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] Add Attachment: 522bef0fe819cc4b70875a48 to index mongoindex / type files (3) [2013-09-11 02:20:57,539][INFO][river.mongodb] [Arsenic] [mongodb][mongodb] Caught file: 522bef230eb5b705cf8ccd91 - /data/Test.java Line (2) means that the file is added to my index. Am not sure what Line (1) and (3) means. Does that mean that those files are not added to index? NOTE: I used the below code to create index: curl -XPUT '[http://localhost:9200/\_river/mongodb/\_meta](http://localhost:9200/_river/mongodb/_meta)' -d '{ "type": "mongodb", "mongodb": { "db": "submission\_data", "collection": "fs", "gridfs": true }, "index": { "name": "mongoindex", "type": "files" } }' Thanks, Kiruthika -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![KS1](https://avatars.discourse-cdn.com/v4/letter/k/77aa72/32.png) [@KS1](https://discuss.elastic.co/u/KS1)\ **Post date:** [September 12, 2013, 2:37am UTC](https://discuss.elastic.co/t/elasticsearch-not-indexing-all-files/13551/2 "2013-09-12T02:37:05Z")
Could anyone please help me in this issue??? I am not able to index all the files using Elasticsearch.... Thanks, Kiruthika On Wednesday, September 11, 2013 2:46:47 AM UTC-7, KS wrote: > Hi, > > I am trying to index all my files stored in MongoDB using Elasticsearch. > But only 180842 files are indexed whereas I have 1637870 files in my DB. I > tried to delete an create an index and everytime it stops after indexing > 180842 files. Any idea why not all documents are indexed? > > I checked Elasticsearch log files and there is no error. But I found the > below lines in my log file. > > (1) [2013-09-11 02:20:57,539][INFO][river.mongodb] [Arsenic] [mongodb][mongodb] Add attachment: 522bef23649dd3bb06a61fd8 > (2) [2013-09-11 02:20:57,539][INFO][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] Add Attachment: 522bef0fe819cc4b70875a48 to index mongoindex / type files > (3) [2013-09-11 02:20:57,539][INFO][river.mongodb] [Arsenic] [mongodb][mongodb] Caught file: 522bef230eb5b705cf8ccd91 - /data/Test.java > > Line (2) means that the file is added to my index. Am not sure what Line > (1) and (3) means. Does that mean that those files are not added to index? > > NOTE: I used the below code to create index: > > curl -XPUT '[http://localhost:9200/\_river/mongodb/\_meta](http://localhost:9200/_river/mongodb/_meta)' -d '{ "type": "mongodb", > "mongodb": { > "db": "submission\_data", > "collection": "fs", > "gridfs": true > }, > "index": { > "name": "mongoindex", > "type": "files" > } > }' > > Thanks, > > Kiruthika -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![sina\_tamanna](https://avatars.discourse-cdn.com/v4/letter/s/a6a055/32.png) [@sina\_tamanna](https://discuss.elastic.co/u/sina_tamanna)\ **Post date:** [September 12, 2013, 7:03am UTC](https://discuss.elastic.co/t/elasticsearch-not-indexing-all-files/13551/3 "2013-09-12T07:03:23Z")
It could be useful to start your elasticsearch node in debug mode and check the queries sent to the mongodb. It would give you hints on why there are less documents. To enable debug mode logging: open /config/logging.yml and change the first line from rootLogger: INFO to rootLogger: DEBUG -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![KS1](https://avatars.discourse-cdn.com/v4/letter/k/77aa72/32.png) [@KS1](https://discuss.elastic.co/u/KS1)\ **Post date:** [September 12, 2013, 1:55pm UTC](https://discuss.elastic.co/t/elasticsearch-not-indexing-all-files/13551/4 "2013-09-12T13:55:20Z")
Hi, I tried it with DEBUG option too, I am not getting any error in the log file but still I am not able to index all the files. Thanks, Kiruthika On Thursday, September 12, 2013 12:03:23 AM UTC-7, [sina.t...@gmail.com](mailto:sina.t...@gmail.com) wrote: > It could be useful to start your elasticsearch node in debug mode and > check the queries sent to the mongodb. It would give you hints on why there > are less documents. > > To enable debug mode logging: open /config/logging.yml and change the > first line from rootLogger: INFO to rootLogger: DEBUG -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![sina\_tamanna](https://avatars.discourse-cdn.com/v4/letter/s/a6a055/32.png) [@sina\_tamanna](https://discuss.elastic.co/u/sina_tamanna)\ **Post date:** [September 12, 2013, 2:31pm UTC](https://discuss.elastic.co/t/elasticsearch-not-indexing-all-files/13551/5 "2013-09-12T14:31:58Z")
If you are running elasticsearch in debug mode you should be able to see the queries issued against mongodb. By looking at those queries you could verify if you are getting all the documents from mongodb to elasticsearch. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:16am UTC](https://discuss.elastic.co/t/elasticsearch-not-indexing-all-files/13551/6 "2017-07-06T02:16:54Z")