# Ultra-slow indexing **URL:** \ **Category:** Elasticsearch\ **Created:** [January 19, 2012, 1:13pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441 "2012-01-19T13:13:56Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Alberto\_Tostado](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@Alberto\_Tostado](https://discuss.elastic.co/u/Alberto_Tostado)\ **Post date:** [January 19, 2012, 1:13pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/1 "2012-01-19T13:13:56Z")
_Good morning._ I'm a ElasticSearch newbie user and we're trying to use it as a search engine for a CouchDB database in a large information system for clinical laboratories. We are blocked because we need index a large amount of data and the indexation is so slow (1 doc per minute... and increasing.....). See table: DocsIndexed docs por minute510...91,6666667833...64,61111...27,81222...22,2 1500...11,121572...4 I suppose we are making something wrong, but we don't have expertise to know what is happening. We'd like to receive some guidelines to diagnose and solve the problem. Here the details.... (thank you for read). Our system produces 1000-3000 new docs per day with 20/30 updates in the first days of the life of the doc. Later, the docs remain unmodified (archived). Before starting the system, we need index the historical documents (10 years).... 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed and searchable. I attach a couple of sample document JSON. They are complex JSONs, but are suitable for our needs. We configure ElasticSearch and the CouchDb river as out of the box. Only one instance, one computer running CouchDb and ElasticSearch side by side. RAM: 6 GB CPU: Xeon W3530 2.8 Ghz. SO: Windows 7 The river is started with this command: %CURL% -XPUT "[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" The indexation is made automagically by the CouchDb river, but we get the same timing indexing by hand with curl: %CURL% -XPOST "[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt Thank you. Alberto Tostado. Spain. ---
**Author:** ![Berkay\_Mollamustafao](https://avatars.discourse-cdn.com/v4/letter/b/22d042/32.png) [@Berkay\_Mollamustafao](https://discuss.elastic.co/u/Berkay_Mollamustafao)\ **Post date:** [January 19, 2012, 1:24pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/2 "2012-01-19T13:24:09Z")
How much memory is assigned to Elasticsearch JVM ? Regards, Berkay Mollamustafaoglu mberkay on yahoo, google and skype On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado [atostado@gmail.com](mailto:atostado@gmail.com) wrote: > _Good morning._ > > I'm a Elasticsearch newbie user and we're trying to use it as a search > engine for a CouchDB database in a large information system for clinical > laboratories. > We are blocked because we need index a large amount of data and the > indexation is so slow (1 doc per minute... and increasing.....). See > table: > > Docs Indexed docs por minute 510... 91,6666667 833... 64,6 1111... 27,8 > 1222... 22,2 1500... 11,12 1572... 4 > > I suppose we are making something wrong, but we don't have expertise to > know what is happening. We'd like to receive some guidelines to diagnose > and solve the problem. > > Here the details.... (thank you for read). > > Our system produces 1000-3000 new docs per day with 20/30 updates in the > first days of the life of the doc. Later, the docs remain unmodified > (archived). > Before starting the system, we need index the historical documents (10 > years).... > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed and > searchable. > > I attach a couple of sample document JSON. They are complex JSONs, but are > suitable for our needs. > > We configure Elasticsearch and the CouchDb river as out of the box. Only > one instance, one computer running CouchDb and Elasticsearch side by side. > > RAM: 6 GB > CPU: Xeon W3530 2.8 Ghz. > SO: Windows 7 > > The river is started with this command: > > %CURL% -XPUT "[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d > "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > The indexation is made automagically by the CouchDb river, but we get the > same timing indexing by hand with curl: > > %CURL% -XPOST "[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > Thank you. > Alberto Tostado. > Spain. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 19, 2012, 1:46pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/3 "2012-01-19T13:46:41Z")
Do you see "update mapping" in logs ? If it's the case, create the right mapping before indexing as it seems that updating the mapping often has a real cost. HTH David 😉 @dadoonet Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu [mberkay@gmail.com](mailto:mberkay@gmail.com) a écrit : > How much memory is assigned to Elasticsearch JVM ? > > Regards, > Berkay Mollamustafaoglu > mberkay on yahoo, google and skype > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado [atostado@gmail.com](mailto:atostado@gmail.com) wrote: > Good morning. > > I'm a Elasticsearch newbie user and we're trying to use it as a search engine for a CouchDB database in a large information system for clinical laboratories. > We are blocked because we need index a large amount of data and the indexation is so slow (1 doc per minute... and increasing.....). See table: > > Docs Indexed docs por minute > 510... 91,6666667 > 833... 64,6 > 1111... 27,8 > 1222... 22,2 > 1500... 11,12 > 1572... 4 > > I suppose we are making something wrong, but we don't have expertise to know what is happening. We'd like to receive some guidelines to diagnose and solve the problem. > > Here the details.... (thank you for read). > > Our system produces 1000-3000 new docs per day with 20/30 updates in the first days of the life of the doc. Later, the docs remain unmodified (archived). > Before starting the system, we need index the historical documents (10 years).... > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed and searchable. > > I attach a couple of sample document JSON. They are complex JSONs, but are suitable for our needs. > > We configure Elasticsearch and the CouchDb river as out of the box. Only one instance, one computer running CouchDb and Elasticsearch side by side. > > RAM: 6 GB > CPU: Xeon W3530 2.8 Ghz. > SO: Windows 7 > > The river is started with this command: > > %CURL% -XPUT "[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > The indexation is made automagically by the CouchDb river, but we get the same timing indexing by hand with curl: > > %CURL% -XPOST "[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > Thank you. > Alberto Tostado. > Spain. ---
**Author:** ![Alberto\_Tostado](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@Alberto\_Tostado](https://discuss.elastic.co/u/Alberto_Tostado)\ **Post date:** [January 19, 2012, 3:59pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/4 "2012-01-19T15:59:10Z")
About "memory" and "update\_mapping"... The server was configured as default 256m\>\>1g, but I've changed it to 3g\>\>3g (following suggestions in documentation). With the change the timings have started better (100 docs/min), but again the rate decreases gradually (after 4000 docs, 35 docs/min). Better than before, but... Yes, I see a lot of "update\_mapping (dynamic)" in the console/log. 3 or 4 every minute. Sorry for the question, but... Is a problem the "update\_mapping"? I'm afraid I'm not doing well. I'm starting with Elasticsearch and I don't have clear concepts yet. Also, I see in console (1 per minute +/-) the following: Example: [2012-01-19 16:49:11,766][INFO][monitor.jvm] [Eddie Brock] [gc][ConcurrentMarkSweep][ 270] took [6.7s]/[4.3s], reclaimed [671.6mb], leaving [1.6gb] used, max [3.1gb] Thank you. Alberto Tostado Spain On Thu, Jan 19, 2012 at 2:46 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Do you see "update mapping" in logs ? > If it's the case, create the right mapping before indexing as it seems > that updating the mapping often has a real cost. > > HTH > David 😉 > @dadoonet > > Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu [mberkay@gmail.com](mailto:mberkay@gmail.com) a > écrit : > > How much memory is assigned to Elasticsearch JVM ? > > Regards, > Berkay Mollamustafaoglu > mberkay on yahoo, google and skype > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado \< [atostado@gmail.com](mailto:atostado@gmail.com) > [atostado@gmail.com](mailto:atostado@gmail.com)\> wrote: > > > _Good morning._ > > > > I'm a Elasticsearch newbie user and we're trying to use it as a search > > engine for a CouchDB database in a large information system for clinical > > laboratories. > > We are blocked because we need index a large amount of data and the > > indexation is so slow (1 doc per minute... and increasing.....). See > > table: > > > > Docs Indexed docs por minute 510... 91,6666667 833... 64,6 1111... 27,8 > > 1222... 22,2 1500... 11,12 1572... 4 > > > > I suppose we are making something wrong, but we don't have expertise to > > know what is happening. We'd like to receive some guidelines to diagnose > > and solve the problem. > > > > Here the details.... (thank you for read). > > > > Our system produces 1000-3000 new docs per day with 20/30 updates in the > > first days of the life of the doc. Later, the docs remain unmodified > > (archived). > > Before starting the system, we need index the historical documents (10 > > years).... > > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed > > and searchable. > > > > I attach a couple of sample document JSON. They are complex JSONs, but > > are suitable for our needs. > > > > We configure Elasticsearch and the CouchDb river as out of the box. Only > > one instance, one computer running CouchDb and Elasticsearch side by side. > > > > RAM: 6 GB > > CPU: Xeon W3530 2.8 Ghz. > > SO: Windows 7 > > > > The river is started with this command: > > > > %CURL% -XPUT " [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta) > > [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d > > "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > > > The indexation is made automagically by the CouchDb river, but we get the > > same timing indexing by hand with curl: > > > > %CURL% -XPOST " [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/) > > [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > > > Thank you. > > Alberto Tostado. > > Spain. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 19, 2012, 4:37pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/5 "2012-01-19T16:37:14Z")
That was the same for me when I started with ES. The problem is that you send 1000 docs with one field. Then, you send 1000 docs (same type) with a new field (so ES have to update the mapping as there is a new field to manage). Then, you send again 1000 docs with a new field. -\> New update... Imagine that when ES has to update the mapping, you continue sending documents. That's a huge work. Thats's why, I recommand to define the full mapping for your documents before you start to inject docs. How to do this ? This is the way I do it. I inject, let's say, 30000 docs in myindex/mytype and then I asked for the mapping curl -XGET [http://localhost:9200/myindex/mytype/\_mapping?pretty=true](http://localhost:9200/myindex/mytype/_mapping?pretty=true) Then, I drop mytype curl -XDELETE [http://localhost:9200/myindex/mytype](http://localhost:9200/myindex/mytype) I create mytype with the mapping I just downloaded : curl -XPUT [http://localhost:9200/myindex/mytype/\_mapping](http://localhost:9200/myindex/mytype/_mapping) -d @$ESJSONFILE And then, I reinject 50000 docs. It should run faster. Then I asked again for the mapping (if it was updated) and do everything again until I can not see anymore the update mapping message in log. HTH David. Le 19 janvier 2012 à 16:59, Alberto Tostado [atostado@gmail.com](mailto:atostado@gmail.com) a écrit : > About "memory" and "update\_mapping"... > > The server was configured as default 256m\>\>1g, but I've changed it > to 3g\>\>3g (following suggestions in documentation). > With the change the timings have started better (100 docs/min), > but again the rate decreases gradually (after 4000 docs, 35 docs/min). > Better than before, but... > > Yes, I see a lot of "update\_mapping (dynamic)" in the console/log. 3 or 4 > every minute. Sorry for the question, but... Is a problem the > "update\_mapping"? I'm afraid I'm not doing well. > I'm starting with Elasticsearch and I don't have clear concepts yet. > > Also, I see in console (1 per minute +/-) the following: > Example: > [2012-01-19 16:49:11,766][INFO][monitor.jvm] [Eddie Brock] > [gc][ConcurrentMarkSweep][ > 270] took [6.7s]/[4.3s], reclaimed [671.6mb], leaving [1.6gb] used, max > [3.1gb] > > Thank you. > Alberto Tostado > Spain > > On Thu, Jan 19, 2012 at 2:46 PM, David Pilato \< [david@pilato.fr](mailto:david@pilato.fr) > [mailto:david@pilato.fr](mailto:david@pilato.fr) \> wrote: > > > Do you see "update mapping" in logs ? > > If it's the case, create the right mapping before indexing as it seems that > > updating the mapping often has a real cost. > > > > HTH > > David 😉 > > @dadoonet > > > > Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu \< [mberkay@gmail.com](mailto:mberkay@gmail.com) > > [mailto:mberkay@gmail.com](mailto:mberkay@gmail.com) \> a écrit : > > > > > How much memory is assigned to Elasticsearch JVM ? > > > > > > Regards, > > > Berkay Mollamustafaoglu > > > mberkay on yahoo, google and skype > > > > > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado \< [atostado@gmail.com](mailto:atostado@gmail.com) > > > [mailto:atostado@gmail.com](mailto:atostado@gmail.com) \> wrote: > > > > > > > Good morning. > > > > > > > > I'm a Elasticsearch newbie user and we're trying to use it as a search > > > > engine for a CouchDB database in a large information system for clinical > > > > laboratories. > > > > We are blocked because we need index a large amount of data and the > > > > indexation is so slow (1 doc per minute... and increasing.....). See > > > > table: > > > > > > > > Docs Indexed docs por minute > > > > 510... 91,6666667 > > > > 833... 64,6 > > > > 1111... 27,8 > > > > 1222... 22,2 > > > > 1500... 11,12 > > > > 1572... 4 > > > > > > > > I suppose we are making something wrong, but we don't have expertise to > > > > know what is happening. We'd like to receive some guidelines to diagnose > > > > and solve the problem. > > > > > > > > Here the details.... (thank you for read). > > > > > > > > Our system produces 1000-3000 new docs per day with 20/30 updates in the > > > > first days of the life of the doc. Later, the docs remain unmodified > > > > (archived). > > > > Before starting the system, we need index the historical documents (10 > > > > years).... > > > > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed > > > > and > > > > searchable. > > > > > > > > I attach a couple of sample document JSON. They are complex JSONs, but > > > > are > > > > suitable for our needs. > > > > > > > > We configure Elasticsearch and the CouchDb river as out of the box. Only > > > > one instance, one computer running CouchDb and Elasticsearch side by > > > > side. > > > > > > > > RAM: 6 GB > > > > CPU: Xeon W3530 2.8 Ghz. > > > > SO: Windows 7 > > > > > > > > The river is started with this command: > > > > > > > > %CURL% -XPUT " [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta) > > > > [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta) " -d > > > > "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > > > > > > > The indexation is made automagically by the CouchDb river, but we get > > > > the > > > > same timing indexing by hand with curl: > > > > > > > > %CURL% -XPOST " [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/) > > > > [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/) " -d @file.txt > > > > > > > > Thank you. > > > > Alberto Tostado. > > > > Spain. -- David Pilato [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) Twitter : @dadoonet ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [January 19, 2012, 7:36pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/6 "2012-01-19T19:36:21Z")
update mapping happens when a document is indexed with new json fields, its not that heavy, at least not one that explains the probelm you have. Yes, increasing the memory will help. I wonder though. The logging you pointed out is enabled only on old versions (sadly, that important logging information no longer happens in newer versions because of a bug in the API that uses to get that data). Which version are you using? I suggest you use bigdesk plugin (check the plugins page on how to install it: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/plugins.html), and make sure to use the latest ES version (it has many improvements, including better memory control in the couchdb river). On Thu, Jan 19, 2012 at 5:59 PM, Alberto Tostado [atostado@gmail.com](mailto:atostado@gmail.com) wrote: > About "memory" and "update\_mapping"... > > The server was configured as default 256m\>\>1g, but I've changed it > to 3g\>\>3g (following suggestions in documentation). > With the change the timings have started better (100 docs/min), > but again the rate decreases gradually (after 4000 docs, 35 docs/min). > Better than before, but... > > Yes, I see a lot of "update\_mapping (dynamic)" in the console/log. 3 or 4 > every minute. Sorry for the question, but... Is a problem the > "update\_mapping"? I'm afraid I'm not doing well. > I'm starting with Elasticsearch and I don't have clear concepts yet. > > Also, I see in console (1 per minute +/-) the following: > Example: > [2012-01-19 16:49:11,766][INFO][monitor.jvm] [Eddie Brock] > [gc][ConcurrentMarkSweep][ > 270] took [6.7s]/[4.3s], reclaimed [671.6mb], leaving [1.6gb] used, max > [3.1gb] > > Thank you. > Alberto Tostado > Spain > > On Thu, Jan 19, 2012 at 2:46 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > Do you see "update mapping" in logs ? > > If it's the case, create the right mapping before indexing as it seems > > that updating the mapping often has a real cost. > > > > HTH > > David 😉 > > @dadoonet > > > > Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu [mberkay@gmail.com](mailto:mberkay@gmail.com) a > > écrit : > > > > How much memory is assigned to Elasticsearch JVM ? > > > > Regards, > > Berkay Mollamustafaoglu > > mberkay on yahoo, google and skype > > > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado \< [atostado@gmail.com](mailto:atostado@gmail.com) > > [atostado@gmail.com](mailto:atostado@gmail.com)\> wrote: > > > > > _Good morning._ > > > > > > I'm a Elasticsearch newbie user and we're trying to use it as a search > > > engine for a CouchDB database in a large information system for clinical > > > laboratories. > > > We are blocked because we need index a large amount of data and the > > > indexation is so slow (1 doc per minute... and increasing.....). See > > > table: > > > > > > Docs Indexed docs por minute 510... 91,6666667 833... 64,6 1111... 27,8 > > > 1222... 22,2 1500... 11,12 1572... 4 > > > > > > I suppose we are making something wrong, but we don't have expertise to > > > know what is happening. We'd like to receive some guidelines to diagnose > > > and solve the problem. > > > > > > Here the details.... (thank you for read). > > > > > > Our system produces 1000-3000 new docs per day with 20/30 updates in the > > > first days of the life of the doc. Later, the docs remain unmodified > > > (archived). > > > Before starting the system, we need index the historical documents (10 > > > years).... > > > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed > > > and searchable. > > > > > > I attach a couple of sample document JSON. They are complex JSONs, but > > > are suitable for our needs. > > > > > > We configure Elasticsearch and the CouchDb river as out of the box. Only > > > one instance, one computer running CouchDb and Elasticsearch side by side. > > > > > > RAM: 6 GB > > > CPU: Xeon W3530 2.8 Ghz. > > > SO: Windows 7 > > > > > > The river is started with this command: > > > > > > %CURL% -XPUT " [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta) > > > [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d > > > "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > > > > > The indexation is made automagically by the CouchDb river, but we get > > > the same timing indexing by hand with curl: > > > > > > %CURL% -XPOST " [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/) > > > [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > > > > > Thank you. > > > Alberto Tostado. > > > Spain. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 19, 2012, 7:45pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/7 "2012-01-19T19:45:32Z")
Hi Shay, Sorry to disagree but I got exactly the same _issue_ : At start indexing rate was 50 docs / sec. Then it starts to decrease down to 1 doc / sec. When I put a _good_ mapping before indexing, indexing has a constant rate. I did not change memory settings. BTW it was some months ago and I think I was using 0.16.x or 0.15.x with default settings (only one node with 5 shards) So I strongly think that updating the mapping very often while having a huge indexing load makes indexing less quick. David 😉 @dadoonet Le 19 janv. 2012 à 20:36, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a écrit : > update mapping happens when a document is indexed with new json fields, its not that heavy, at least not one that explains the probelm you have. > > Yes, increasing the memory will help. I wonder though. The logging you pointed out is enabled only on old versions (sadly, that important logging information no longer happens in newer versions because of a bug in the API that uses to get that data). Which version are you using? > > I suggest you use bigdesk plugin (check the plugins page on how to install it: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/plugins.html), and make sure to use the latest ES version (it has many improvements, including better memory control in the couchdb river). > > On Thu, Jan 19, 2012 at 5:59 PM, Alberto Tostado [atostado@gmail.com](mailto:atostado@gmail.com) wrote: > About "memory" and "update\_mapping"... > > The server was configured as default 256m\>\>1g, but I've changed it > to 3g\>\>3g (following suggestions in documentation). > With the change the timings have started better (100 docs/min), > but again the rate decreases gradually (after 4000 docs, 35 docs/min). > Better than before, but... > > Yes, I see a lot of "update\_mapping (dynamic)" in the console/log. 3 or 4 > every minute. Sorry for the question, but... Is a problem the > "update\_mapping"? I'm afraid I'm not doing well. > I'm starting with Elasticsearch and I don't have clear concepts yet. > > Also, I see in console (1 per minute +/-) the following: > Example: > [2012-01-19 16:49:11,766][INFO][monitor.jvm] [Eddie Brock] [gc][ConcurrentMarkSweep][ > 270] took [6.7s]/[4.3s], reclaimed [671.6mb], leaving [1.6gb] used, max [3.1gb] > > Thank you. > Alberto Tostado > Spain > > On Thu, Jan 19, 2012 at 2:46 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Do you see "update mapping" in logs ? > If it's the case, create the right mapping before indexing as it seems that updating the mapping often has a real cost. > > HTH > David 😉 > @dadoonet > > Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu [mberkay@gmail.com](mailto:mberkay@gmail.com) a écrit : > > > How much memory is assigned to Elasticsearch JVM ? > > > > Regards, > > Berkay Mollamustafaoglu > > mberkay on yahoo, google and skype > > > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado [atostado@gmail.com](mailto:atostado@gmail.com) wrote: > > Good morning. > > > > I'm a Elasticsearch newbie user and we're trying to use it as a search engine for a CouchDB database in a large information system for clinical laboratories. > > We are blocked because we need index a large amount of data and the indexation is so slow (1 doc per minute... and increasing.....). See table: > > > > Docs Indexed docs por minute > > 510... 91,6666667 > > 833... 64,6 > > 1111... 27,8 > > 1222... 22,2 > > 1500... 11,12 > > 1572... 4 > > > > I suppose we are making something wrong, but we don't have expertise to know what is happening. We'd like to receive some guidelines to diagnose and solve the problem. > > > > Here the details.... (thank you for read). > > > > Our system produces 1000-3000 new docs per day with 20/30 updates in the first days of the life of the doc. Later, the docs remain unmodified (archived). > > Before starting the system, we need index the historical documents (10 years).... > > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed and searchable. > > > > I attach a couple of sample document JSON. They are complex JSONs, but are suitable for our needs. > > > > We configure Elasticsearch and the CouchDb river as out of the box. Only one instance, one computer running CouchDb and Elasticsearch side by side. > > > > RAM: 6 GB > > CPU: Xeon W3530 2.8 Ghz. > > SO: Windows 7 > > > > The river is started with this command: > > > > %CURL% -XPUT "[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > > > The indexation is made automagically by the CouchDb river, but we get the same timing indexing by hand with curl: > > > > %CURL% -XPOST "[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > > > Thank you. > > Alberto Tostado. > > Spain. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [January 19, 2012, 7:53pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/8 "2012-01-19T19:53:50Z")
David, were you using the index API or bulk API? index API will, by default, wait for the mapping to be applied (might wait talking to the master of the cluster), but bulk will not (which is what the couchdriver will do). Also, all this process is much faster in newer versions, wondering if you would see the same behavior there. On Thu, Jan 19, 2012 at 9:45 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Hi Shay, > > Sorry to disagree but I got exactly the same _issue_ : > At start indexing rate was 50 docs / sec. > Then it starts to decrease down to 1 doc / sec. > > When I put a _good_ mapping before indexing, indexing has a constant rate. > > I did not change memory settings. > > BTW it was some months ago and I think I was using 0.16.x or 0.15.x with > default settings (only one node with 5 shards) > > So I strongly think that updating the mapping very often while having a > huge indexing load makes indexing less quick. > > David 😉 > @dadoonet > > Le 19 janv. 2012 à 20:36, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a écrit : > > update mapping happens when a document is indexed with new json fields, > its not that heavy, at least not one that explains the probelm you have. > > Yes, increasing the memory will help. I wonder though. The logging you > pointed out is enabled only on old versions (sadly, that important logging > information no longer happens in newer versions because of a bug in the API > that uses to get that data). Which version are you using? > > I suggest you use bigdesk plugin (check the plugins page on how to install > it: [http://www.elasticsearch.org/guide/reference/modules/plugins.html](http://www.elasticsearch.org/guide/reference/modules/plugins.html) > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/plugins.html), and > make sure to use the latest ES version (it has many improvements, including > better memory control in the couchdb river). > > On Thu, Jan 19, 2012 at 5:59 PM, Alberto Tostado \< [atostado@gmail.com](mailto:atostado@gmail.com) > [atostado@gmail.com](mailto:atostado@gmail.com)\> wrote: > > > About "memory" and "update\_mapping"... > > > > The server was configured as default 256m\>\>1g, but I've changed it > > to 3g\>\>3g (following suggestions in documentation). > > With the change the timings have started better (100 docs/min), > > but again the rate decreases gradually (after 4000 docs, 35 docs/min). > > Better than before, but... > > > > Yes, I see a lot of "update\_mapping (dynamic)" in the console/log. 3 or 4 > > every minute. Sorry for the question, but... Is a problem the > > "update\_mapping"? I'm afraid I'm not doing well. > > I'm starting with Elasticsearch and I don't have clear concepts yet. > > > > Also, I see in console (1 per minute +/-) the following: > > Example: > > [2012-01-19 16:49:11,766][INFO][monitor.jvm] [Eddie > > Brock] [gc][ConcurrentMarkSweep][ > > 270] took [6.7s]/[4.3s], reclaimed [671.6mb], leaving [1.6gb] used, max > > [3.1gb] > > > > Thank you. > > Alberto Tostado > > Spain > > > > On Thu, Jan 19, 2012 at 2:46 PM, David Pilato \< [david@pilato.fr](mailto:david@pilato.fr) > > [david@pilato.fr](mailto:david@pilato.fr)\> wrote: > > > > > Do you see "update mapping" in logs ? > > > If it's the case, create the right mapping before indexing as it seems > > > that updating the mapping often has a real cost. > > > > > > HTH > > > David 😉 > > > @dadoonet > > > > > > Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu \< [mberkay@gmail.com](mailto:mberkay@gmail.com) > > > [mberkay@gmail.com](mailto:mberkay@gmail.com)\> a écrit : > > > > > > How much memory is assigned to Elasticsearch JVM ? > > > > > > Regards, > > > Berkay Mollamustafaoglu > > > mberkay on yahoo, google and skype > > > > > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado \< [atostado@gmail.com](mailto:atostado@gmail.com)[atostado@gmail.com](mailto:atostado@gmail.com) > > > [atostado@gmail.com](mailto:atostado@gmail.com)\> wrote: > > > > > > > _Good morning._ > > > > > > > > I'm a Elasticsearch newbie user and we're trying to use it as a search > > > > engine for a CouchDB database in a large information system for clinical > > > > laboratories. > > > > We are blocked because we need index a large amount of data and the > > > > indexation is so slow (1 doc per minute... and increasing.....). See > > > > table: > > > > > > > > Docs Indexed docs por minute 510... 91,6666667 833... 64,6 1111... > > > > 27,8 1222... 22,2 1500... 11,12 1572... 4 > > > > > > > > I suppose we are making something wrong, but we don't have expertise to > > > > know what is happening. We'd like to receive some guidelines to diagnose > > > > and solve the problem. > > > > > > > > Here the details.... (thank you for read). > > > > > > > > Our system produces 1000-3000 new docs per day with 20/30 updates in > > > > the first days of the life of the doc. Later, the docs remain unmodified > > > > (archived). > > > > Before starting the system, we need index the historical documents (10 > > > > years).... > > > > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed > > > > and searchable. > > > > > > > > I attach a couple of sample document JSON. They are complex JSONs, but > > > > are suitable for our needs. > > > > > > > > We configure Elasticsearch and the CouchDb river as out of the box. > > > > Only one instance, one computer running CouchDb and Elasticsearch side by > > > > side. > > > > > > > > RAM: 6 GB > > > > CPU: Xeon W3530 2.8 Ghz. > > > > SO: Windows 7 > > > > > > > > The river is started with this command: > > > > > > > > %CURL% -XPUT " [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta) > > > > [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d > > > > "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > > > > > > > The indexation is made automagically by the CouchDb river, but we get > > > > the same timing indexing by hand with curl: > > > > > > > > %CURL% -XPOST " [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/) > > > > [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > > > > > > > Thank you. > > > > Alberto Tostado. > > > > Spain. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 19, 2012, 8:01pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/9 "2012-01-19T20:01:00Z")
You're right. I was using index API and not bulk API. David 😉 @dadoonet Le 19 janv. 2012 à 20:53, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a écrit : > David, were you using the index API or bulk API? index API will, by default, wait for the mapping to be applied (might wait talking to the master of the cluster), but bulk will not (which is what the couchdriver will do). Also, all this process is much faster in newer versions, wondering if you would see the same behavior there. > > On Thu, Jan 19, 2012 at 9:45 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Hi Shay, > > Sorry to disagree but I got exactly the same _issue_ : > At start indexing rate was 50 docs / sec. > Then it starts to decrease down to 1 doc / sec. > > When I put a _good_ mapping before indexing, indexing has a constant rate. > > I did not change memory settings. > > BTW it was some months ago and I think I was using 0.16.x or 0.15.x with default settings (only one node with 5 shards) > > So I strongly think that updating the mapping very often while having a huge indexing load makes indexing less quick. > > David 😉 > @dadoonet > > Le 19 janv. 2012 à 20:36, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a écrit : > > > update mapping happens when a document is indexed with new json fields, its not that heavy, at least not one that explains the probelm you have. > > > > Yes, increasing the memory will help. I wonder though. The logging you pointed out is enabled only on old versions (sadly, that important logging information no longer happens in newer versions because of a bug in the API that uses to get that data). Which version are you using? > > > > I suggest you use bigdesk plugin (check the plugins page on how to install it: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/plugins.html), and make sure to use the latest ES version (it has many improvements, including better memory control in the couchdb river). > > > > On Thu, Jan 19, 2012 at 5:59 PM, Alberto Tostado [atostado@gmail.com](mailto:atostado@gmail.com) wrote: > > About "memory" and "update\_mapping"... > > > > The server was configured as default 256m\>\>1g, but I've changed it > > to 3g\>\>3g (following suggestions in documentation). > > With the change the timings have started better (100 docs/min), > > but again the rate decreases gradually (after 4000 docs, 35 docs/min). > > Better than before, but... > > > > Yes, I see a lot of "update\_mapping (dynamic)" in the console/log. 3 or 4 > > every minute. Sorry for the question, but... Is a problem the > > "update\_mapping"? I'm afraid I'm not doing well. > > I'm starting with Elasticsearch and I don't have clear concepts yet. > > > > Also, I see in console (1 per minute +/-) the following: > > Example: > > [2012-01-19 16:49:11,766][INFO][monitor.jvm] [Eddie Brock] [gc][ConcurrentMarkSweep][ > > 270] took [6.7s]/[4.3s], reclaimed [671.6mb], leaving [1.6gb] used, max [3.1gb] > > > > Thank you. > > Alberto Tostado > > Spain > > > > On Thu, Jan 19, 2012 at 2:46 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > Do you see "update mapping" in logs ? > > If it's the case, create the right mapping before indexing as it seems that updating the mapping often has a real cost. > > > > HTH > > David 😉 > > @dadoonet > > > > Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu [mberkay@gmail.com](mailto:mberkay@gmail.com) a écrit : > > > > > How much memory is assigned to Elasticsearch JVM ? > > > > > > Regards, > > > Berkay Mollamustafaoglu > > > mberkay on yahoo, google and skype > > > > > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado [atostado@gmail.com](mailto:atostado@gmail.com) wrote: > > > Good morning. > > > > > > I'm a Elasticsearch newbie user and we're trying to use it as a search engine for a CouchDB database in a large information system for clinical laboratories. > > > We are blocked because we need index a large amount of data and the indexation is so slow (1 doc per minute... and increasing.....). See table: > > > > > > Docs Indexed docs por minute > > > 510... 91,6666667 > > > 833... 64,6 > > > 1111... 27,8 > > > 1222... 22,2 > > > 1500... 11,12 > > > 1572... 4 > > > > > > I suppose we are making something wrong, but we don't have expertise to know what is happening. We'd like to receive some guidelines to diagnose and solve the problem. > > > > > > Here the details.... (thank you for read). > > > > > > Our system produces 1000-3000 new docs per day with 20/30 updates in the first days of the life of the doc. Later, the docs remain unmodified (archived). > > > Before starting the system, we need index the historical documents (10 years).... > > > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed and searchable. > > > > > > I attach a couple of sample document JSON. They are complex JSONs, but are suitable for our needs. > > > > > > We configure Elasticsearch and the CouchDb river as out of the box. Only one instance, one computer running CouchDb and Elasticsearch side by side. > > > > > > RAM: 6 GB > > > CPU: Xeon W3530 2.8 Ghz. > > > SO: Windows 7 > > > > > > The river is started with this command: > > > > > > %CURL% -XPUT "[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > > > > > The indexation is made automagically by the CouchDb river, but we get the same timing indexing by hand with curl: > > > > > > %CURL% -XPOST "[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > > > > > Thank you. > > > Alberto Tostado. > > > Spain. ---
**Author:** ![Alberto\_Tostado](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@Alberto\_Tostado](https://discuss.elastic.co/u/Alberto_Tostado)\ **Post date:** [January 19, 2012, 8:37pm UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/10 "2012-01-19T20:37:35Z")
Thank you for your comments. The version could be outdated (0.16.2), but I tried a newer version (??) a month ago and obtain the same sensation. Tomorrow I'll try with a newer version (last?) and will apply the David idea of update mapping. Also, will review bigdesk plugin. Best regards. Alberto Spain On Thu, Jan 19, 2012 at 9:01 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > You're right. I was using index API and not bulk API. > > David 😉 > @dadoonet > > Le 19 janv. 2012 à 20:53, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a écrit : > > David, were you using the index API or bulk API? index API will, by > default, wait for the mapping to be applied (might wait talking to the > master of the cluster), but bulk will not (which is what the couchdriver > will do). Also, all this process is much faster in newer versions, > wondering if you would see the same behavior there. > > On Thu, Jan 19, 2012 at 9:45 PM, David Pilato \< [david@pilato.fr](mailto:david@pilato.fr) > [david@pilato.fr](mailto:david@pilato.fr)\> wrote: > > > Hi Shay, > > > > Sorry to disagree but I got exactly the same _issue_ : > > At start indexing rate was 50 docs / sec. > > Then it starts to decrease down to 1 doc / sec. > > > > When I put a _good_ mapping before indexing, indexing has a constant rate. > > > > I did not change memory settings. > > > > BTW it was some months ago and I think I was using 0.16.x or 0.15.x with > > default settings (only one node with 5 shards) > > > > So I strongly think that updating the mapping very often while having a > > huge indexing load makes indexing less quick. > > > > David 😉 > > @dadoonet > > > > Le 19 janv. 2012 à 20:36, Shay Banon \< [kimchy@gmail.com](mailto:kimchy@gmail.com)[kimchy@gmail.com](mailto:kimchy@gmail.com)\> > > a écrit : > > > > update mapping happens when a document is indexed with new json fields, > > its not that heavy, at least not one that explains the probelm you have. > > > > Yes, increasing the memory will help. I wonder though. The logging you > > pointed out is enabled only on old versions (sadly, that important logging > > information no longer happens in newer versions because of a bug in the API > > that uses to get that data). Which version are you using? > > > > I suggest you use bigdesk plugin (check the plugins page on how to > > install it: [http://www.elasticsearch.org/guide/reference/modules/plugins.html](http://www.elasticsearch.org/guide/reference/modules/plugins.html)[http://www.elasticsearch.org/guide/reference/modules/plugins.html](http://www.elasticsearch.org/guide/reference/modules/plugins.html) > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/plugins.html), and > > make sure to use the latest ES version (it has many improvements, including > > better memory control in the couchdb river). > > > > On Thu, Jan 19, 2012 at 5:59 PM, Alberto Tostado \< [atostado@gmail.com](mailto:atostado@gmail.com)[atostado@gmail.com](mailto:atostado@gmail.com) > > [atostado@gmail.com](mailto:atostado@gmail.com)\> wrote: > > > > > About "memory" and "update\_mapping"... > > > > > > The server was configured as default 256m\>\>1g, but I've changed it > > > to 3g\>\>3g (following suggestions in documentation). > > > With the change the timings have started better (100 docs/min), > > > but again the rate decreases gradually (after 4000 docs, 35 docs/min). > > > Better than before, but... > > > > > > Yes, I see a lot of "update\_mapping (dynamic)" in the console/log. 3 or > > > 4 > > > every minute. Sorry for the question, but... Is a problem the > > > "update\_mapping"? I'm afraid I'm not doing well. > > > I'm starting with Elasticsearch and I don't have clear concepts yet. > > > > > > Also, I see in console (1 per minute +/-) the following: > > > Example: > > > [2012-01-19 16:49:11,766][INFO][monitor.jvm] [Eddie > > > Brock] [gc][ConcurrentMarkSweep][ > > > 270] took [6.7s]/[4.3s], reclaimed [671.6mb], leaving [1.6gb] used, max > > > [3.1gb] > > > > > > Thank you. > > > Alberto Tostado > > > Spain > > > > > > On Thu, Jan 19, 2012 at 2:46 PM, David Pilato \< [david@pilato.fr](mailto:david@pilato.fr)[david@pilato.fr](mailto:david@pilato.fr) > > > [david@pilato.fr](mailto:david@pilato.fr)\> wrote: > > > > > > > Do you see "update mapping" in logs ? > > > > If it's the case, create the right mapping before indexing as it seems > > > > that updating the mapping often has a real cost. > > > > > > > > HTH > > > > David 😉 > > > > @dadoonet > > > > > > > > Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu \< [mberkay@gmail.com](mailto:mberkay@gmail.com)[mberkay@gmail.com](mailto:mberkay@gmail.com) > > > > [mberkay@gmail.com](mailto:mberkay@gmail.com)\> a écrit : > > > > > > > > How much memory is assigned to Elasticsearch JVM ? > > > > > > > > Regards, > > > > Berkay Mollamustafaoglu > > > > mberkay on yahoo, google and skype > > > > > > > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado \< [atostado@gmail.com](mailto:atostado@gmail.com)[atostado@gmail.com](mailto:atostado@gmail.com)[atostado@gmail.com](mailto:atostado@gmail.com) > > > > [atostado@gmail.com](mailto:atostado@gmail.com)\> wrote: > > > > > > > > > _Good morning._ > > > > > > > > > > I'm a Elasticsearch newbie user and we're trying to use it as a search > > > > > engine for a CouchDB database in a large information system for clinical > > > > > laboratories. > > > > > We are blocked because we need index a large amount of data and the > > > > > indexation is so slow (1 doc per minute... and increasing.....). See > > > > > table: > > > > > > > > > > Docs Indexed docs por minute 510... 91,6666667 833... 64,6 1111... > > > > > 27,8 1222... 22,2 1500... 11,12 1572... 4 > > > > > > > > > > I suppose we are making something wrong, but we don't have expertise > > > > > to know what is happening. We'd like to receive some guidelines to diagnose > > > > > and solve the problem. > > > > > > > > > > Here the details.... (thank you for read). > > > > > > > > > > Our system produces 1000-3000 new docs per day with 20/30 updates in > > > > > the first days of the life of the doc. Later, the docs remain unmodified > > > > > (archived). > > > > > Before starting the system, we need index the historical documents (10 > > > > > years).... > > > > > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs preindexed > > > > > and searchable. > > > > > > > > > > I attach a couple of sample document JSON. They are complex JSONs, but > > > > > are suitable for our needs. > > > > > > > > > > We configure Elasticsearch and the CouchDb river as out of the box. > > > > > Only one instance, one computer running CouchDb and Elasticsearch side by > > > > > side. > > > > > > > > > > RAM: 6 GB > > > > > CPU: Xeon W3530 2.8 Ghz. > > > > > SO: Windows 7 > > > > > > > > > > The river is started with this command: > > > > > > > > > > %CURL% -XPUT " [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta) > > > > > [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d > > > > > "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > > > > > > > > > The indexation is made automagically by the CouchDb river, but we get > > > > > the same timing indexing by hand with curl: > > > > > > > > > > %CURL% -XPOST " [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/) > > > > > [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > > > > > > > > > Thank you. > > > > > Alberto Tostado. > > > > > Spain. ---
**Author:** ![kayngee](https://avatars.discourse-cdn.com/v4/letter/k/43a26b/32.png) [@kayngee](https://discuss.elastic.co/u/kayngee)\ **Post date:** [November 9, 2012, 10:49am UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/11 "2012-11-09T10:49:47Z")
Were you able to solve this problem? On Thursday, 19 January 2012 21:37:35 UTC+1, Toastman wrote: > Thank you for your comments. > > The version could be outdated (0.16.2), but I tried a newer version (??) > a month ago and obtain the same sensation. > Tomorrow I'll try with a newer version (last?) and will apply the David > idea of update mapping. Also, will review bigdesk plugin. > > Best regards. > Alberto > Spain > > On Thu, Jan 19, 2012 at 9:01 PM, David Pilato \<[da...@pilato.fr](mailto:da...@pilato.fr)\ > > > wrote: > > > You're right. I was using index API and not bulk API. > > > > David 😉 > > @dadoonet > > > > Le 19 janv. 2012 à 20:53, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) \\> a > > écrit : > > > > David, were you using the index API or bulk API? index API will, by > > default, wait for the mapping to be applied (might wait talking to the > > master of the cluster), but bulk will not (which is what the couchdriver > > will do). Also, all this process is much faster in newer versions, > > wondering if you would see the same behavior there. > > > > On Thu, Jan 19, 2012 at 9:45 PM, David Pilato \< \ > > [da...@pilato.fr](mailto:da...@pilato.fr) \\> wrote: > > > > > Hi Shay, > > > > > > Sorry to disagree but I got exactly the same _issue_ : > > > At start indexing rate was 50 docs / sec. > > > Then it starts to decrease down to 1 doc / sec. > > > > > > When I put a _good_ mapping before indexing, indexing has a constant > > > rate. > > > > > > I did not change memory settings. > > > > > > BTW it was some months ago and I think I was using 0.16.x or 0.15.x with > > > default settings (only one node with 5 shards) > > > > > > So I strongly think that updating the mapping very often while having a > > > huge indexing load makes indexing less quick. > > > > > > David 😉 > > > @dadoonet > > > > > > Le 19 janv. 2012 à 20:36, Shay Banon \< \[kim...@gmail.com](mailto:kim...@gmail.com)\\> > > > a écrit : > > > > > > update mapping happens when a document is indexed with new json fields, > > > its not that heavy, at least not one that explains the probelm you have. > > > > > > Yes, increasing the memory will help. I wonder though. The logging you > > > pointed out is enabled only on old versions (sadly, that important logging > > > information no longer happens in newer versions because of a bug in the API > > > that uses to get that data). Which version are you using? > > > > > > I suggest you use bigdesk plugin (check the plugins page on how to > > > install it: [http://www.elasticsearch.org/guide/reference/modules/plugins.html](http://www.elasticsearch.org/guide/reference/modules/plugins.html)[http://www.elasticsearch.org/guide/reference/modules/plugins.html](http://www.elasticsearch.org/guide/reference/modules/plugins.html) > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/plugins.html), and > > > make sure to use the latest ES version (it has many improvements, including > > > better memory control in the couchdb river). > > > > > > On Thu, Jan 19, 2012 at 5:59 PM, Alberto Tostado \< \\ > > > [atos...@gmail.com](mailto:atos...@gmail.com) \\> wrote: > > > > > > > About "memory" and "update\_mapping"... > > > > > > > > The server was configured as default 256m\>\>1g, but I've changed it > > > > to 3g\>\>3g (following suggestions in documentation). > > > > With the change the timings have started better (100 docs/min), > > > > but again the rate decreases gradually (after 4000 docs, 35 docs/min). > > > > Better than before, but... > > > > > > > > Yes, I see a lot of "update\_mapping (dynamic)" in the console/log. 3 or > > > > 4 > > > > every minute. Sorry for the question, but... Is a problem the > > > > "update\_mapping"? I'm afraid I'm not doing well. > > > > I'm starting with Elasticsearch and I don't have clear concepts yet. > > > > > > > > Also, I see in console (1 per minute +/-) the following: > > > > Example: > > > > [2012-01-19 16:49:11,766][INFO][monitor.jvm] [Eddie > > > > Brock] [gc][ConcurrentMarkSweep][ > > > > 270] took [6.7s]/[4.3s], reclaimed [671.6mb], leaving [1.6gb] used, > > > > max [3.1gb] > > > > > > > > Thank you. > > > > Alberto Tostado > > > > Spain > > > > > > > > On Thu, Jan 19, 2012 at 2:46 PM, David Pilato \< \\ > > > > [da...@pilato.fr](mailto:da...@pilato.fr) \\> wrote: > > > > > > > > > Do you see "update mapping" in logs ? > > > > > If it's the case, create the right mapping before indexing as it seems > > > > > that updating the mapping often has a real cost. > > > > > > > > > > HTH > > > > > David 😉 > > > > > @dadoonet > > > > > > > > > > Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu \< \\ > > > > > [mbe...@gmail.com](mailto:mbe...@gmail.com) \\> a écrit : > > > > > > > > > > How much memory is assigned to Elasticsearch JVM ? > > > > > > > > > > Regards, > > > > > Berkay Mollamustafaoglu > > > > > mberkay on yahoo, google and skype > > > > > > > > > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado \< \\\ > > > > > [atos...@gmail.com](mailto:atos...@gmail.com) \\> wrote: > > > > > > > > > > > _Good morning._ > > > > > > > > > > > > I'm a Elasticsearch newbie user and we're trying to use it as a > > > > > > search engine for a CouchDB database in a large information system for > > > > > > clinical laboratories. > > > > > > We are blocked because we need index a large amount of data and the > > > > > > indexation is so slow (1 doc per minute... and increasing.....). See > > > > > > table: > > > > > > > > > > > > Docs Indexed docs por minute 510... 91,6666667 833... 64,6 1111... > > > > > > 27,8 1222... 22,2 1500... 11,12 1572... 4 > > > > > > > > > > > > I suppose we are making something wrong, but we don't have expertise > > > > > > to know what is happening. We'd like to receive some guidelines to diagnose > > > > > > and solve the problem. > > > > > > > > > > > > Here the details.... (thank you for read). > > > > > > > > > > > > Our system produces 1000-3000 new docs per day with 20/30 updates in > > > > > > the first days of the life of the doc. Later, the docs remain unmodified > > > > > > (archived). > > > > > > Before starting the system, we need index the historical documents > > > > > > (10 years).... > > > > > > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs > > > > > > preindexed and searchable. > > > > > > > > > > > > I attach a couple of sample document JSON. They are complex JSONs, > > > > > > but are suitable for our needs. > > > > > > > > > > > > We configure Elasticsearch and the CouchDb river as out of the box. > > > > > > Only one instance, one computer running CouchDb and Elasticsearch side by > > > > > > side. > > > > > > > > > > > > RAM: 6 GB > > > > > > CPU: Xeon W3530 2.8 Ghz. > > > > > > SO: Windows 7 > > > > > > > > > > > > The river is started with this command: > > > > > > > > > > > > %CURL% -XPUT " [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta) > > > > > > [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d > > > > > > "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > > > > > > > > > > > The indexation is made automagically by the CouchDb river, but we get > > > > > > the same timing indexing by hand with curl: > > > > > > > > > > > > %CURL% -XPOST " [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/) > > > > > > [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > > > > > > > > > > > Thank you. > > > > > > Alberto Tostado. > > > > > > Spain. -- ---
**Author:** ![kayngee](https://avatars.discourse-cdn.com/v4/letter/k/43a26b/32.png) [@kayngee](https://discuss.elastic.co/u/kayngee)\ **Post date:** [November 14, 2012, 11:54am UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/12 "2012-11-14T11:54:53Z")
Did you manage to solve this problem? On Thursday, 19 January 2012 21:37:35 UTC+1, Toastman wrote: > Thank you for your comments. > > The version could be outdated (0.16.2), but I tried a newer version (??) > a month ago and obtain the same sensation. > Tomorrow I'll try with a newer version (last?) and will apply the David > idea of update mapping. Also, will review bigdesk plugin. > > Best regards. > Alberto > Spain > > On Thu, Jan 19, 2012 at 9:01 PM, David Pilato \<[da...@pilato.fr](mailto:da...@pilato.fr)\ > > > wrote: > > > You're right. I was using index API and not bulk API. > > > > David 😉 > > @dadoonet > > > > Le 19 janv. 2012 à 20:53, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) \\> a > > écrit : > > > > David, were you using the index API or bulk API? index API will, by > > default, wait for the mapping to be applied (might wait talking to the > > master of the cluster), but bulk will not (which is what the couchdriver > > will do). Also, all this process is much faster in newer versions, > > wondering if you would see the same behavior there. > > > > On Thu, Jan 19, 2012 at 9:45 PM, David Pilato \< \ > > [da...@pilato.fr](mailto:da...@pilato.fr) \\> wrote: > > > > > Hi Shay, > > > > > > Sorry to disagree but I got exactly the same _issue_ : > > > At start indexing rate was 50 docs / sec. > > > Then it starts to decrease down to 1 doc / sec. > > > > > > When I put a _good_ mapping before indexing, indexing has a constant > > > rate. > > > > > > I did not change memory settings. > > > > > > BTW it was some months ago and I think I was using 0.16.x or 0.15.x with > > > default settings (only one node with 5 shards) > > > > > > So I strongly think that updating the mapping very often while having a > > > huge indexing load makes indexing less quick. > > > > > > David 😉 > > > @dadoonet > > > > > > Le 19 janv. 2012 à 20:36, Shay Banon \< \[kim...@gmail.com](mailto:kim...@gmail.com)\\> > > > a écrit : > > > > > > update mapping happens when a document is indexed with new json fields, > > > its not that heavy, at least not one that explains the probelm you have. > > > > > > Yes, increasing the memory will help. I wonder though. The logging you > > > pointed out is enabled only on old versions (sadly, that important logging > > > information no longer happens in newer versions because of a bug in the API > > > that uses to get that data). Which version are you using? > > > > > > I suggest you use bigdesk plugin (check the plugins page on how to > > > install it: [http://www.elasticsearch.org/guide/reference/modules/plugins.html](http://www.elasticsearch.org/guide/reference/modules/plugins.html)[http://www.elasticsearch.org/guide/reference/modules/plugins.html](http://www.elasticsearch.org/guide/reference/modules/plugins.html) > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/plugins.html), and > > > make sure to use the latest ES version (it has many improvements, including > > > better memory control in the couchdb river). > > > > > > On Thu, Jan 19, 2012 at 5:59 PM, Alberto Tostado \< \\ > > > [atos...@gmail.com](mailto:atos...@gmail.com) \\> wrote: > > > > > > > About "memory" and "update\_mapping"... > > > > > > > > The server was configured as default 256m\>\>1g, but I've changed it > > > > to 3g\>\>3g (following suggestions in documentation). > > > > With the change the timings have started better (100 docs/min), > > > > but again the rate decreases gradually (after 4000 docs, 35 docs/min). > > > > Better than before, but... > > > > > > > > Yes, I see a lot of "update\_mapping (dynamic)" in the console/log. 3 or > > > > 4 > > > > every minute. Sorry for the question, but... Is a problem the > > > > "update\_mapping"? I'm afraid I'm not doing well. > > > > I'm starting with Elasticsearch and I don't have clear concepts yet. > > > > > > > > Also, I see in console (1 per minute +/-) the following: > > > > Example: > > > > [2012-01-19 16:49:11,766][INFO][monitor.jvm] [Eddie > > > > Brock] [gc][ConcurrentMarkSweep][ > > > > 270] took [6.7s]/[4.3s], reclaimed [671.6mb], leaving [1.6gb] used, > > > > max [3.1gb] > > > > > > > > Thank you. > > > > Alberto Tostado > > > > Spain > > > > > > > > On Thu, Jan 19, 2012 at 2:46 PM, David Pilato \< \\ > > > > [da...@pilato.fr](mailto:da...@pilato.fr) \\> wrote: > > > > > > > > > Do you see "update mapping" in logs ? > > > > > If it's the case, create the right mapping before indexing as it seems > > > > > that updating the mapping often has a real cost. > > > > > > > > > > HTH > > > > > David 😉 > > > > > @dadoonet > > > > > > > > > > Le 19 janv. 2012 à 14:24, Berkay Mollamustafaoglu \< \\ > > > > > [mbe...@gmail.com](mailto:mbe...@gmail.com) \\> a écrit : > > > > > > > > > > How much memory is assigned to Elasticsearch JVM ? > > > > > > > > > > Regards, > > > > > Berkay Mollamustafaoglu > > > > > mberkay on yahoo, google and skype > > > > > > > > > > On Thu, Jan 19, 2012 at 8:13 AM, Alberto Tostado \< \\\ > > > > > [atos...@gmail.com](mailto:atos...@gmail.com) \\> wrote: > > > > > > > > > > > _Good morning._ > > > > > > > > > > > > I'm a Elasticsearch newbie user and we're trying to use it as a > > > > > > search engine for a CouchDB database in a large information system for > > > > > > clinical laboratories. > > > > > > We are blocked because we need index a large amount of data and the > > > > > > indexation is so slow (1 doc per minute... and increasing.....). See > > > > > > table: > > > > > > > > > > > > Docs Indexed docs por minute 510... 91,6666667 833... 64,6 1111... > > > > > > 27,8 1222... 22,2 1500... 11,12 1572... 4 > > > > > > > > > > > > I suppose we are making something wrong, but we don't have expertise > > > > > > to know what is happening. We'd like to receive some guidelines to diagnose > > > > > > and solve the problem. > > > > > > > > > > > > Here the details.... (thank you for read). > > > > > > > > > > > > Our system produces 1000-3000 new docs per day with 20/30 updates in > > > > > > the first days of the life of the doc. Later, the docs remain unmodified > > > > > > (archived). > > > > > > Before starting the system, we need index the historical documents > > > > > > (10 years).... > > > > > > 1500 docs \* 30 days \* 12 months \* 10 years = 5.400.000 docs > > > > > > preindexed and searchable. > > > > > > > > > > > > I attach a couple of sample document JSON. They are complex JSONs, > > > > > > but are suitable for our needs. > > > > > > > > > > > > We configure Elasticsearch and the CouchDb river as out of the box. > > > > > > Only one instance, one computer running CouchDb and Elasticsearch side by > > > > > > side. > > > > > > > > > > > > RAM: 6 GB > > > > > > CPU: Xeon W3530 2.8 Ghz. > > > > > > SO: Windows 7 > > > > > > > > > > > > The river is started with this command: > > > > > > > > > > > > %CURL% -XPUT " [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)[http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta) > > > > > > [http://127.0.0.1:9200/\_river/hm/\_meta](http://127.0.0.1:9200/_river/hm/_meta)" -d > > > > > > "{"type":"couchdb","couchdb":{"host":"127.0.0.1","port":5984,"db":"hm","filter":null}}" > > > > > > > > > > > > The indexation is made automagically by the CouchDb river, but we get > > > > > > the same timing indexing by hand with curl: > > > > > > > > > > > > %CURL% -XPOST " [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)[http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/) > > > > > > [http://127.0.0.1:9200/hm/order/](http://127.0.0.1:9200/hm/order/)" -d @file.txt > > > > > > > > > > > > Thank you. > > > > > > Alberto Tostado. > > > > > > Spain. -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:04am UTC](https://discuss.elastic.co/t/ultra-slow-indexing/6441/13 "2017-07-06T03:04:24Z")