# Very slow ElasticSearch Index **URL:** \ **Category:** Elasticsearch\ **Created:** [December 17, 2013, 7:58am UTC](https://discuss.elastic.co/t/very-slow-elasticsearch-index/14907 "2013-12-17T07:58:56Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![lekkie](https://avatars.discourse-cdn.com/v4/letter/l/46a35a/32.png) [@lekkie](https://discuss.elastic.co/u/lekkie)\ **Post date:** [December 17, 2013, 7:58am UTC](https://discuss.elastic.co/t/very-slow-elasticsearch-index/14907/1 "2013-12-17T07:58:56Z")
Hi guys, We index a document at 4000 milliseconds on ElasticSearch, which I think is too slow especially for the amount of resources we have setup. We will like to index at the rate of 500tps. Each document weighs between 20K and 30K. How many indexes are advisable to be done at once (assuming we can afford to send multiple http index request to the server at once)? I understand bulk indexing is a preferred approach, for a 30K document how much can be bulked at once? How many http bulk request (supposing I am using a multi-threaded http client ot make requests) is advisable to make? I will appreciate suggestions and how to index this document as fast as possible. We have two nodes set up, the config below is for one out of the two: Shards: 5 Replica: 1 "nodes" : { "T5l5mvIdQsW3je7WmSPOcg" : { "name" : "SEARCH-01", "version" : "0.90.7", "attributes" : { "rack\_id" : "prod", "max\_local\_storage\_nodes" : "1" }, "settings" : { "node.rack\_id" : "prod", "action.disable\_delete\_all\_indices" : "true", "cloud.node.auto\_attributes" : "true", "indices.ttl.interval" : "90d", "node.max\_local\_storage\_nodes" : "1", "bootstrap.mlockall" : "true", "index.mapper.dynamic" : "true", "cluster.routing.allocation.awareness.attributes" : "rack\_id", "discovery.zen.minimum\_master\_nodes" : "3", "gateway.expected\_nodes" : "1", "discovery.zen.ping.unicast.hosts" : "172.25.15.170,172.25.15.172,172.46.1.170,172.46.1.172", "discovery.zen.ping.multicast.enabled" : "false", "action.auto\_create\_index" : "true" }, "os" : { "refresh\_interval" : 1000, "available\_processors" : 8, "cpu" : { "vendor" : "Intel", "model" : "Xeon", "mhz" : 2600, "total\_cores" : 8, "total\_sockets" : 2, "cores\_per\_socket" : 4, "cache\_size" : "20kb", "cache\_size\_in\_bytes" : 20480 }, "mem" : { "total" : "17.5gb", "total\_in\_bytes" : 18836545536 }, "swap" : { "total" : "5.8gb", "total\_in\_bytes" : 6274670592 } }, "process" : { "refresh\_interval" : 1000, "id" : 3459, "max\_file\_descriptors" : 64000 }, "jvm" : { "pid" : 3459, "version" : "1.7.0\_45", "vm\_name" : "OpenJDK 64-Bit Server VM", "vm\_version" : "24.45-b08", "vm\_vendor" : "Oracle Corporation", "start\_time" : 1386953353018, "mem" : { "heap\_init" : "10.5gb", "heap\_init\_in\_bytes" : 11301552128, "heap\_max" : "10.4gb", "heap\_max\_in\_bytes" : 11231821824, "non\_heap\_init" : "23.1mb", "non\_heap\_init\_in\_bytes" : 24313856, "non\_heap\_max" : "214mb", "non\_heap\_max\_in\_bytes" : 224395264, "direct\_max" : "10.4gb", "direct\_max\_in\_bytes" : 11231821824 } }, "thread\_pool" : { "generic" : { "type" : "cached", "keep\_alive" : "30s" }, "index" : { "type" : "fixed", "min" : 8, "max" : 8, "queue\_size" : "200" }, "get" : { "type" : "fixed", "min" : 8, "max" : 8, "queue\_size" : "1k" }, "snapshot" : { "type" : "scaling", "min" : 1, "max" : 4, "keep\_alive" : "5m" }, "merge" : { "type" : "scaling", "min" : 1, "max" : 4, "keep\_alive" : "5m" }, "suggest" : { "type" : "fixed", "min" : 8, "max" : 8, "queue\_size" : "1k" }, "bulk" : { "type" : "fixed", "min" : 8, "max" : 8, "queue\_size" : "50" }, "optimize" : { "type" : "fixed", "min" : 1, "max" : 1 }, "warmer" : { "type" : "scaling", "min" : 1, "max" : 4, "keep\_alive" : "5m" }, "flush" : { "type" : "scaling", "min" : 1, "max" : 4, "keep\_alive" : "5m" }, "search" : { "type" : "fixed", "min" : 24, "max" : 24, "queue\_size" : "1k" }, "percolate" : { "type" : "fixed", "min" : 8, "max" : 8, "queue\_size" : "1k" }, "management" : { "type" : "scaling", "min" : 1, "max" : 5, "keep\_alive" : "5m" }, "refresh" : { "type" : "scaling", "min" : 1, "max" : 4, "keep\_alive" : "5m" } }, "network" : { "refresh\_interval" : 5000 }, "http" : { "max\_content\_length" : "100mb", "max\_content\_length\_in\_bytes" : 104857600 }, "plugins" : [] } } The Mapping is dynamically created because we create types daily and it looks like: { "consumers-20131216": { "properties": { "requestData": { "type": "string" }, "requestTimestamp": { "type": "date", "format": "dateOptionalTime" }, "responseData": { "type": "string" }, "responseTimestamp": { "type": "date", "format": "dateOptionalTime" }, "sequenceId": { "type": "long" }, "service": { "type": "string" }, "systemResponseCode": { "type": "string" }, "systemResponseMessage": { "type": "string" }, "transactionComponentTypeId": { "type": "long" }, "transactionLogId": { "type": "long" }, "user": { "type": "string" } } } } Regards. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4af4e271-048a-442a-bdc6-2f529824df06%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4af4e271-048a-442a-bdc6-2f529824df06%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:00am UTC](https://discuss.elastic.co/t/very-slow-elasticsearch-index/14907/2 "2017-07-06T02:00:43Z")