# Queries get slow while indexing documents **URL:** \ **Category:** Elasticsearch\ **Created:** [October 7, 2020, 9:13am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243 "2020-10-07T09:13:01Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![taichi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/taichi/32/46431_2.png) [@taichi](https://discuss.elastic.co/u/taichi)\ **Post date:** [October 7, 2020, 9:13am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/1 "2020-10-07T09:13:01Z")
I have an index with approximately 2.3M documents. It usually returns results of query within 1~2 seconds, but it gets very slow (sometimes as long as 20~30s) while I'm inserting data into the index. I insert data in RDB into the index in a batch process using Bulk API (about 100 documents per second). Why does this happen? I could not find any documentations about the behavior. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 7, 2020, 9:28am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/2 "2020-10-07T09:28:40Z")
What kind of query are you running? Could you share an example? What is the response? Could you share are least the first 20 lines? Also what kind of hardware do you have? Heap size, SSD vs HDD... ---
**Author:** ![taichi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/taichi/32/46431_2.png) [@taichi](https://discuss.elastic.co/u/taichi)\ **Post date:** [October 8, 2020, 1:59am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/3 "2020-10-08T01:59:41Z")
Counting all documents ( like `_count {"query": "match_all": {}}` ). Other search queries also sometimes get slow. I cannot share the response, but response size is small. I'm using m5.xlarge.elasticsearch \* 2 (4 vCPU, 16 GB RAM, 512 GB SSD) for data nodes. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 8, 2020, 6:15am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/4 "2020-10-08T06:15:54Z")
> [@taichi](#): > > I cannot share the response, but response size is small. Why? There's nothing secret in the first lines. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 8, 2020, 6:17am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/5 "2020-10-08T06:17:58Z")
BTW a `match_all` taking normally some seconds is a way too slow. What is the output of: ```auto GET / GET /_cat/nodes?v GET /_cat/health?v GET /_cat/indices?v ``` If some outputs are too big, please share them on [gist.github.com](http://gist.github.com) and link them here. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 8, 2020, 6:30am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/6 "2020-10-08T06:30:55Z")
Indexing can be both CPU and dusk I/O intensive. I do not know what monitoring you have access to but it would be good to try to identify if CPU or dusk I/O is limiting performance. If you are using gp2 EBS it gets IOPS proportional to size and since large scroll queries results in a lot of disk I/O so that is probably what I would start with. ---
**Author:** ![taichi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/taichi/32/46431_2.png) [@taichi](https://discuss.elastic.co/u/taichi)\ **Post date:** [October 8, 2020, 6:31am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/7 "2020-10-08T06:31:19Z")
Sorry, search response is like this (just one of examples): ``` { "took": 6554, "timed_out": false, "_shards": { "total": 2, "successful": 2, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": 6.086506, "hits": [...] } } ``` count response is: ``` { "count": 2300000, "_shards": { "total": 2, "successful": 2, "skipped": 0, "failed": 0 } } ``` ---
**Author:** ![taichi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/taichi/32/46431_2.png) [@taichi](https://discuss.elastic.co/u/taichi)\ **Post date:** [October 8, 2020, 6:37am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/8 "2020-10-08T06:37:56Z")
I created gist for this. > > > There are more than three files. show original ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 8, 2020, 6:52am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/9 "2020-10-08T06:52:01Z")
You may also look at making indexing more efficient, e.g. by increasing the refresh interval of the index if you have not already done so. You might even disable it during the bulk load and just enable it afterwards. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 5, 2020, 6:52am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243/10 "2020-11-05T06:52:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.