# High fetch times **URL:** \ **Category:** Elasticsearch\ **Created:** [January 15, 2014, 2:45pm UTC](https://discuss.elastic.co/t/high-fetch-times/15244 "2014-01-15T14:45:57Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Arik\_Gaisler](https://avatars.discourse-cdn.com/v4/letter/a/6f9a4e/32.png) [@Arik\_Gaisler](https://discuss.elastic.co/u/Arik_Gaisler)\ **Post date:** [January 15, 2014, 2:45pm UTC](https://discuss.elastic.co/t/high-fetch-times/15244/1 "2014-01-15T14:45:57Z")
Im using ElasticHQ to monitor our cluster and I am noticing that our Search Fetch times are about 5X more then query times, which seems unreasonable (20ms vs ~4ms) Im also noticing a 1.44 mb swap space on one of the nodes (the other node is at 0) We're using a 2 node cluster, 2 shards 1 replica on EC2 m1.small instances Our index contains about 27K docs, total size of ~ 60mb (pretty small - hence the small instance) Any suggestions to why? Arik -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7307cbfe-55d3-41df-a2dc-86b74450a3bf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7307cbfe-55d3-41df-a2dc-86b74450a3bf%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [January 15, 2014, 2:48pm UTC](https://discuss.elastic.co/t/high-fetch-times/15244/2 "2014-01-15T14:48:58Z")
What is the query type you are using? > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) On Wed, Jan 15, 2014 at 4:45 PM, Arik Gaisler [arikgaisler@gmail.com](mailto:arikgaisler@gmail.com) wrote: > Im using ElasticHQ to monitor our cluster and I am noticing that our > Search Fetch times are about 5X more then query times, which seems > unreasonable (20ms vs ~4ms) > > Im also noticing a 1.44 mb swap space on one of the nodes (the other node > is at 0) > > We're using a 2 node cluster, 2 shards 1 replica on EC2 m1.small instances > > Our index contains about 27K docs, total size of ~ 60mb (pretty small - > hence the small instance) > > Any suggestions to why? > > Arik > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/7307cbfe-55d3-41df-a2dc-86b74450a3bf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7307cbfe-55d3-41df-a2dc-86b74450a3bf%40googlegroups.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtqvE-OkuAMNOm%3DKDA1SED\_x86NK8z2bkL%2BRoZuzO%3DZhA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtqvE-OkuAMNOm%3DKDA1SED_x86NK8z2bkL%2BRoZuzO%3DZhA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Arik\_Gaisler](https://avatars.discourse-cdn.com/v4/letter/a/6f9a4e/32.png) [@Arik\_Gaisler](https://discuss.elastic.co/u/Arik_Gaisler)\ **Post date:** [January 15, 2014, 3:31pm UTC](https://discuss.elastic.co/t/high-fetch-times/15244/3 "2014-01-15T15:31:12Z")
We are doing a query then fetch serach type On Wednesday, January 15, 2014 4:48:58 PM UTC+2, Itamar Syn-Hershko wrote: > What is the query type you are using? > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html) > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) > > On Wed, Jan 15, 2014 at 4:45 PM, Arik Gaisler \<[arikg...@gmail.com](mailto:arikg...@gmail.com)\ > > > wrote: > > > Im using ElasticHQ to monitor our cluster and I am noticing that our > > Search Fetch times are about 5X more then query times, which seems > > unreasonable (20ms vs ~4ms) > > > > Im also noticing a 1.44 mb swap space on one of the nodes (the other node > > is at 0) > > > > We're using a 2 node cluster, 2 shards 1 replica on EC2 m1.small instances > > > > Our index contains about 27K docs, total size of ~ 60mb (pretty small - > > hence the small instance) > > > > Any suggestions to why? > > > > Arik > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/7307cbfe-55d3-41df-a2dc-86b74450a3bf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7307cbfe-55d3-41df-a2dc-86b74450a3bf%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/fc28fd75-67c9-46f2-99b2-03cb48bac34b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fc28fd75-67c9-46f2-99b2-03cb48bac34b%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [January 15, 2014, 3:33pm UTC](https://discuss.elastic.co/t/high-fetch-times/15244/4 "2014-01-15T15:33:51Z")
I believe this makes sense then - search is entirely in-memory operation (after a certain warm up), and fetch involves disk IO. Try keep your docs as small as possible and don't disable \_source, but other than that you're bounded by the speed of your disks. -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) On Wed, Jan 15, 2014 at 5:31 PM, Arik Gaisler [arikgaisler@gmail.com](mailto:arikgaisler@gmail.com) wrote: > We are doing a query then fetch serach type > > On Wednesday, January 15, 2014 4:48:58 PM UTC+2, Itamar Syn-Hershko wrote: > > > What is the query type you are using? [http://www](http://www). > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://elasticsearch.org/guide/en/elasticsearch/reference/) > > current/search-request-search-type.html > > > > -- > > > > Itamar Syn-Hershko > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > Freelance Developer & Consultant > > Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) > > > > On Wed, Jan 15, 2014 at 4:45 PM, Arik Gaisler [arikg...@gmail.com](mailto:arikg...@gmail.com) wrote: > > > > > Im using ElasticHQ to monitor our cluster and I am noticing that our > > > Search Fetch times are about 5X more then query times, which seems > > > unreasonable (20ms vs ~4ms) > > > > > > Im also noticing a 1.44 mb swap space on one of the nodes (the other > > > node is at 0) > > > > > > We're using a 2 node cluster, 2 shards 1 replica on EC2 m1.small > > > instances > > > > > > Our index contains about 27K docs, total size of ~ 60mb (pretty small - > > > hence the small instance) > > > > > > Any suggestions to why? > > > > > > Arik > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/7307cbfe-55d3-41df-a2dc-86b74450a3bf% > > > [40googlegroups.com](http://40googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/fc28fd75-67c9-46f2-99b2-03cb48bac34b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fc28fd75-67c9-46f2-99b2-03cb48bac34b%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZvJiXYOBt-cEUvK6BXPUDm\_jRRxHOhGNsOCp7BMg1bwdg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZvJiXYOBt-cEUvK6BXPUDm_jRRxHOhGNsOCp7BMg1bwdg%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Arik\_Gaisler](https://avatars.discourse-cdn.com/v4/letter/a/6f9a4e/32.png) [@Arik\_Gaisler](https://discuss.elastic.co/u/Arik_Gaisler)\ **Post date:** [January 15, 2014, 3:56pm UTC](https://discuss.elastic.co/t/high-fetch-times/15244/5 "2014-01-15T15:56:41Z")
Thanks Itamar Im also seeing that the refresh is pretty high137.4ms Another factor is that the fetch times are constantly growing - while the query times are cinctantly the same Arik On Wednesday, January 15, 2014 5:33:51 PM UTC+2, Itamar Syn-Hershko wrote: > I believe this makes sense then - search is entirely in-memory operation > (after a certain warm up), and fetch involves disk IO. Try keep your docs > as small as possible and don't disable \_source, but other than that you're > bounded by the speed of your disks. > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) > > On Wed, Jan 15, 2014 at 5:31 PM, Arik Gaisler \<[arikg...@gmail.com](mailto:arikg...@gmail.com)\ > > > wrote: > > > We are doing a query then fetch serach type > > > > On Wednesday, January 15, 2014 4:48:58 PM UTC+2, Itamar Syn-Hershko wrote: > > > > > What is the query type you are using? [http://www](http://www). > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://elasticsearch.org/guide/en/elasticsearch/reference/) > > > current/search-request-search-type.html > > > > > > -- > > > > > > Itamar Syn-Hershko > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > Freelance Developer & Consultant > > > Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) > > > > > > On Wed, Jan 15, 2014 at 4:45 PM, Arik Gaisler [arikg...@gmail.com](mailto:arikg...@gmail.com)wrote: > > > > > > > Im using ElasticHQ to monitor our cluster and I am noticing that our > > > > Search Fetch times are about 5X more then query times, which seems > > > > unreasonable (20ms vs ~4ms) > > > > > > > > Im also noticing a 1.44 mb swap space on one of the nodes (the other > > > > node is at 0) > > > > > > > > We're using a 2 node cluster, 2 shards 1 replica on EC2 m1.small > > > > instances > > > > > > > > Our index contains about 27K docs, total size of ~ 60mb (pretty small - > > > > hence the small instance) > > > > > > > > Any suggestions to why? > > > > > > > > Arik > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/7307cbfe-55d3-41df-a2dc-86b74450a3bf% > > > > [40googlegroups.com](http://40googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/fc28fd75-67c9-46f2-99b2-03cb48bac34b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fc28fd75-67c9-46f2-99b2-03cb48bac34b%40googlegroups.com) > > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c1cba3ba-bd12-42f8-ba4d-06b64dc821a4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c1cba3ba-bd12-42f8-ba4d-06b64dc821a4%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [January 15, 2014, 4:03pm UTC](https://discuss.elastic.co/t/high-fetch-times/15244/6 "2014-01-15T16:03:15Z")
If such latency worries you, use in-memory structures to pull the original documents based on the IDs of the results. Your dataset is small enough to do that, really. But I suspect this is premature optimization that you are trying to do. -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) On Wed, Jan 15, 2014 at 5:56 PM, Arik Gaisler [arikgaisler@gmail.com](mailto:arikgaisler@gmail.com) wrote: > Thanks Itamar > > Im also seeing that the refresh is pretty high137.4ms > > Another factor is that the fetch times are constantly growing - while the > query times are cinctantly the same > > Arik > > On Wednesday, January 15, 2014 5:33:51 PM UTC+2, Itamar Syn-Hershko wrote: > > > I believe this makes sense then - search is entirely in-memory operation > > (after a certain warm up), and fetch involves disk IO. Try keep your docs > > as small as possible and don't disable \_source, but other than that you're > > bounded by the speed of your disks. > > > > -- > > > > Itamar Syn-Hershko > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > Freelance Developer & Consultant > > Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) > > > > On Wed, Jan 15, 2014 at 5:31 PM, Arik Gaisler [arikg...@gmail.com](mailto:arikg...@gmail.com) wrote: > > > > > We are doing a query then fetch serach type > > > > > > On Wednesday, January 15, 2014 4:48:58 PM UTC+2, Itamar Syn-Hershko > > > wrote: > > > > > > > What is the query type you are using? [http://www.elasticsearc](http://www.elasticsearc) > > > > [h.org/guide/en/elasticsearch/reference/current/search-request-search-](http://h.org/guide/en/elasticsearch/reference/current/search-request-search-) > > > > type.html > > > > > > > > -- > > > > > > > > Itamar Syn-Hershko > > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > > Freelance Developer & Consultant > > > > Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) > > > > > > > > On Wed, Jan 15, 2014 at 4:45 PM, Arik Gaisler [arikg...@gmail.com](mailto:arikg...@gmail.com)wrote: > > > > > > > > > Im using ElasticHQ to monitor our cluster and I am noticing that our > > > > > Search Fetch times are about 5X more then query times, which seems > > > > > unreasonable (20ms vs ~4ms) > > > > > > > > > > Im also noticing a 1.44 mb swap space on one of the nodes (the other > > > > > node is at 0) > > > > > > > > > > We're using a 2 node cluster, 2 shards 1 replica on EC2 m1.small > > > > > instances > > > > > > > > > > Our index contains about 27K docs, total size of ~ 60mb (pretty small > > > > > > > > > > - hence the small instance) > > > > > > > > > > Any suggestions to why? > > > > > > > > > > Arik > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > msgid/elasticsearch/7307cbfe-55d3-41df-a2dc-86b74450a3bf%40goo > > > > > [glegroups.com](http://glegroups.com). > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/fc28fd75-67c9-46f2-99b2-03cb48bac34b% > > > > [40googlegroups.com](http://40googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/c1cba3ba-bd12-42f8-ba4d-06b64dc821a4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c1cba3ba-bd12-42f8-ba4d-06b64dc821a4%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zt866hyaW%3DCZB2N8W2AN2nLVWpr155zHvUAMcv7tUan9w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zt866hyaW%3DCZB2N8W2AN2nLVWpr155zHvUAMcv7tUan9w%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:56am UTC](https://discuss.elastic.co/t/high-fetch-times/15244/7 "2017-07-06T01:56:43Z")