# Setup to minimize search latency

**URL:** https://discuss.elastic.co/t/setup-to-minimize-search-latency/17653
**Category:** Elasticsearch
**Created:** [May 21, 2014, 7:33pm UTC](https://discuss.elastic.co/t/setup-to-minimize-search-latency/17653 "2014-05-21T19:33:00Z")
**Posts on this page:** 1
**Showing post:** 2

<div class="post-metadata">

### Author: ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)
#### Post date: [May 23, 2014, 12:27am UTC](https://discuss.elastic.co/t/setup-to-minimize-search-latency/17653/2 "2014-05-23T00:27:55Z")

</div>

On Wed, May 21, 2014 at 9:33 PM, Dustin Boswell [dboswell@gmail.com](mailto:dboswell@gmail.com) wrote:

> Hi everyone,  
> I have a 10 million document index, and multiple high-memory (e.g. 250GB  
> ram, 32 cores) machines available. I'd like to do everything possible to  
> keep search latency as low as possible (\< 50ms ideally), especially during  
> a high-throughput environment. I know it depends a lot on the query, but  
> to start with I'm asking about general index/cluster settings.
> 
> Here's a list of things I'm doing so far:
> 
> - ES\_HEAP\_SIZE=100g
> - machine has no swap
> - 20 shards
> 
> Are there any other settings or search parameters I should be aware of?

If you care about latency, it might make sense to configure a smaller heap  
size. The issue with large heaps is that they take longer to collect,  
especially for collections of the old generation (I'm talking about minutes  
here). I would recommend setting the heap size at at most 30GB (which will  
also allow you to benefit from compressed pointers), one way to go this  
route could be to start several nodes per physical machine.

> Also, I'm wondering how many shards is recommended in my case. Having more  
> shards helps reduce latency by parallelizing the work, but at some point  
> the overhead of fanning out the requests and collecting the partial results  
> will take over and latency would get worse. Is there a rule of thumb for a  
> sweet spot that others have found?

If you don't have a lot of traffic, you could think about configuring  
num\_shards = total\_num\_cpus / num\_concurrent\_queries. But as you said,  
there is also some overhead to large numbers of shards so this deserves  
testing.

> The volume of updates to the index is relatively small (500K/day), but  
> bursty. From initial testing, it seems like updates being issued can  
> increase the search latency happening on the same machine. Is there a good  
> way to "isolate" search and updates, either by some setting, or splitting  
> up the cluster somehow to have dedicated update nodes and dedicated search  
> nodes? (Not sure how you'd deploy a setup like this, or control where the  
> search/update calls went.)

> The query I'm optimizing for will have a text search component and a  
> geo-restrict component, maybe something like this:  
> {  
> "query": {  
> // query may get more complex in the future  
> "match": { "\_all": "my search terms" }  
> },  
> "filter": {  
> "geo\_distance": {  
> "distance": "100km",  
> "location": {  
> "lat": 34.04,  
> "lon": -118.49  
> }  
> }  
> }  
> }
> 
> For the geo filter, I've tried the optimize\_bbox option, and the default  
> of "memory" seemed to work the best, surprisingly. I haven't tried using  
> geohash yet, and I can't tell from the docs how one might use it, but maybe  
> that is inherently faster since it uses indexes?

The bbox optimization is useful if your geo query matches a small portion  
of your index. Maybe the issue here is that with such a large radius, you  
match most of your documents?

> Unfortunately, there are a lot of unique locations in my query stream, so  
> I don't know if caching this filter will work. (Each filter cache consumes  
> about 1 bit in memory per document, is that right? So about 1.25MB in my  
> case. Storing the most frequent 10,000 of these would take up about 12.5GB  
> of ram. So maybe that's doable...)

One thing to beware of is that if you cache a geo filter, Elasticsearch  
will need to evaluate it against all documents from your index before  
caching it. On the other hand by default (if the filter is not cached), the  
filter will be only evaluated on documents that match the query (but for  
all queries not the first one). So unless you have good reasons to think  
that your geo filter will be reused, I would recommend against caching it.

--  
Adrien Grand

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j6Y%2BdgRns8JwcuigDprR0\_Yr8PN58cyfKhYVcFYnE4m7g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j6Y%2BdgRns8JwcuigDprR0_Yr8PN58cyfKhYVcFYnE4m7g%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

_[View the full topic](https://discuss.elastic.co/t/setup-to-minimize-search-latency/17653)._