Can I enable "Shard Request Cache" without performance loss?

Elastic Stack Elasticsearch
1

Hi.

I realized that ES 2.x has two types of caches (i. Node Query Cache, ii. Shard Request Cache)

What I really wanted is "Shard Request Cache" which is disabled by default, so that I'd like to enable it. Before enabling it, I'm wondering why "Shard Request Cache" is disabled by default and why it's size is so small (1% of heap size)

My question is..

  • Is there any performance loss if I enable shard query cache?
    • e.g) Cache lock contention or indexing rate decreases for any reasons.
  • Is 1% sufficient for shard level cache? if hard to answer, why is it small compared to "Node Query Cache" (10% of heap by default)

I'm using ES under the below circumstances:

  • ES 2.3.2
  • 20GB of Heap
  • daily index (only today's index is indexed by 5k/sec)
  • use only constant_score query. no scoring
2

Hi @Brad_Jungsu_Heo,

Before Elasticsearch 5, some queries were not benefiting enough from this cache (see also a related Github issue). In Elasticsearch 5, we significantly improved it and were finally able to enable it by default in Elasticsearch 5.

Based on the usefulness of this cache and the vast range of use-cases this is a conservative default.

It is always hard to guess upfront whether you'll benefit from changing a setting. I suggest you create a benchmark and compare the two alternatives. Then you can make an educated guess. We develop a dedicated tool, called Rally for exactly that purpose.

Daniel

3

Excellent!

Helped me a lot. Thanks.

4

Great, you're welcome. :slight_smile:

5

Sorry to piggyback on another thread, but I have a very similar question --

Is there a way to enable this for an index template? It's not maintainable to have to set this setting for every daily index, as seen in the 'Shard Request cache' page.

6

Hi @seth.yes,

I don't see any reason why this shouldn't work. This should do:

PUT /_template/default_template
{
    "template" : "*",
    "order" : 0,
    "settings" : {
        "index.requests.cache.enable": true
    }
}

Daniel

1 Like
7

That indeed worked. Thanks Daniel!