Per type Query Cache invalidation

Elastic Stack Elasticsearch
1

Hi all,

I am confused about cache invalidation. It seams that filter cache is per segment. That means that I can add data without totally invalidating this cache. Is it the same for ShardQueryCache? Can I add data without invalidating that cache?

Thank you.

2

Hi,

I finally found the answer in the following thread:
http://elasticsearch-users.115913.n3.nabble.com/Shard-query-cache-size-td4074827.html
where Adrien Grand mentions that :
"- the query cache caches entire requests per index, and is competely invalidated across all data every time the refresh interval ticks over AND there have been changes since the last refresh

  • the filter cache caches matching documents per segment, it is invalidated per segment only when a segment goes away (typically because it's been merged to a larger segment), which is unfrequent for large segments
  • the fielddata cache caches the document->value mapping per segment and has the same invalidation rules as the filter cache"
    So the shard query cache is not per segment. It is a pity because if we have an index with two types : Updating data of one type will invalidate the cache for the other type.
    Do you know if this is an issue that can be addressed or if there is underlying restriction due to the shard structure that prevent from doing per type query cache.

Thank you,

3

Sorry I think there is some confusion due to the fact that we merged queries and filters and renamed the filter cache to query cache, and the query cache to request cache.

In short the query cache caches data for individual queries per segment. This makes it likely to be reused even if you keep adding data since large segments typically get merged very rarely.

On the other hand, the request cache caches entire search requests and is discarded every time you do a refresh, so every one second by default if you keep indexing data. This cache is typically useful if eg. you have an analytics dashboard that many clients keep refreshing, which ends up making the same request to elasticsearch all the time.