Shard query cache - TTL invalidation?

Elastic Stack Elasticsearch
1

Hi,

The way I understand the Shard Query Cache
http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/index-modules-shard-query-cache.html,
it invalidates *ALL *cached results in a shard when the shard refreshes,
given that *ANY *data in the shard has changed.

My data changes all the time, but I'd like to keep some of my query results
cached for a while - say 1 minute.

My refresh interval (5 sec) cannot be changed because I need near-realtime
searches for other query types.

I can read that there is a "indices.cache.query.expire" config setting
where the general TTL of all Shard Query Caches can be set, but the docs
specify that this shouldn't really be used because all cached results are
invalidated anyway upon refresh.

Two questions:

  1. Is my understanding of the Shard Query Cache invalidation mechanism
    correct?
  2. Is there a way to specify a TTL on the query-level, that makes the
    cached result survive the refresh interval?

Thanks,
Lasse

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c6dd35c7-e1f8-4bd1-9136-6cfaacfabf1e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hi,

On Mon, Nov 24, 2014 at 11:51 AM, Lasse Schou lasseschou@gmail.com wrote:

The way I understand the Shard Query Cache
http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/index-modules-shard-query-cache.html,
it invalidates *ALL *cached results in a shard when the shard refreshes,
given that *ANY *data in the shard has changed.

This is correct.

My data changes all the time, but I'd like to keep some of my query
results cached for a while - say 1 minute.

My refresh interval (5 sec) cannot be changed because I need near-realtime
searches for other query types.

I can read that there is a "indices.cache.query.expire" config setting
where the general TTL of all Shard Query Caches can be set, but the docs
specify that this shouldn't really be used because all cached results are
invalidated anyway upon refresh.

The thing is that we wanted this cache to give accurate results: no matter
if it is enabled or not you are going to get the same results. (just like
the filter caches and fielddata caches).

In your case, I think the best option would be to put a cache in front of
elasticsearch?

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j4u3LMmm3EcLiPH3aacx4QJM70Vfbn2kzKWhVZOxDY52Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Got it, thanks.

Lasse

Den tirsdag den 25. november 2014 skrev Adrien Grand <
adrien.grand@elasticsearch.com>:

Hi,

On Mon, Nov 24, 2014 at 11:51 AM, Lasse Schou <lasseschou@gmail.com
<javascript:_e(%7B%7D,'cvml','lasseschou@gmail.com');>> wrote:

The way I understand the Shard Query Cache
http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.4/index-modules-shard-query-cache.html,
it invalidates *ALL *cached results in a shard when the shard refreshes,
given that *ANY *data in the shard has changed.

This is correct.

My data changes all the time, but I'd like to keep some of my query
results cached for a while - say 1 minute.

My refresh interval (5 sec) cannot be changed because I need
near-realtime searches for other query types.

I can read that there is a "indices.cache.query.expire" config setting
where the general TTL of all Shard Query Caches can be set, but the docs
specify that this shouldn't really be used because all cached results are
invalidated anyway upon refresh.

The thing is that we wanted this cache to give accurate results: no matter
if it is enabled or not you are going to get the same results. (just like
the filter caches and fielddata caches).

In your case, I think the best option would be to put a cache in front of
elasticsearch?

--
Adrien Grand

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/mJGQ6TWwIXc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com
<javascript:_e(%7B%7D,'cvml','elasticsearch%2Bunsubscribe@googlegroups.com');>
.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j4u3LMmm3EcLiPH3aacx4QJM70Vfbn2kzKWhVZOxDY52Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j4u3LMmm3EcLiPH3aacx4QJM70Vfbn2kzKWhVZOxDY52Q%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADERWXpyGCfxxB6pbR%3D4Q-J-EKMXAN3DgYaOs8p5FX9W2LfSnw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.