Caching with time_zone

[shaharmor]

Hi,

I'm trying to improve our caching of ES, and I thought about making a "warm-up" job to pre-cache some common queries from our users, but the problem is that each user is making the same request but using a different time_zone property.

How will that work in ES? If I make a request using time_zone X, will it help with the caching for the exact same request just with time_zone Y?

If not, is there any way I can make it help?

Thanks

[colings86 (Colin Goodheart-Smithe)]

When you say that each user is making the same request but with different time zones do you mean the start and end values are exactly the same but with different timezones (e.g. user A searches from 9pm GMT to 10pm GMT and user B searches from 9pm CET to 10pm CET, so user A is searching from 9pm UTC to 10pm UTC and user B is searching from 10pm UTC to 11pm UTC), or do you mean the different users are actually searching the same exact time in UTC (e.g. user A searches from 9pm GMT to 10pm GMT and user B searches from 10pm CET to 11pm CET, so both users are actually searching from 9pm UTC to 10pm UTC)?

The reason I ask this is that ES will convert all dates to UTC internally and will actually search using the UTC time, so if the users are not searching the same UTC time, the query cache will not help as much. The cache will still help if a single user or two users in the same timezone run the same query. The other case where the query cache will still help here is when on a shard the query either matches all documents (the date range of the query entirely overlaps the range of date values on the shard) or the query does not overlap the date range of the shard at all. In these cases the date range query is 'rewritten' to a MatchAllQuery or a MatchNoneQuery respectively. more details of this can be found in this blog post.

If your users are searching the same UTC time all of the above still applies. However, there is another case that is not covered. In the case where the users are searching for the same UTC time range we should be able to have the query cache help us by caching the results when the query overlaps the range of values on the shard but not entirely. In this case since both users are making the same underlying search on the index you would think that the query cache would be used. Unfortunately this is not the case because in the case where the query doesn't completely overlap the shards values we don't rewrite the query at all, so what ends up being added to the cache is the original query in whatever timezone was submitted. I have opened https://github.com/elastic/elasticsearch/issues/22412 to discuss this issue and proposed a fix for it.

Hope the above helps,

[shaharmor]

hey @colings86,
Thank you for the thorough response and the issue you opened.

Regarding the exact timing question you asked, we are using Kibana so the users will run the same "Last 7 Days"/"Yesterday" etc time ranges (Which we changed to be rounded to the last minute/hour depending on the time range, so I guess that in UTC it will be the same time.. but the timezone will be different for each of them.

This is mostly the case for me, but I guess there are others who might have different use cases.