I'm trying to do some performance test on my cluster and because of that I want to run all my search queries as if they are running for the first time, but it's impossible because of the caching.
I tried setting up the "indices.fielddata.cache.size: 0%" in my elasticsearch.yml file and checked the stats of field data (it shows 0) but once i run the query the stats of feilddata gets changed. So is there any way to disable the caching?
Any suggestions could be helpful.
Unfortunately disabling the field data cache isn't enough to give you results as if running from cold, because other caches (e.g. the page cache) will still be hot.
Yet, I'm curious, why do you want to benchmark the cluster as if running each search for the first time? Normally it's more important to measure its performance in some kind of steady state, because that's what production environments look like.
Thanks for the reply. We converted our timestamps from milliseconds to seconds and i'm running some queries to check if it has any effect on our performance but i couldn't exactly figure it out and it looks like the timings are all over the place sometime second timestamps executes faster and vice-versa. After doing some research I came to conclusion that data is caching for the queries and intern making queries to run faster next time. So i thought of disabling the cache to check it.
Do you think is this the better way to do it because i'm not entirely sure, and i really wanted to know does changing timestamps from milliseconds to second will have any effect on performance. Suggestions could be really helpful.
FYI, I found this article about timestamps and wanted to try that.
Normally when benchmarking it's better to go the other way round: start generating load before you start measuring, so that the caches are realistically warm. This is the state that a production environment is normally in. Daniel gave a good talk recently about how to benchmark effectively here: https://www.elastic.co/elasticon/conf/2018/sf/the-seven-deadly-sins-of-elasticsearch-benchmarking (spoiler: sin #2 is trying to measure performance with cold caches).
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.