Hi,
I need to speed-up query_string to always run as fast as when it is warmed-up.
I have an index with couple of million documents (about 10GB big) and two string fields that are used in query_string (title and perex of article). I have 1 instance of Elastic with 24GB of RAM (64GB total on the server). Search is fairly simple as I need to obtain all document ids that match given query.
When I warm-up the queries, search takes about 300ms (depending on number of matched documents). But after couple of hours (sometimes minutes), it takes 10-20x more to run same query.
I need to be able to run arbitrary query_string (as a filter) in hundreds of milliseconds.
I tried to add ['fielddata' => ['loading' => 'eager_global_ordinals']] into mapping of those string properties. I'm also trying to run aggregation on those fields to keep them in memory. I tried not indexing new documents to keep index segments immutable. Nothing is working.
I run fairly old version of ES (1.3), will upgrade to 5.0 mean massive improvement?
Is there a way to keep inverted index of string field always in memory and run as fast as possible? There should be enough memory (jvm heap fluctuates between 20% and 75%).
Thank you for any advice.