Elastic 2.0 slower query execution speed as 1.3

Elastic Stack Elasticsearch
1

Hi,
I'm having an index of around 140M entries, still running elastic 1.3 in production and using filtered queries to do lookups, most of the queries execute in sub 100ms, between 40 and 100ms.

I now have a new server with almost the same hardware specs and upgraded to 2.0.
I followed the guidelines that were posted here: https://www.elastic.co/blog/better-query-execution-coming-elasticsearch-2-0

The query I'm running is :


And first hit is around 900ms all subsequent ones take around 230ms

I'm a bit dissapointed with this not so "fast" speed. .
Are there any things I can optimize further to this query or things you see I'm doing wrong with it?
I just want to make sure I'm making full use of the Roaring bitmaps and that is everything is wrapped in constantScore.

This is the explain of some of the results:

Thanks,
Jayme

2

Hi Jayme,

I was having a very similar problem. Though it was with queries and not filters.

You could have a look at "query_cache" property in _nodes/stats?pretty=true for both 2.0 as well as for 1.3, if you are seeing any differences there. (https://www.elastic.co/guide/en/elasticsearch/reference/2.0/query-cache.html)

You can try tweaking the cache size, to see if that solves/ reduces your problem.

In my case, the "query_cache" was being used even though the queries were not being executed as a filter. And the miss count was quite high.
More details : Search is slower in ElasticSearch 2.0.0 than 1.7.3 when the document count is high ( > 2.5 million)

Unfortunately, I do not have any more information on this. Hopefully an expert can throw more light on these issues.

Thanks,
Vignesh

3

Hi Jayme,

Indeed, there is one optimization that was lost when moving to the bool query instead of filtered, we are going to add it back through https://issues.apache.org/jira/browse/LUCENE-6889.

In the mean time you should be able to get back similar speed as before by replacing queries that look like this:

{
  "bool": {
    "must": { "match_all": {} },
   "filter": { "some_filter...": {}}
  }
}

to

{
  "constant_score": {
   "filter": { "some_filter...": {}}
  }
}

Both queries will match the same documents and give the same scores. This is something that was performed automatically by the filtered query before but that bool does not do until LUCENE-6889 is in.

1 Like
4

Great, thanks for the help!
That indeed looks already much better in terms of execution speed.

1 Like
5

I got similar issue after I migrated to elasticsearch 2.0.2. I tried to change the bool query to constant_score as suggested above, but I did not get any performance up.

I also tried to change the query cache to cache everthing with bigger cache size/count, which gave me a little bit of speep up but still slower than 1.7.3 vastly against the same queries.

Most of my queries are filter only, no scoring required, is there any suggestion to make the 2.0.2 works as the same speed as 1.7.3?

6

try putting index.queries.cache.everything: true
in your config, it made a big difference for me