[elasticserach performance] Elasticsearch-spend-all-time-in-build-scorer

(czve) #1

I use ES 5.5.0,and I have the same problem that the time almost cost at build_scorer, detail as follow:

(Zachary Tong) #2

Is the field "views" mapped as a numeric of some variety?

In ES 5.x, numerics use a new datastructure (BKD tree). This allows better compression, faster numeric operations and lower memory usage... but it is not ideal for "point lookups" like a term query. E.g. it is designed for numeric style operations like ranges, but not single value lookups.

If that field is really an identifier of some type, where you mainly do point lookups, you should re-map it as a keyword. Keyword fields are optimized for single-term lookups like you're doing.

More info: https://www.elastic.co/guide/en/elasticsearch/reference/master/tune-for-search-speed.html#_map_identifiers_as_literal_keyword_literal

To dive a bit more into technicals, the BKD datastructure doesn't support sorted iteration, so it has to collect all matches, sort the array and then return an iterator to that sorted array (paraphrasing). That process happens during the build_scorer step, which is why it is slow and you see it showing up in the profiler output. This process isn't bad when dealing with numeric ranges, but can get expensive when asking for a bunch of individual points.

2.x didn't have BKD trees, hence the difference in performance.

(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.