CustomScore and biggest value from all docs

Elastic Stack Elasticsearch
1

Hi now i have customscore "_score + ((parseInt(doc.ad_when.value) -
oldestAd) / paramNow) * 2". Is possible to use in custom score biggest
value from all docs.

Example:
in all docs i have popularity field and i want use biggest value from all
docs to customscore. is it possible?

Thanks for answers. :slight_smile:

--

2

Hi,

Just so I understand, you want to include the largest value from your
popularity field, in the custom score of the documents matching your query?
Do you want the largest value from all documents with a popularity field,
or just those which match your query?

On Wednesday, September 26, 2012 8:26:37 PM UTC+12, ESFan wrote:

Hi now i have customscore "_score + ((parseInt(doc.ad_when.value) -
oldestAd) / paramNow) * 2". Is possible to use in custom score biggest
value from all docs.

Example:
in all docs i have popularity field and i want use biggest value from all
docs to customscore. is it possible?

Thanks for answers. :slight_smile:

--

3

Yes. i want from all docs get biggest value.
Example:
I have docs:
A | 12
B | 41
C | 88
D | 21
E | 14
F | 15

Formula will be like (it will be use 88, because it's biggest value):
_score + ((parseInt(doc.ad_when.value) - 88) / doc.id.value)) * 2.

It will be easy if i send 2 requests to server. But i don't want to make
that. :slight_smile:

--

4

Unfortunately computing this won't be easy. I took a look at how the
maximum value is computed in the Statistical facets
(Elasticsearch Platform — Find real-time answers at scale | Elastic)
and it uses a Collector to compute the maximum value (which basically
involves iterating over all the values for each document).

Does your data change often? Could you compute the value and cache it
somehow?

On Thursday, September 27, 2012 5:30:44 PM UTC+12, ESFan wrote:

Yes. i want from all docs get biggest value.
Example:
I have docs:
A | 12
B | 41
C | 88
D | 21
E | 14
F | 15

Formula will be like (it will be use 88, because it's biggest value):
_score + ((parseInt(doc.ad_when.value) - 88) / doc.id.value)) * 2.

It will be easy if i send 2 requests to server. But i don't want to make
that. :slight_smile:

--

5

You are recommend make two different queries? Nope data not change often
(in day we adding about -3 000 new docs). Yes i can cache it.

On Friday, September 28, 2012 8:14:02 AM UTC+3, Chris Male wrote:

Unfortunately computing this won't be easy. I took a look at how the
maximum value is computed in the Statistical facets (
Elasticsearch Platform — Find real-time answers at scale | Elastic)
and it uses a Collector to compute the maximum value (which basically
involves iterating over all the values for each document).

Does your data change often? Could you compute the value and cache it
somehow?

On Thursday, September 27, 2012 5:30:44 PM UTC+12, ESFan wrote:

Yes. i want from all docs get biggest value.
Example:
I have docs:
A | 12
B | 41
C | 88
D | 21
E | 14
F | 15

Formula will be like (it will be use 88, because it's biggest value):
_score + ((parseInt(doc.ad_when.value) - 88) / doc.id.value)) * 2.

It will be easy if i send 2 requests to server. But i don't want to make
that. :slight_smile:

--

6

I'm a little confused about why you need to cache it. If you use an
OpenBitSet based field cache, it remains there for the duration of your
index currency, doesn't it? How do you get the largest value? You sub-class
the cached field and add the numeric value field you're after. (If its
represented as a string forget this idea). When the indexreader is
opened/re-opened you re-init your the subCacheClass, and compute the
largest value 1x. That value remains current until the docs that have the
value are removed, or the cache needs a full reinit.

We did this at a large video site successfully (however my description may
be unsuccessful). Inside a custom scorer (with no sort fields needed) it
was straightforward to pull the needed value from the bitset based object
(since the bits represent the docids)

now, if that's confusing an older idea: generate a
GlobalStatisticsDocument, that has values in the fields that can never be
retrieved. And store the latest big value in a field there. Always re-index
that document into an "invisible" docid, that can only be instantiated (and
should need to be 1x per index open) for your scoring use (and other
nefarious non-user purposes.)

On Wednesday, September 26, 2012 4:26:37 AM UTC-4, ESFan wrote:

Hi now i have customscore "_score + ((parseInt(doc.ad_when.value) -
oldestAd) / paramNow) * 2". Is possible to use in custom score biggest
value from all docs.

Example:
in all docs i have popularity field and i want use biggest value from all
docs to customscore. is it possible?

Thanks for answers. :slight_smile:

--