Sorting cache?

Elastic Stack Elasticsearch
1

Hi,

A question that is related to my question here:
http://elasticsearch-users.115913.n3.nabble.com/Sorting-tables-tt3839391.html

I'm doing a really slow MVEL sort script and need to solve that in some way. I know that there is a filter cache where you can cache the filter. Does there exist any sorting cache that could save a specific id sort order into memory?

Thanks.

2

Marcus,

I've looked at your question a few times and have not commented because I
don't really understand your data or your requirements. Could you try
restating what it is you're trying to accomplish? Perhaps some samples?

--Mike

On Tue, Mar 20, 2012 at 11:30 AM, marcus marcus.1.johansson@nokia.comwrote:

Hi,

A question that is related to my question here:

http://elasticsearch-users.115913.n3.nabble.com/Sorting-tables-tt3839391.html

I'm doing a really slow MVEL sort script and need to solve that in some
way.
I know that there is a filter cache where you can cache the filter. Does
there exist any sorting cache that could save a specific id sort order into
memory?

Thanks.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Sorting-cache-tp3842734p3842734.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

3

So what I want to do is basically use ES in a way it probably wasn't meant to be used. I would like to do the sorting of the filtering/querying against a joined table instead of the natural search score sorting.

So a simple example would be that I have documents with just an id and a text field. I first do a query against the text field to find the matches of the search query, but instead of letting the _score be used for sorting I want to sort those matches according to a fixed preset sorting on the id's.

I could of course save this preset sorting as a not analyzed field in ES, my problem is that I want to do this sorting different depending on some attributes, so I will have more preset sorting lists then the amount of fields you should have in an index (100000+).

I have solved this problem using memcache to get the preset sorting and then use MVEL to implement that sorting. My problem is that it is to slow, so I was wondering if there is some way to store those different sort lists in memory via some cache.

4

On Tue, Mar 20, 2012 at 2:26 PM, marcus marcus.1.johansson@nokia.comwrote:

So what I want to do is basically use ES in a way it probably wasn't meant
to
be used. I would like to do the sorting of the filtering/querying against a
joined table instead of the natural search score sorting.

So a simple example would be that I have documents with just an id and a
text field. I first do a query against the text field to find the matches
of
the search query, but instead of letting the _score be used for sorting I
want to sort those matches according to a fixed preset sorting on the id's.

I could of course save this preset sorting as a not analyzed field in ES,
my
problem is that I want to do this sorting different depending on some
attributes, so I will have more preset sorting lists then the amount of
fields you should have in an index (100000+).

Have you tried going this route? How many potential sort fields are you
talking about? How many do you sort by in an average or complex query? I've
used ES to do 2 and 3 field on reasonably sized collections and it's been
fine after the caches have warned up.

I have solved this problem using memcache to get the preset sorting and
then
use MVEL to implement that sorting. My problem is that it is to slow, so I
was wondering if there is some way to store those different sort lists in
memory via some cache.

I can imagine a few approaches but would like to know if the vanilla one
has been tried. ES has been quite speedy for me thus far.

5

Have you tried going this route? How many potential sort fields are you talking about? How many do you sort by in an average or complex query? I've used ES to do 2 and 3 field on reasonably sized collections and it's been fine after the caches have warned up.

We have around 100000 different combinations.

I can imagine a few approaches but would like to know if the vanilla one has been tried. ES has been quite speedy for me thus far.

We have tried the vanilla solution to just create one not analyzed field for each combination, but it was to slow (1 second+).