Sorting tables

Elastic Stack Elasticsearch
1

Hi,

I have a problem that I haven't been able to solve that someone might have a better solution to.

I have an index with a couple of 100000 documents. Depending on attributes like country, language etc. I want the sorting to be different. I have precalculated these sorting tables in Hadoop and there are currently about 100000 different combinations of values, so I can't store them as fields in the documents.

I have tried these solutions already, they all work, but are to slow basically:

  1. Fetching the sorting tables from Hadoop, setting it as a hashed array in the parameters to MVEL, and using scripting to sort them.
  2. Doing the same, but sending it as an normal array and using indexof instead to sort.
  3. Saving all the different combinations comma-separated in a field and using MVEL to split and sort.

I also tried to separate all the attributes and giving them a score value and calculate the sorting order that way, but the result is not exact enough for our needs.

Does anyone see any another solution to this?

Thanks!

2

Hi Marcus,

I might not understand your problem well enough, but if you try to
sort the documents directly form Elasticsearch, it takes too long?

In my experience, it can sort query results really quickly.

If I get your problem wrong, can you explain more about what you're
after? Are you trying to do a query that will return all indexed
documents, sorted by a specific field?

Best regards,
Radu

On 19 mar., 17:51, marcus marcus.1.johans...@nokia.com wrote:

Hi,

I have a problem that I haven't been able to solve that someone might have a
better solution to.

I have an index with a couple of 100000 documents. Depending on attributes
like country, language etc. I want the sorting to be different. I have
precalculated these sorting tables in Hadoop and there are currently about
100000 different combinations of values, so I can't store them as fields in
the documents.

I have tried these solutions already, they all work, but are to slow
basically:

  1. Fetching the sorting tables from Hadoop, setting it as a hashed array in
    the parameters to MVEL, and using scripting to sort them.
  2. Doing the same, but sending it as an normal array and using indexof
    instead to sort.
  3. Saving all the different combinations comma-separated in a field and
    using MVEL to split and sort.

I also tried to separate all the attributes and giving them a score value
and calculate the sorting order that way, but the result is not exact enough
for our needs.

Does anyone see any another solution to this?

Thanks!

--
View this message in context:http://elasticsearch-users.115913.n3.nabble.com/Sorting-tables-tp3839...
Sent from the Elasticsearch Users mailing list archive at Nabble.com.