Java client, random query generates data too large exception

Elastic Stack Elasticsearch
1

I have the following query which works well with a REST client and returns 3 random results, which is great.

{
  "size":3,
  "query": {
    "function_score": {
      "functions": [
        {
          "filter": { "terms": { "text": ["flu", "influenza"] }},
          "weight": 1
        },
        {
          "random_score": { 
            "seed":  "1331" 
          }
        }
      ]
    }
  }
}

Now I try to do the same in Java:

FunctionScoreQueryBuilder functionScoreQueryBuilder = new FunctionScoreQueryBuilder();
functionScoreQueryBuilder.add(new TermsQueryBuilder(samplingField, terms), new WeightBuilder().setWeight(1));
functionScoreQueryBuilder.add(new RandomScoreFunctionBuilder());

SearchRequestBuilder builder = client.prepareSearch(collection).setTypes(collection).setSize(3);
builder.setQuery(functionScoreQueryBuilder);
builder.addField("text");

And this query fails with a Data too large exception. I had a look into the code and it seems that the Java client is trying to load all the data into memory and then try to apply the size restriction. Is there any work around that?

2

Just noticed that I was executing the queries on two different indexes, one smaller than the other. It is in fact a server side exception, so I am looking at other approaches to randomize the search.

3

In case you look for a solution for picking a random sample of your data, the random sorting solution works. Here is my Java code.

builder.addSort(new ScriptSortBuilder(new Script("Math.random()"), "number").order(SortOrder.ASC));

You can use this with any query. Please be sure that you enable groovy scripting for inline/search.