Elastic search java client unable to handle scores more than 1

Elastic Stack Elasticsearch
1

I am using elastic search java client with elastic search version 2.3 In one of my queries, the search score come out in the order of 4-5. But elastic search java client is converting all the score above 1 to 1 and therefore I am not getting correct results. What can be done in this case to get the correct result? I don't need the scores but I need to sort the search result in the order of score which is not possible now.

2

Any chance you can share some code?

3

When I am doing using curl command, I am getting proper results
{
"took" : 105,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 8212,
"max_score" : 5.198194,
....

But when I use Java client, using the following code. I am getting max score as 1 and the result are not properly sorted.
searchResponse = client.prepareSearch() .setIndices((String) entitySearchParams.get(INDEX_NAME)) .setTypes(entityTypesArray) .setQuery(queryBuilder) .setFrom(0) .setSize(10) .setFetchSource(include, exclude) .get();

4

I am using ngram analyzer and I have around 30,000 documents.

   "analyzer": {
                "twofifteengram": {
                  "type": "custom",
                  "tokenizer": "standard",
                  "filter": [
                    "lowercase",
                    "twofifteengram_filter"
                  ]
                }
              }
            }
          }
5

Please format your code using </> icon. It will make your post more readable.

What do you have in queryBuilder and filter?
What is the exact curl command you are sending?

6 
 QueryBuilder queryBuilder = new TemplateQueryBuilder(templateInstance);

I am using a mustache template

Template templateInstance  = new Template(
                searchQueryParamInstance.getTemplateName(),
                ScriptService.ScriptType.INDEXED,
                MustacheScriptEngineService.NAME,
                null,
                searchQueryParamInstance.getParams()
        );

The curl command I am using is
curl -XGET 'localhost:9200/indexname/typename/_search/template?pretty' -d@temp.json
where temp.json contains the following json:

 {
    "template": {
        "id": "templateName" 
    },
    "params": {
        "param1": "attribute.ngram",
        "value1":"firstword secondword thirdword "
    }
}

Same params and template name is passed in java client as in the curl command

Filter can be removed.

7

The mustache template is
"{ \"query\" : { \"bool\" : { \"should\" : [ {{#param1}} {{#value1}} { \"match\" : { \"{{param1}}\" : \"{{value1}}\" } } {{/value1}} {{/param1}} {{#param2}} {{#value2}} , { \"match\" : { \"{{param2}}\" : \"{{value2}}\" } } {{/value2}} {{/param2}} ...

The same is continued for 10 params and values

8

Can you print the queryBuilder.toString() or string() (don't recall from the top of my head)?

9

Even better. Don't call get() on the prepareSearch but print its content with toString().

10

Query builder

  {
          "template" : {
            "id" : "templateName",
            "lang" : "mustache",
            "params" : {
              "param1" : "attributesInfo.en-US.xyz.ngram",
              "value1" : "Random company name CO., LTD OF"
            }
          }
        }
11

prepareSearch:

{
  "from" : 0,
  "size" : 5,
  "query" : {
    "template" : {
      "id" : "templateName",
      "lang" : "mustache",
      "params" : {
        "param1" : "attributesInfo.en-US.xyz.ngram",
        "value1" : "Random company name CO., LTD OF"
      }
    }
  },
  "_source" : {
    "includes" : [ ],
    "excludes" : [ ]
  }
}
12

Response from curl command

{
"took" : 105,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 8212,
"max_score" : 5.198194,
....

Response that I am getting from java client:

{
  "took" : 11,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 8212,
    "max_score" : 1.0,
    "hits" : [ {

...

As we can see total number of results is same, just the max_score is 1 in later case. Therefore when we are setting the size as 5 , we are not getting expected result

13

Can you run again this?

curl localhost:9200/_search -d '{
  "from" : 0,
  "size" : 5,
  "query" : {
    "template" : {
      "id" : "templateName",
      "lang" : "mustache",
      "params" : {
        "param1" : "attributesInfo.en-US.xyz.ngram",
        "value1" : "Random company name CO., LTD OF"
      }
    }
  },
  "_source" : {
    "includes" : [ ],
    "excludes" : [ ]
  }
}'
14

If I am giving this request, I got the match score as 1 just like the java client

{
  "took" : 11,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 8212,
    "max_score" : 1.0,
    "hits" : [ {
17

I don't understand why the following 2 commands give different results

curl localhost:9200/indexname/typename/_search  -d '{
  "from" : 0,
  "size" : 5,
  "query" : {
    "template" : {
      "id" : "templateName",
      "lang" : "mustache",
      "params" : {
        "param1" : "attributesInfo.en-US.xyz.ngram",
        "value1" : "Random company name CO., LTD OF"
      }
    }
  },
  "_source" : {
    "includes" : [ ],
    "excludes" : [ ]
  }
}'

and

curl -XGET 'localhost:9200/indexname/typename/_search/template?pretty' -d '
     {
        "template": {
            "id": "templateName" 
        },
        "params": {
         "param1" : "attributesInfo.en-US.xyz.ngram",
            "value1" : "Random company name CO., LTD OF"
        }
    }'
18

If the method used by me using templateQuerybuilder won't work then what is the best way to use search templates via java client?

I found a method using setTemplate() method but if I use that method , setSize isn't working

SearchResponse searchResponse  =client.prepareSearch()
                .setIndices("new")
                .setTypes("sku")
                .setTemplate(templateInstance)
                .get();