Query Builder API

Elastic Stack Elasticsearch
1

Hi all I have a question.
I have the query
q=field1:"$value" field2:"$value"^10 field3:"$value" field4:"$value"^0.5 field5:"$value"^0.5

I passed this query via uri search
http://es.endpoint:9200/indexname/_search?q=...

Now I'm using the java transport client in my java code
Which query builder should I use to perform the same query?

2

If you want to duplicate exactly the same request, you can use a QueryStringQuery

But I'd use a BoolQuery with should clauses and matchQuery inside.

3

I tried the QueryStringQuery and BoolQuery and I've got really strange thing with the performance.
When I pass the URI query ES is able to serve 1200 requests per second with disabled http cache and the CPU level on 45-50%
When I pass BoolQuery the CPU level is under 100% on when the load reaches 100 requests per second only, however it keeps the response time good.
QueryStringQuery is even worse. 3-4 requests per second 100% utilization and the huge response time.
What can impact the performance so much when the transport client is used?

4

Many factors.

But impossible to know without knowing what you are doing with which machines/settings...

5

ElasticSearch runs on amazon c4.large instance (RAM 3.75Gb, CPU(s) 2) inside the Docker container
OS Ubuntu 14.04

Index size is ~1GB
Documents count ~3 million

ES settings:
"refresh_interval" : "10s",
"number_of_shards" : "1",
"number_of_replicas" : "0"
other settings are default

JVM settings:
-Xms2500m -Xmx2500m -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true

mlock_all is on

I understand that there can be a lot of reasons of high CPU consuming, however the main question is why uri search works fine with the same settings?
I tested the DSL queries by http before and I had horrible results, hot_threads output said that query parsing consumed a lot of resources, thus I started to use uri search for initial tests.
I assume that ES transport serializes/deserializes query objects to something different from json strings, am I right?

6

What is the query you wrote exactly?

7 
QueryBuilders.boolQuery()
    .must(QueryBuilders.multiMatchQuery(value)
            .field("name")
            .field("description", 0.5f)
            .field("type",0.5f)
            .field("exactname", 3)
            .field("text")
            .type(MultiMatchQueryBuilder.Type.MOST_FIELDS)
    )

Also I tried this query

QueryBuilders.queryStringQuery(value)
            .field("name")
            .field("description", 0.5f)
            .field("type", 0.5f)
            .field("exactname", 3)
            .field("text")       
            .escape(true);

I have some custom filters and analyzers:

"filter" : {
    "name_autocomplete_filter": {
       "type": "edge_ngram",
       "min_gram": 1,
       "max_gram":30
     } 
},

"analyzer" : {
     "name_autocomplete" : {
     "type" : "custom",
     "tokenizer" : "standard",
     "filter" : ["lowercase", "name_autocomplete_filter", "asciifolding"]
},
}

The mapping is:

"name" : {
    "type": "string",
    "search_analyzer": "search_standard",
    "analyzer": "name_autocomplete"
 },
"exactname" : {
    "type": "string",
    "analyzer" : "keyword"
},
"type": {
  "type": "string",
  "analyzer": "english"
},
"description": {
  "type": "string",
  "analyzer": "english"
},
"text": {
  "type" : "string",
  "analyzer" : "english",
  "position_increment_gap": 15
}
8

Can you try

QueryBuilders.queryStringQuery("field1:\"$value\" field2:\"$value\"^10 field3:\"$value\" field4:\"$value\"^0.5 field5:\"$value\"^0.5");

I can't imagine that you see any difference between the transport client and the REST layer.

9

And BTW please format your code using </> icon. It will make your post more readable.