Java - build query due to the many fields

Elastic Stack Elasticsearch
1

I use Java and Java High Level REST Client.
I need to build query that will find documents. This query should contain many fields for searching.
For example, I have such json:

{
"timeStamp": "Fri, 29 Dec 2017 15:32:22 +0000",
"value": 314,
"operation": "http://software-testing.ru/library/testing/testing-tools/2638-postman",
"type": "duration",
"system":"front-admin"
}

and I need find all documents due to the all fields.
I understand how constract query due to the one field (parametr) but don't know how I can use a lot of fields for query:

@PostMapping("/findMetricsByValues")
@Transactional
public ResponseEntity findMetricsByValues(@RequestBody ElasticSearchMetrics metrics){

    SearchRequest searchRequest = new SearchRequest();

    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    sourceBuilder.query(QueryBuilders.termQuery("value",  metrics.getValue()));
    searchRequest.source(sourceBuilder);
    SearchResponse searchResponse = null;
    try {
        searchResponse = client.search(searchRequest);
    } catch (IOException e) {
        e.printStackTrace();
    }
    return new ResponseEntity<>(new GenericResponse(searchResponse, CODE_200), HttpStatus.OK);
}

Could somebody help with it or give a hint?
Thanks a lot.

2

To format your code, please use the code button, not the citation button.

Basically you want to run a Bool Query, right?

Have a look at: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/6.1/java-rest-high-query-builders.html#_compound_queries

I just wrote an example here: Elastic search depreciated classes

3

I think, Bool Query - it's exactly what I've been searching for.
I'll try.
Thanks a lot.

4

And I have one more question.
What if some of this fields will have null value. For example, in my case "timeStamp" and "system" will be empty, but I want get all document due to the fields that won't have null value be in json:

{
	"timeStamp": NULL,
	"value": 314,
	"operation": "http://software-testing.ru/library/testing/testing-tools/2638-postman",
	"type": "accuracy",
	"system": NULL
}

@PostMapping("/findMetricsByValues")
@Transactional
public ResponseEntity findMetricsByValues(@RequestBody ElasticSearchMetrics metrics){

    SearchRequest searchRequest = new SearchRequest();

    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    sourceBuilder.query(QueryBuilders.boolQuery()
            .filter(QueryBuilders.termQuery("value", metrics.getValue()))
            .filter(QueryBuilders.termQuery("type", metrics.getType()))
            .filter(QueryBuilders.termQuery("operation", metrics.getOperation()))
            .filter(QueryBuilders.termQuery("operation", metrics.getOperation()))
            .filter(QueryBuilders.termQuery("timeStamp", metrics.getTimeStamp()))
    );
    searchRequest.source(sourceBuilder);
    SearchResponse searchResponse = null;
    try {
        searchResponse = client.search(searchRequest);
    } catch (IOException e) {
        e.printStackTrace();
    }
    return new ResponseEntity<>(new GenericResponse(searchResponse, CODE_200), HttpStatus.OK);
}

Than I'll have empty response

What should I do to allow searching by the rest fields in such case ?

5

So you want to search for documents which have:

  • field: foo

or

  • field: NULL

Is that right?

You can use a bool query, with a should clause which contains 2 clauses:

  • A term query on field field with value foo
  • An exist query (wrapped inside a bool -> must_not clause)

I suggest that you first try to write such a query in Kibana console, then translate it to some Java code.

6

I just want to get result(records) of existing metrics when one or a few of the fields will be NULL and other fields will exist in ES.
But when one of the fields will be incorrect (there no such records in ES) than I won't get any result (records).

7 
SearchRequest searchRequest = new SearchRequest();

        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(QueryBuilders.boolQuery()
                .should(QueryBuilders.termQuery("value", metrics.getValue()))
                .should(QueryBuilders.termQuery("type", metrics.getType()))
                .should(QueryBuilders.termQuery("system", metrics.getSystem()))

        );
        searchRequest.source(sourceBuilder);

In such way when one of the fields will be NULL, ES will find records according to other fields even if one of the fields will be inccorrect (there not such record in ES). But I need when one of the fields won't exsist in ES, but not NULL, then I don't get any result.

8

Could you provide a full recreation script as described in About the Elasticsearch category. It will help to better understand what you want to do exactly. Please, try to keep the example as simple as possible (like one or two fields should be enough).

9 
POST _search
{
  "query": {
    "bool" : {
     "filter": [
        { "term" : { "value" : "test" } },
        { "term" : { "type" : "testType" } }
      ],
 }
}

but I don't know how to add NULL condition. So, for example value can be "test" or NULL, than I will get records.

10

I can't reproduce with that. Please read carefully About the Elasticsearch category. There is an example of what is a recreation script.
Here I can't just copy and paste and run.

BTW there is a typo here:

      ],
11 
 POST _search
   {
  "query":{
    "bool":{
     "filter":[
        { "term" : { "value" : "test"} },
        { "term" : { "type" : "testType" } }
      ]
 }
}
}
12

Can I add in this query condition: if meaning of "value" will be NULL then just don't execute this term ? In other case just execute the term with other terms.

13

Ok. Apparently, I'm not clear enough about what is a reproduction script. So let me show you what happens when I run your script:

POST _search
   {
  "query":{
    "bool":{
     "filter":[
        { "term" : { "value" : "test"} },
        { "term" : { "type" : "testType" } }
      ]
 }
}
}

{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 9,
    "successful": 9,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 0,
    "max_score": null,
    "hits": []
  }
}

I have nothing. Well. I have no data to search for.
From that, I can't help. Is that more clear? I'm going to copy and paste here the full example that is available at https://discuss.elastic.co/t/about-the-elasticsearch-category/21:

DELETE index
PUT index/type/1
{
  "foo": "bar"
}
GET index/type/_search
{
  "query": {
    "match": {
      "foo": "bar"
    }
  }
}

When I run this ^^^, I'm getting:

{
  "took": 14,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 0.2876821,
    "hits": [
      {
        "_index": "index",
        "_type": "type",
        "_id": "1",
        "_score": 0.2876821,
        "_source": {
          "foo": "bar"
        }
      }
    ]
  }
}
14

Ok, let's forget about privious example.
I have another way of executing my task.
Here example:

sourceBuilder.query(QueryBuilders.boolQuery()
                .filter(QueryBuilders.termQuery("value", metrics.getValue()))

I need: if metrics.getValue() will respond with NULL than just don't execute this term.
Can I do it in some way?

15 
BoolQueryBuilder bqb = QueryBuilders.boolQuery();
if (metrics.getValue() != null) {
  bqb.filter(QueryBuilders.termQuery("value", metrics.getValue());
}
sourceBuilder.query(bqb);
16

Thanks, that helps.

17

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.