Painless If doc['field'].value in List

[MPurcell (Matthew J Purcell)]

Hey quick question. I have a script/painless section in a bool-must section of my code. Basically I want to check if the classification number, which could be = 17, 18, 19, 20, or 21 is either 18, 19, or 20. For some reason, I can't get the or statement in my code to work properly. It only matches, at the most, against the right most term. Here is the part of my query in question.

The second bool statement works as expected, the first does not. Thanks for any/all help.

[rjernst (Ryan Ernst)]

Please do not put code in images. Use </> in the message editor toolbar.

Doing a bitwise OR of the values will not work, as the bits are not disjoint. You should do a linear scan through your values (since there is a very small number of them). For this, List has the contains method. Try something like this:

[18L, 19L, 20L].contains(doc['Columns.Lines.CharacterClassification'].value)

Note that the L suffix is necessary because numeric fields (non floating point) are exposed as long values.

[MPurcell (Matthew J Purcell)]

@rjernst Will do. And that worked perfectly, with no negative impact on the query speed. Thanks for your help!

[MPurcell (Matthew J Purcell)]

@rjernst What would be another way to do this - that's as fast or faster - if there are more values to search for and more possible values the field could be? For example searching for all the numbers listed above, [17, 18, 19, 20, 21, 30] in a field that has values 1-100?

[rjernst (Ryan Ernst)]

The faster way would be to index the values, and use a conjunction. I assume the values are already indexed (that would be the default for a numeric field). So instead of doing a script query, add a filter clause to your bool query. Inside there, add a bool query with a should clause containing match queries for each of your values.

[MPurcell (Matthew J Purcell)]

@rjernst I believe this is what you meant by the bool should statement. Is there any particular reason you specified I should nest this 'bool-should' statement in a bool-filter rather then a bool-must? Is the difference purely semantics or is there a speed difference in these two sections?

{
            "bool": {
              "should": [
                {"match": {
                  "Columns.Lines.CharacterClassification": {
                    "query": 18
                  }
                }},
                {"match": {
                  "Columns.Lines.CharacterClassification": {
                    "query": 19
                  }
                }},
                {"match": {
                  "Columns.Lines.CharacterClassification": {
                    "query": 23
                  }
                }},
                {"match": {
                  "Columns.Lines.CharacterClassification": {
                    "query": 27
                  }
                }},
                {"match": {
                  "Columns.Lines.CharacterClassification": {
                    "query": 31
                  }
                }}
                ],
                "minimum_should_match": 1
                }}

[rjernst (Ryan Ernst)]

A filter will act like a must, but without scoring. At least currently, Lucene/Elasticsearch isn't smart enough to automatically move this clause to a filter. Instead, it would return a score of 1.

Also, you can more succinctly specify your match queries, no need for the object based representation (since you do not use additional options):

"match": {
  "Columns.Lines.CharacterClassification": 27
}