Multiple range queries within "should" clause

[emirozer]

Hi!

I wanted to ask here first before opening an issue in github as i am not %100 sure i am doing the correct thing.

I use the hosted elastic cloud from you. es version is 5.1.2

Utilizing the following mapping:

{
  "template": "infra_metrics-*",
  "settings": {
    "index": {
      "refresh_interval": "5s"
    }
  },
  "mappings": {
    "_default_": {
      "dynamic_templates": [
        {
          "strings": {
            "match": "*",
            "match_mapping_type": "string",
            "mapping":   { "type": "string",  "doc_values": true, "index": "not_analyzed" }
          }
        }
      ],
      "_all":            { "enabled": false },
      "_source":         { "enabled": true },
      "properties": {
        "timestamp":    { "type": "date",    "doc_values": true},
        "source_vm":     { "type": "keyword", "doc_values": true },
        "cpu_user_percentage":        { "type": "float",   "doc_values": true },
        "cpu_sys_percentage":         { "type": "float",   "doc_values": true },
        "cpu_wait_percentage":        { "type": "float",   "doc_values": true },
        "mem_percentage":         { "type": "float",   "doc_values": true },
        "ephemeral_disk_percentage":         { "type": "float",   "doc_values": true },
        "persistent_disk_percentage":        { "type": "float",   "doc_values": true },
        "system_disk_percentage":         { "type": "float",   "doc_values": true },
        "swap_percentage":         { "type": "float",   "doc_values": true }
      }
    }
  }
}

I create the following watch

{
    "trigger": {
        "schedule": {
            "interval": "5m"
        }
    },
    "input": {
        "search": {
            "request": {
                "indices": [
                    "infra_metrics-*"
                ],
                "body": {
                    "query": {
                        "bool": {
                            "filter": {
                                "range": {
                                    "timestamp": {
                                        "from": "now-5m",
                                        "to": "now"
                                    }
                                }
                            },
                            "should": [
                                {
                                    "range": {
                                        "cpu_sys_percentage": {
                                            "gte": 80.0
                                        }
                                    }
                                },
                                {
                                    "range": {
                                        "cpu_user_percentage": {
                                            "gte": 80.0
                                        }
                                    }
                                },
                                {
                                    "range": {
                                        "cpu_wait_percentage": {
                                            "gte": 80.0
                                        }
                                    }
                                }
                            ]
                        }
                    }
                }
            }
        }
    },
    "condition": {
        "compare": {
            "ctx.payload.hits.total": {
                "gt": 0
            }
        }
    }

I removed the last action part.

So what happens is that this watch executes and returns all the datapoints in the timestamp filter i applied.
Its like all the should ranges are ignored.

if i swap should with must the watch works as expected.

Am i doing something wrong? basically i want to trigger an email if one of these values go above 80.0

Thx,

Emir

[spinscale (Alexander Reelsen)]

Hey,

the main question is, what do you expect, when you configure the should query? It seems you expect that all conditions have to match, however this is what the must query is for. The shouldquery contributes to score. You may want to check out the documentation for the bool query https://www.elastic.co/guide/en/elasticsearch/reference/5.2/query-dsl-bool-query.html - which states, that if you have a filterclause, all parts of a should query become optional.

--Alex

[emirozer]

Well i didn't know about it being a contributor to the score. as far as the documentation goes, i expect should to behave exactly like SQL OR

So, In the above watch, when i run it, i expect the following:

• Return search results if there is a hit of one or more conditions in the should block

What do i get:

• All entries of that index within the timestamp range i give.

[spinscale (Alexander Reelsen)]

Hey,

there's the catch. It does not behave at all like an SQL OR. The above linked docs state

In a boolean query with no must or filter clauses, one or more should clauses must match a document.

As you specified a filter in the bool query, none of the should clauses have to match. The interesting part for you however is the next note about a bool query in a filter context. If you put a new bool query with the filter and should clauses inside of the filter, then the query will behave as you expect.

--Alex

[emirozer]

thank you @spinscale that clarifies it for me!