How to check only the documents with in the interval?

Elastic Stack Elasticsearch
1

I want to check only the documents which was indexed within the interval of watcher execution but result.input.payload.hits.total is returning all the matched documents in the index matching the query.

My watcher definition is below.

PUT _xpack/watcher/watch/cpu_monitoring
{
  "metadata" : { 
    "color" : "red"
  },
  "trigger" : { 
    "schedule" : {
      "interval" : "1m"
    }
  },
  "input" : { 
    "search" : {
      "request" : {
        "indices" : "metricbeat-*",
        "body" : {
          "query" : {
            "bool" : {
              "must" : [
                {"match" : { "metricset.name" : "cpu" }},
                {"range" : { "system.cpu.idle.pct" : {"lte" : "0.97" } } }
              ]
            }
          }
        }
      }
    }
  },
  "condition" : { 
    "compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
  },
  "actions" : { 
    "email_administrator" : {
      "email" : {
        "to" : "root@localhost.localdomain",
        "subject" : "Encountered {{ctx.payload.hits.total}} errors",
        "body" : "Too many error in the system, see attached data",
        "attachments" : {
          "attached_data" : {
            "data" : {
              "format" : "json"
            }
          }
        },
        "priority" : "high"
      }
    }
  }
}

I appreciate if someone can help me how to fix my query.

2

Hey,

you need to add another filter to your must query that filters for time. Also you should use date math for correct index filtering.

See https://www.elastic.co/guide/en/elasticsearch/reference/2.4/common-options.html#date-math
and https://www.elastic.co/guide/en/watcher/2.4/transform.html#transform-search-template for an example how to filter by date.

--Alex

3

Thanks @spinscale

I just figured out it too.

  "input" : { 
    "search" : {
      "request" : {
        "indices" : "metricbeat-*",
        "body" : {
          "query" : {
            "bool" : {
              "must" : [
                {"match" : { "metricset.name" : "cpu" }},
                {"range" : { "system.cpu.idle.pct" : {"lte" : "0.97" } } },
                {"range" : { "@timestamp" : {"gte" : "now-1m", "lte" : "now" } } }
              ]
            }
          }
        }
      }
    }
  },