How to set a time range in the watcher?

[KeithTt]

Can I set a time range about watcher?

Is there a guide for this setting?

I found a sample here:

https://www.elastic.co/guide/en/x-pack/5.x/watching-meetup-data.html

Here is my configuration:

PUT _xpack/watcher/watch/error_fzf
{
  "metadata" : { 
    "color" : "red"
  },
  "trigger" : { 
    "schedule" : { 
      "interval" : "1m" 
    }
  },
  "input" : {
    "search" : {
      "request" : {
        "indices" : [ 
          "kibana-nginx-access-*",
          "zixun-nginx-access-*"
        ],
        "body" : {
          "size": 0,
          "query" : {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-3m"
                    }
                  },
                  "match": {
                    "message": "404"
                  }
                }               
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },
  "actions": {
    "send_email": {
      "throttle_period": "3m",
      "email": {
        "from": "shengyongp@oupeng.com", 
        "to": [
          "PeiShengyong shengyongp@oupeng.com",
          "DuanWei weiduan@oupeng.com"
        ],  
        "subject": "Watcher Notification from {{ctx.payload.hits.hits.0._source.beat.name}}",
        "body": {
          "text": "Found {{ctx.payload.hits.total}} 404 errors in the logs"
        }
      }
    }
  }
}

BUT, it don't send notification...............is there anything wrong or missed something?

[richcollier]

Hello,

The documentation you seek is found here

Also, we have a public github repo that has a bunch of example watches.

At first blush - your setting of range of @timestamp to be "gte": "now-3m" looks correct. Are you sure, however, that there are actually some 404 errors your logs in the last 3 minutes?

[KeithTt]

@richcollier

The documentation you seek is found here

I am sorry, I don't find the info about time range in the link, which section do you mean?

@richcollier

Are you sure, however, that there are actually some 404 errors your logs in the last 3 minutes?

I am sure there are some 404 errors in the log, and I have never received a notification...

[richcollier]

Sorry for the misunderstanding - I thought originally you were looking for information on the search input for a Watch. If you're looking for information on options for the range query, you can find it here - it is a core feature of the elasticsearch query DSL, not just a feature of a watch.

Taking a harder look at your setup, you should modify your query clause to look like this:

          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-3m"
                    }
                  }
                },
                {
                  "term": {
                    "message": "404"
                  }
                }
              ]
            }
          }

In other words, do a term filter.

Another thing to double check is that your time field in your index truly is @timestamp instead of other possibilities (like just plain timestamp)

Finally, to take any possible setup issues related to the email notification out of the picture, you can add the following logging action in addition to the email:

  "actions": {
    "log": {
      "logging": {
        "level": "info",
        "text": "Watcher Notification from {{ctx.payload.hits.hits.0._source.beat.name}} - Found {{ctx.payload.hits.total}} 404 errors in the logs"
      }
    },
    "send_email": {
      "throttle_period_in_millis": 180000,
      "email": {
        "profile": "standard",
        "from": "shengyongp@oupeng.com",
        "to": [
          "PeiShengyong shengyongp@oupeng.com",
          "DuanWei weiduan@oupeng.com"
        ],
        "subject": "Watcher Notification from {{ctx.payload.hits.hits.0._source.beat.name}}",
        "body": {
          "text": "Found {{ctx.payload.hits.total}} 404 errors in the logs"
        }
      }
    }
  },

So that the text of the watch results simply show up in the elasticsearch.log file.

Hope that helps!

[KeithTt]

hi rich,

I found there are many error in my ES log:

[2017-07-24T11:27:03,007][ERROR][o.e.x.w.i.s.ExecutableSimpleInput] [uy-s-169] failed to execute [search] input for watch [error_fzf], reason [[range] malformed query, expected [END_OBJECT] but found [FIELD_NAME]]

[richcollier]

Hi,

If you haven't changed your query clause to what I recommend, you will get this error because there is a syntax problem in your query clause. You are not properly making an array of clauses in the filter section. You have to look close at the matching {} brackets to see.

Please modify it to the following and you'll be fine:

          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "timestamp": {
                      "gte": "now-3m"
                    }
                  }
                },
                {
                  "match": {
                    "message": "404"
                  }
                }
              ]
            }
          }