Watcher "Failed to execute" error

I am new to Watcher and will readily confess that I'm having difficulty figuring out how to create an alert when we're experiencing timeouts on our transaction processors.

Here's the goal: If we have 5 or more timeouts (which is exec_time greater than or equal to 3.5) in the last 30 seconds, send an email alert. Not worried about throttling yet, just proving out a concept.

Here's what I've tried:

{
  "trigger": {
    "schedule": {
      "interval": "30s"
    }
  },
  "input": {
    "search": {
      "request": {
        "indices": [
          "httplog-*"
        ],
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-30s"
                    }
                  }
                },
                {
                  "match": {
                    "exec_time": "[3.5 TO *]"
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gte": 5
      }
    },
    "actions": {
      "send_email": {
        "email": {
          "to": "alerts@redacted.com",
		  "subject": "Timeout Notification",
		  "body": "{{ctx.payload.hits.total}} Timeouts occurred in the last 30 seconds."
        }
      }
    }
  }
}

It's not working, though. Here's what the logs say:

Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[httplog-2017.11.12, httplog-2017.11.13, httplog-2017.11.10, httplog-2017.11.21, httplog-2017.11.11, httplog-2017.11.20, httplog-2017.11.09, httplog-2017.11.07, httplog-2017.11.18, httplog-2017.11.08, httplog-2017.11.19, httplog-2017.11.16, httplog-2017.11.17, httplog-2017.11.14, httplog-2017.11.15], indicesOptions=IndicesOptions[id=7, ignore_unavailable=true, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_alisases_to_multiple_indices=true, forbid_closed_indices=false], types=[], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=5, batchedReduceSize=512, preFilterShardSize=128, source={
  "size" : 0,
  "query" : {
    "bool" : {
      "filter" : [
        {
          "range" : {
            "@timestamp" : {
              "from" : "now-1m",
              "to" : null,
              "include_lower" : true,
              "include_upper" : true,
              "boost" : 1.0
            }
          }
        },
        {
          "match" : {
            "exec_time" : {
              "query" : "[3.5 TO *]",
              "operator" : "OR",
              "prefix_length" : 0,
              "max_expansions" : 50,
              "fuzzy_transpositions" : true,
              "lenient" : false,
              "zero_terms_query" : "NONE",
              "boost" : 1.0
            }
          }
        }
      ],
      "disable_coord" : false,
      "adjust_pure_negative" : true,
      "boost" : 1.0
    }
  }
}}] lastShard [true]
org.elasticsearch.transport.RemoteTransportException: [EMCTS499][90.21.65.98:8081][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.index.query.QueryShardException: failed to create query: {
  "bool" : {
    "filter" : [
      {
        "range" : {
          "@timestamp" : {
            "from" : "now-1m",
            "to" : null,
            "include_lower" : true,
            "include_upper" : true,
            "boost" : 1.0
          }
        }
      },
      {
        "match" : {
          "exec_time" : {
            "query" : "[3.5 TO *]",
            "operator" : "OR",
            "prefix_length" : 0,
            "max_expansions" : 50,
            "fuzzy_transpositions" : true,
            "lenient" : false,
            "zero_terms_query" : "NONE",
            "boost" : 1.0
          }
        }
      }
    ],
    "disable_coord" : false,
    "adjust_pure_negative" : true,
    "boost" : 1.0
  }
}
Caused by: java.lang.NumberFormatException: For input string: "[3.5 TO *]"

I'm having a lot of difficulty finding good examples of how to format the query to return the results I seek. Any advice?

I figured it out. Not sure why I didn't think of this in the first place:

Before:

"match": {
    "exec_time": "[3.5 TO *]"
}

After:

"range": {
  "exec_time": {
    "gte": "3.5"
  }
}
1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.