Metricbeat memory watch - no conditions are met

[manankalra]

Hey, I'm having trouble with setting alerts for my memory usage. I'm using Metricbeat which is stashing only the memory metrics to Elasticsearch.

This is the JSON for the watch that I've configured:

{
"trigger" : {
    "schedule" : {
    "interval": "10s"
    }
},
"input" : {
    "search" : {
    "request": {
        "search_type": "query_then_fetch",
        "indices" : [ "metricbeat-2017.07.06" ],
        "types": [],
        "body": {
            "query": {
                "match_all": {}
            }
        }
      }
    }
},
"condition": {
    "compare": {
        "ctx.payload.hits.hits._source.system.memory.actual.used.pct": {
            "gt": 0.5
        }
    }
},
"actions": {
    "logging": {
        "logging": {
            "level": "info",
                "text": "MEMORY USAGE ALERT!!! GREATER THAN 50%."
        }
    }
} 
}

I'm using match_all{} because all my metricsets are of memory type. It shows that my memory usage is around 0.85 but still no actions are triggered and the condition.met is always false. Need help asap!

[spinscale (Alexander Reelsen)]

please refrain from posting more than once. See the response on the other thread.

[manankalra]

Hey @spinscale, I'm still stuck. Can you please help?

[manankalra]

I did revise my configuration again, same problems.

PUT _xpack/watcher/watch/temp2
{
  "input" : {
    "search" : {
      "request" : {
        "indices" : "metricbeat-2017.07.07",
        "body" : {
          "size" : 0,
          "query" : {
            "bool" : {
              "filter" : {
                "range" : {
                  "@timestamp": {
                    "from" : "now-2m",
                    "to" : "now"
                  }
                }
              }
            }
          },
          "aggs" : {
            "my_average" : {
              "avg": {
                "field": "_source.system.memory.actual.used.pct"
              }
            }
          }
        }
      }
    }
  },
  
  "trigger" : {
    "schedule" : {
      "interval" : "10s"
    }
  }, 
  
  "condition" : {
    "compare" : {
      "ctx.payload.hits.hits.0._source.system.memory.actual.used.pct" : {
        "gte" : 0.50
      }
    }
  },
  
  "actions" : {
    "logging" : {
      "logging" : {
        "text" : "CPU USAGE ALERT!!! GREATER THAN 50%."
      }
    }
  }
}

[spinscale (Alexander Reelsen)]

Hey,

first, please be aware that this is a forum, so there is no obligation to keep to any kind of SLA when providing answers, everything here is voluntary.

Second, please take your time and make sure you understood how watcher works, before writing complex watches.

You have changed you query to not contain any hits, but only contain aggregation results. However your condition still expects hits to be returned, even though you would need to check for thresholds now in the aggregation data.

It may be hard to debug watches, but you should really take the time and read the long blog post to understand how watches can be debugged, so that writing and maintaining watches is less tedious. The blog post is available at https://www.elastic.co/blog/watching-the-watches-writing-debugging-and-testing-watches

--Alex