Having trouble understand Watcher condition

[victor.nilsson]

Hi,

I've recently begun testing the Watcher functionality and have been testing out a simple alert that will send an email to me whenever a persistent queue of Logstash exceeds over 1GB in Bytes.

However, i'm having trouble getting the condition right. It seems to send me alerts even though the queue never goes over 1GB in size. Here is the alert:

{
  "trigger": {
    "schedule": {
      "interval": "1m"
    }
  },
  "input": {
    "http": {
      "request": {
        "scheme": "http",
        "host": "84.19.155.71",
        "port": 9600,
        "method": "get",
        "path": "/_node/stats/pipelines",
        "params": {},
        "headers": {}
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.pipelines.main.queue.capacity.queue_size_in_bytes": {
        "gt": "1073741824"
      }
    }
  },
  "actions": {
    "send_email": {
      "email": {
        "profile": "standard",
        "to": [
          "supersecretmail@maimail.com"
        ],
        "subject": "Logstash Persistent Queue rising",
        "body": {
          "text": "Logstash Persistent Queue is rising. Max Queue Sise is {{ctx.payload.pipelines.main.queue.capacity.max_queue_size_in_bytes}}. Current Queue Size is {{ctx.payload.pipelines.main.queue.capacity.queue_size_in_bytes}}"
        }
      }
    }
  }
}

Here is an example from the simulation of the alert:

"condition": {
      "type": "compare",
      "status": "success",
      "met": true,
      "compare": {
        "resolved_values": {
          "ctx.payload.pipelines.main.queue.capacity.queue_size_in_bytes": 176440702
        }
      }
    }

Since the resolved value is lower than the one i set, i expected it NOT to generate an alert. How come i still get the alerts?

[victor.nilsson]

I can confirm that the issue has NOT been resolved by removing and adding the alert again.

[Christian_Dahlqvist]

@victor.nilsson

"gt": "1073741824"

Why are you specifying this as a string? Will that not cause a string based comparison rather than a numerical one?

[victor.nilsson]

Nice catch. I had no idea that surrounding the value in quotes would make it a string. That is probably the issue. I've removed the quotes and will check back to post the results.

EDIT: That was indeed the issue, Thanks.