Watcher: http input specify current date in path

Elastic Stack Elasticsearch
1

I'm trying to create a watcher that monitors the size of a specific index. I have daily indices with this naming convention: "indexname-YYYY.MM.dd", example: "test-index-2022.01.03".

I want to monitor the size of that specific index every hour, if it grows more than 50 gigabytes, Elasticsearch must notify. To be honest, I am not 100% sure if this can be done using ML or any other technology, I am trying to do it using Watchers.

The problem is that I can't specify an static date in the watcher, I need to do it dynamically to match the current date of execution.

So I have something like this:

{
    "trigger": {
      "schedule": {
        "interval": "60m"
      }
    },
    "input": {
      "http": {
        "request": {
          "host" : "hostname",
          "port" : 9200,
          "path" : "/test-index-{{ctx.trigger.triggered_time}}/_stats/store",
          "auth" : {
              "basic" : {
                  "username" : "user",
                  "password" : "password"
              }
          }
        }
      }
    },
    "condition": {
      "compare": {
        "ctx.payload._all.total.store.size_in_bytes": {
          "gte": "53687091200"
        }
      }
    },
    "actions": {
      "my-logging-action": {
        "logging": {
          "text": "Alert message here..."
        }
      }
    }
  }

The problem is in this specific section:
"path" : "/test-index-{{ctx.trigger.triggered_time}}/_stats/store",

It gives me the time with milliseconds and other stuff, is there any way to get only the date. I don't need what is in bold below, and the date should be formatted separated by dot, not dash:

test-index-2022-01-03T23:49:00.275285Z

I think some kind of script would help, but I couldn't find any way to make it work on the input-http section

Thanks!

2

Relevant: Watcher for index size

3

Ok I found a way to do this, summarizing, this is what I needed:

  1. Get only the date (current day - execution of watcher)
  2. Separate the date by dot, not by dash (default)
  3. Months and days must be 2 digit long (example: 2022.01.05 | NOT 2022.1.5)
  4. Fire an event when the index gets bigger than 50 GB.

Here is how I did it:

{
  "trigger": {
    "schedule": {
      "interval": "60m"
    }
  },
  "input": {
    "chain": {
      "inputs": [
        {
          "year": {
            "transform": {
              "script": {
                "source": " return [ '_value' : ctx.trigger.triggered_time.getYear() ] ",
                "lang": "painless"
              }
            }
          }
        },
        {
          "month": {
            "transform": {
              "script": {
                "source": " def _value = ctx.trigger.triggered_time.getMonthValue().toString(); if (_value.length() == 1) { _value = \"0\" + ctx.trigger.triggered_time.getMonthValue()} return _value",
                "lang": "painless"
              }
            }
          }
        },
        {
          "day": {
            "transform": {
              "script": {
                "source": " def _value = ctx.trigger.triggered_time.getDayOfMonth().toString(); if (_value.length() == 1) { _value = \"0\" + ctx.trigger.triggered_time.getDayOfMonth()} return _value",
                "lang": "painless"
              }
            }
          }
        },
        {
          "fulldate": {
            "transform": {
              "script": {
                "source": " return [ '_value' : ctx.payload.year._value + '.' + ctx.payload.month._value + '.' + ctx.payload.day._value ]",
                "lang": "painless"
              }
            }
          }
        },
        {
          "http": {
            "http": {
              "request": {
                "scheme": "http",
                "host": "HOST.NAME",
                "port": 9200,
                "method": "get",
                "path": "/TEST-INDEX-{{ctx.payload.fulldate._value}}/_stats/store",
                "params": {},
                "headers": {},
                "auth": {
                  "basic": {
                    "username": "USERNAME",
                    "password": "PASSWORD"
                  }
                }
              }
            }
          }
        }
      ]
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.http._all.total.store.size_in_bytes": {
        "gte": "53687091200"
      }
    }
  },
  "actions": {
    "send_email": {
      "email": {
        "profile": "standard",
        "to": [
          "EMAILADDRESS@DOMAIN.com"
        ],
        "subject": "ELK Quota Usage exceeded for index: TEST-INDEX-{{ctx.payload.fulldate._value}}",
        "body": {
          "html": "The NAME_OF_DEPARTMENT team exceeded the agreement and sent more data than expected to the index named: TEST-INDEX-{{ctx.payload.fulldate._value}} "
        }
      }
    }
  }
}

I created one entry in the chain for each part of the date (year, month, day) and then merge them together, of course you can simplify this and put it all together in a single "script" key, but this way worked for me better, because I need to adapt the watcher to other indices as well, using weeks instead of days and stuff like that.

4

Hey,

you probably don't need to run scripts, but can use Elasticsearchs built-in functionality for date math support in indices?

--Alex

5

Hey Alex, thanks!
Yes, I tried that at the beginning but couldn't make it work. The date-math expressions doesn't seem to be supported in the "path" of an "http" input. Here is the result of a simulation:

image
image1200×475 27.1 KB

Unfortunately, as you can see it is taken as a literal string.. if you have any idea on how to make it work with date-math expressions I will appreciate it!

6

Hey,

this is actually correct in the simulation, as the exact string needs to be sent like that to Elasticsearch. There could still be encoding issues, so the output of hte execute watch API without simulation for the input would help.

--Alex

7

Thanks, Alexander.
I tried with a real execution and got the exact same behavior, the {now/d} placeholder is not being replaced, see the screenshot:

image
image2945×1270 186 KB

You can give it a try in a dev environment of yours or similar, I will continue to use the script I posted a couple posts before, thanks!

8

Thisis also correct in the execution, the replacement happens on the Elasticsearch side, so you only notice this based on the returned documents and which index they are in.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.