Access to now() in watcher painless script. How to pass current time in as param?

[Dan_Gardner]

I'm converting watches to be compatible with elastic cloud 5.2.2 and moving from groovy to painless in the process.

Our current watches have an inline script field in the query that does the following:
doc['@timestamp'].date.getSecondOfDay() < DateTime.now().getSecondOfDay()

I've now read that painless inline scripts no longer support getting now(). Instant.now() It is suggested to pass in the current datetime as a param to the script. How can I do this for an inline script ?

[spinscale (Alexander Reelsen)]

Hi

A watch contains the scheduled time and the execution time in its context, which you could pass on.Maybe you can share your watch, so we can work on a solution.

--Alex

[Dan_Gardner]

Thx for your response Alexander, I've only just seen this (email notification broken?).

I need now() in the input query, is that accessible ? I was imagining access to execution & scheduler time was only available in the response context, not the request context.

Either way, here is our watch pre-detokenisation (@env@) & with a few things replaced (by BLABLA) to make it possible to share. The condition & transform is still a WIP. The important bit is the script in the input search. Thats where I need to know now().

Thank you for any help you may have time to offer, hugely appreciated

{
"trigger": {
"schedule": {
"cron": "BLABLA"
}
},
"input": {
"search": {
"request": {
"indices": [
"logstash-@env@-*"
],
"body": {
"query": {
"bool": {
"must": [
{
"wildcard": {
"key": "BLABLA.latest.logs.crawl.log"
}
},
{
"match": {
"http_status": "200"
}
},
{
"match": {
"mimeType": "text/html"
}
},
{
"range": {
"@timestamp": {
"from": "now/d-7d",
"to" : "now"
}
}
},
{
"script": {
"script": {
"inline" : "doc['@timestamp'].date.getSecondOfDay() < DateTime.now().getSecondOfDay()",
"lang" : "painless"
}
}
}
]
}
},
"size": 0,
"aggregations": {
"jobs": {
"terms": {
"script": {
"inline" : "doc['key'].value.subSequence(26,doc['key'].value.length() - 22)",
"lang" : "painless"
}
},
"aggregations": {
"history": {
"date_range": {
"field": "@timestamp",
"format": "yyyy-MM-dd",
"ranges": [
{
"key" : "Yesterday",
"from": "now/d-1d",
"to": "now-1d"
},
{
"key" : "2 days ago",
"from": "now/d-2d",
"to": "now-2d"
},
{
"key" : "5 days ago",
"from": "now/d-5d",
"to": "now-5d"
},
{
"key" : "3 days ago",
"from": "now/d-3d",
"to": "now-3d"
},
{
"key" : "4 days ago",
"from": "now/d-4d",
"to": "now-4d"
},
{
"key" : "6 days ago",
"from": "now/d-6d",
"to": "now-6d"
},
{
"key" : "7 days ago",
"from": "now/d-7d",
"to": "now-7d"
}
]
},
"aggregations": {
"count": {
"value_count": {
"field": "key"
}
}
}
},
"current": {
"date_range": {
"field": "@timestamp",
"format": "yyyy-MM-dd",
"ranges": [
{
"key": "Today",
"from": "now/d",
"to": "now"
}
]
}
},
"avg_historic_daily_count": {
"avg_bucket": {
"buckets_path": "history>count"
}
}
}
}
}
}
}
}
},
"condition": {
"script": {
"inline": "historicCompare = ctx.payload.aggregations.jobs.buckets[0].current.buckets[0].doc_count / ctx.payload.aggregations.jobs.buckets[0].avg_historic_daily_count.value; return ( historicCompare < lowerThreshold ) || ( historicCompare > upperThreshold )",
"lang" : "groovy",
"params" : {
"upperThreshold" : @watcher.BLABLA.upperThreshold@,
"lowerThreshold" : @watcher.BLABLA.lowerThreshold@
}
}
},
"transform": {
"script" : {
"inline" : "avg_historic_daily_count = ctx.payload.aggregations.jobs.buckets[0].avg_historic_daily_count.value; return [ jobs : ctx.payload.aggregations.jobs.buckets.collect { [ avg_historic_daily_count : avg_historic_daily_count.round(), history : it.history.buckets.collect { [ day : it.key, doc_count : it.doc_count, color : ( ( it.doc_count / avg_historic_daily_count ) > upperThreshold || ( it.doc_count / avg_historic_daily_count ) < lowerThreshold ) ? '@watcher.to.slack.alert.colour@' : 'good' ] }, current : it.current.buckets.collect { [ day : it.key, doc_count : it.doc_count, color : ( ( it.doc_count / avg_historic_daily_count ) > upperThreshold || ( it.doc_count / avg_historic_daily_count ) < lowerThreshold ) ? '@watcher.to.slack.alert.colour@' : '#000000' ] } ] } ]",
"lang" : "groovy",
"params" : {
"upperThreshold" : @watcher.BLABLA.upperThreshold@,
"lowerThreshold" : @watcher.BLABLA.lowerThreshold@
}
}
},
"actions" : {
"notify-slack" : {

... cut short to fit inside 7000 post limit

[spinscale (Alexander Reelsen)]

Hey,

sooooo... that's a tricky one. But you can make it work (not saying it's nice).

First, you could use the date histogram aggregation to make your request a bit shorter (if you are after full days, in addition you could also use the offset parameter).

Second, I came up with this - which is using a http input instead of a search one (note, the three double ticks is a leftover from kibana, where this allows you to use multi line strings, you need to properly escape everything, as at the end the body is a single string

POST _xpack/watcher/watch/_execute
{
  "watch": {
    "trigger": {
      "schedule": {
        "interval": "10s"
      }
    },
    "input": {
      "http": {
        "request" : {
        "url" : "http://localhost:9200/foo/_search",
        "auth" : {
          "basic" : {
            "username" : "elastic",
            "password" : "changeme"
          }
        },
        "body": """{
          "query": {
            "bool": {
              "must": [
                {
                  "script": {
                    "script": {
                      "inline": "def x = DateTimeFormatter.ISO_OFFSET_DATE_TIME.parse(params.p1) ; doc['@timestamp'].date.getSecondOfDay() > x.get(ChronoField.SECOND_OF_DAY) ",
                      "lang": "painless",
                      "params": {
                        "p1": "{{ctx.trigger.scheduled_time}}"
                      }
                    }
                  }
                }
              ]
            }
          }
        }"""
        }
      }
    },
    "actions": {
      "logme": {
        "logging": {
          "text": "{{ctx.payload}}"
        }
      }
    }
  }
}

Hope this clears things up a bit. Yet, I dont come up with a better idea.

--Alex