Passing ctx fields to stored proc as parameter

[Matt_McGovern]

I am managing hundreds of scripts that all use an inline script to go through the nested buckets inside the ctx.payload structure. We've recently run into the circuit break error of: "reason": "[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting".

We can increase that setting, but going forward it would be better if I could create a stored script and have all the watchers call it. The problem is I haven't been able to send the ctx.payload structure through as a parameter.

Is there any way of doing this?

[spinscale (Alexander Reelsen)]

Can you explain what you are doing instead of sending ctx.payload as a parameter? Can you show the request you want to generate?

Maybe just share a whole sample watch, if possible?

--Alex

[elastock (Guillaume Dufrenne)]

stored scripts have access to the payload .

POST _scripts/the_script`
 {"script": {
     "lang": "painless",
    "code": """ def docs = []
                docs.add(['_id': ctx.payload.hits.hits[0]._id]);
          """
  }
 }

they must be executed after the payload has been returned

[Matt_McGovern]

//here is the stored script _scripts/IsAppTotalsInErrPct:
if(params.ApplicationTotals.buckets.length>0)
{
for(int a=0;a<params.ApplicationTotals.buckets.length;++a)
{
if(params.ApplicationTotals.buckets[a].ContractTotals.buckets.length>0)
{
for(int c=0;c<params.ApplicationTotals.buckets[a].ContractTotals.buckets.length;++c)
{
if(params.ApplicationTotals.buckets[a].ContractTotals.buckets[c].Errorpercent.value>=params.Threshold)
{
if(params.ApplicationTotals.buckets[a].ContractTotals.buckets[c].Contract_Errors.doc_count>=params.MinimumErrRecords)
{
return true;
}
}
}
}
}
}
return false;

//here is the watcher that will call the stored script above:
{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"prod-logs*"
],
"types": [],
"body": {
"size": 0,
"query": {
"bool": {
"filter": [
{
"term": {
"BaseUrl": "https://login.comcast.net"
}
},
{
"range": {
"@timestamp": {
"gte": "now-10m"
}
}
}
]
}
},
"aggs": {
"ApplicationTotals": {
"terms": {
"field": "AppName"
},
"aggs": {
"ContractTotals": {
"terms": {
"field": "Resource"
},
"aggs": {
"Contract_Errors": {
"filter": {
"term": {
"Severity": "high"
}
}
},
"Errorpercent": {
"bucket_script": {
"buckets_path": {
"totalcount": "_count",
"failurecount": "Contract_Errors._count"
},
"script": "Math.round(params.failurecount / params.totalcount * 10000.000) / 100.000"
}
}
}
}
}
}
}
}
}
}
},
"condition": {
"script": {
"id": "IsAppTotalsInErrPct",
"params": {
"ApplicationTotals": "{{ctx.payload.aggregations.ApplicationTotals}}",
"Threshold": "0",
"MinimumErrRecords": "0"
}
}
},
"actions": {
"notify-slack": {
"throttle_period_in_millis": 300000,
"transform" : {
"script" : {
"source" : "String errMsg='';if(ctx.payload.aggregations.ApplicationTotals.buckets.length>0){for(int a=0;a<ctx.payload.aggregations.ApplicationTotals.buckets.length;++a){if(ctx.payload.aggregations.ApplicationTotals.buckets[a].ContractTotals.buckets.length>0){for(int c=0;c<ctx.payload.aggregations.ApplicationTotals.buckets[a].ContractTotals.buckets.length;++c){if(ctx.payload.aggregations.ApplicationTotals.buckets[a].ContractTotals.buckets[c].Errorpercent.value>=<>){if(ctx.payload.aggregations.ApplicationTotals.buckets[a].ContractTotals.buckets[c].Contract_Errors.doc_count>=<>){errMsg+=ctx.payload.aggregations.ApplicationTotals.buckets[a].key+', '+ctx.payload.aggregations.ApplicationTotals.buckets[a].ContractTotals.buckets[c].key+' '+ctx.payload.aggregations.ApplicationTotals.buckets[a].ContractTotals.buckets[c].Errorpercent.value+'%\n'+ctx.payload.aggregations.ApplicationTotals.buckets[a].ContractTotals.buckets[c].Contract_Errors.doc_count + ' errors out of ' + ctx.payload.aggregations.ApplicationTotals.buckets[a].ContractTotals.buckets[c].doc_count + ' documents.\n'}}}}}}return errMsg;",
"lang" : "painless"
}
},
"slack": {
"message": {
"from": "Kibana Watcher",
"to": [
"#slackchannel"
],
"text": "Exception rate alert received",
"attachments": [
{
"color": "danger",
"title": "Elevated Error Rate above <>% for time: <>",
"title_link": "Our Kibana URL",
"text": "{{ctx.payload._value}}"
}
]
}
}
}
}
}

[Matt_McGovern]

Elastock, to do this, my watcher (posted above) would have to first post the payload to the docs array, then call the stored script which would access it?

[Matt_McGovern]

I was able to solve my problem....the ctx is not passed through as a parameter; it is recognized by session. I have many watchers all calling the same stored script which references ctx.payload. When the script is called it references the ctx that corresponds to the session of the watcher that calls it at the time. I didn't think this would work but that is exactly how it works.