Use condition script to compare a value from aggregation part

alerting

(Pransh) #1

{
"took": 529,
"timed_out": false,
"_shards": {
"total": 1040,
"successful": 1040,
"failed": 0
},
"hits": {
"total": 46898,
"max_score": 0,
"hits": []
},
"aggregations": {
"2": {
"buckets": [{
"6": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [{
"1": {
"value": null
},
"key": "xxxt",
"doc_count": 4745
}, {
"1": {
"value": null
},
"key": "g2t4455c.austin.hpecorp.net",
"doc_count": 4714
}]
},
"key_as_string": "2016-11-09T23:00:00.000Z",
"key": 1478732400000,
"doc_count": 9459
}, {
"6": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [{
"1": {
"value": 90.5307057745188
},
"key": "xxt",
"doc_count": 18714
}, {
"1": {
"value": 83.9568277803848
},
"key": "xx",
"doc_count": 18721
}]
}
}]
}
}

This is my query output i want a condition where it checks if "value" is greater than 80 using script or something but not able to achieve it if anyone can help me with

i am currently using
"condition" : {
"array_compare" : {
"ctx.payload.aggregations.2.buckets": {
"path": "doc_count",
"gte": {
"value": 0,
"quantifier": "some"
}
}
}
},

How to enter the nested bucket
Please help me with this @warkolm @spinscale @beckerdo


(Dan Becker) #2

Hi Pransh,

Which instance of Elastic are you using?

In general the scripting must be enabled by the DB admin on that server. What is the error you are seeing from the watch?


(Pransh) #3

Hi @beckerdo
Thanks for the reply

Script is enabled, that is not a problem

In the condition part I am not able to parse the inner array in aggregation part while writting the condition

if you have an alternate approach to put the condition at query level only for e.g something like this:

I want something like this

aggs:{
"avg":
{ some value}

and this avg value should be greater than some threshold...

I am using watcher 2.2, please let me know if you don't understand the question, I will explain further


(Pransh) #4

I am able to get the hits using the condition i posted originaly but i want to compare with a field from the inner array of the aggregation output,

Aggregation -> 2 -> buckets -> 6 -> buckets -> 1-> vaule > threshold


(Pransh) #5
{
"query": {
	"filtered": {
		"query": {
			"query_string": {
				"query": "type:*-itg AND  Log_message:\"Processed with status: true\" AND Module: \"AcceptFile\"",
				"analyze_wildcard": true
			}
		},
		"filter": {
			"bool": {
				"must": [{
					"query": {
						"query_string": {
							"analyze_wildcard": true,
							"query": "*"
						}
					}
				}, {
					"range": {
						"@timestamp": {
							"from": "now-10h",
							"to": "now"
						}
					}
				}],
				"must_not": []
			}
		}
	}
},
"size": 0,
"aggs": {
	"2": {
		"date_histogram": {
			"field": "TIME",
			"interval": "30m",
			"time_zone": "UTC",
			"min_doc_count": 1,
			"extended_bounds": {
				"min": 1478736000000,
				"max": 1478822399999
			}
		},
		"aggs": {
			"6": {
				"terms": {
					"field": "hostname.raw",
					"size": 5,
					"order": {
						"1": "desc"
					}
				},
				"aggs": {
					"1": {
						"avg": {
							"field": "NfsCopyTime"
						}

						
					}
				}
			}
		}
	}
}

}

Query that i am using


(Dan Becker) #6

Hello,

I do recall that if you want to do work with aggregation, you tend to have to promote the aggregation buckets. Here is an example aggregation that we ran, found some buckets, and then put the buckets into a new index. I hope this example helps.

{
"trigger" : {
"schedule" : {
"daily" : { "at" : "11:00" }
}
},
"input" : {
"http" : {
"request" : {
"host" : “",
"port" : 9200,
"headers": {
"Content-Type" : "application/json",
"Accept" : "application/json"
},
"path" : “//logs/_search",
"method": "post",
"body" : “{}"
},
"extract": ["hits.total", "aggregations.tpv.buckets"]
}
},
"actions": {
"log": {
"logging": {
"text": "Watcher ran {{ctx.watch_id}} at {{ctx.execution_time}}. Total docs={{ctx.payload.hits.total}}."
}
},
"put_daily_agg": {
"transform": {
"script": "return [ _doc : ctx.payload.aggregations.tpv.buckets ]"
},
"index": {
"index": “",
"doc_type": “"
}
}
}
}
Thanks, Dan Becker
Office: 512 691-4358, Slack: @dabecker
PayPal, 7700 W. Parmer Lane, Austin, TX 78729


(Alexander Reelsen) #7

Hey,

it would help a lot, if you added some formatting to your JSON output - just pasting it in here makes it nearly impossible to read and you also loose the nice looking indendation - see this discourse post.

Also the array condition can only be used to walk through one array, but you got buckets and subbuckets, where you need a script condition.

--Alex


(Pransh) #8

Hi @spinscale

Yes sure I will add the code with proper indentation, Yes you understood my problem . I am not able to design the script to reach to subbuckets, please help me how to compare elements from subbuckate, i want an Integer value to be greater than some threshold.


(Alexander Reelsen) #9

Hey,

you should share what you came up with (including a full blown example), so people can chime in and help. There is an older blogpost about building an anomaly detector shows some example groovy code.

--Alex


(system) #10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.