One query that monitors multiple results. - Need help

[t.Farestad]

Essentially, I'd like to write one query/watch that can essentially do a for each on the results.

We have multiple services (each defined within the key) and we'd like to write one query that can compare the success/failure rate of each individual key. My results are below, but each key will have a success and error bucket (if error exists).

How can I write one query that will tell me the individual service that has a high error rate from this query, vs writing a multitude of queries, one for each individual services?

Here is a shortened version of my results:
"aggregations": {
"services": {
"doc_count_error_upper_bound": 1190,
"sum_other_doc_count": 480216,
"buckets": [
{
"key": "searchincidentmgmtdata",
"doc_count": 93852,
"histo": {
"buckets": [
{
"key_as_string": "2017-03-03T04:00:00.000Z",
"key": 1488513600000,
"doc_count": 1226,
"status": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "success",
"doc_count": 1226
}
]
}
},
{
"key_as_string": "2017-03-03T08:00:00.000Z",
"key": 1488528000000,
"doc_count": 297,
"status": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "success",
"doc_count": 297
}
]
}
},
{
"key_as_string": "2017-03-03T12:00:00.000Z",
"key": 1488542400000,
"doc_count": 12673,
"status": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "success",
"doc_count": 12673
}
]
}
},
{
"key_as_string": "2017-03-03T16:00:00.000Z",
"key": 1488556800000,
"doc_count": 30519,
"status": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "success",
"doc_count": 30519
}
]
}
},
{
"key_as_string": "2017-03-03T20:00:00.000Z",
"key": 1488571200000,
"doc_count": 33711,
"status": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "success",
"doc_count": 33711
}
]
}
},
{
"key_as_string": "2017-03-04T00:00:00.000Z",
"key": 1488585600000,
"doc_count": 14764,
"status": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "success",
"doc_count": 14764
}
]
}
},
{
"key_as_string": "2017-03-04T04:00:00.000Z",
"key": 1488600000000,
"doc_count": 662,
"status": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "success",
"doc_count": 662
}
]
}
}
]
}
},
{
"key": "getclientchannel"
"doc_count": 40823,
"histo": {
"buckets": [
{
"key_as_string": "2017-03-03T04:00:00.000Z",
"key": 1488513600000,
"doc_count": 4016,
"status": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "success",
"doc_count": 3896
},
{
"key": "error",
"doc_count": 120
}
]
}
}

[Christian_Dahlqvist]

How many services do you have? What action will you take based on the error rates you find? Are you only interested in services with errors?

[t.Farestad]

Here is the query to get the # of success/errors for each service in a particular bucket.

{
"size": 0,
"query": {
"range": {
"@timestamp": {
"gte": "now-1d"
}
}
},
"aggs": {
"services": {
"terms": {
"field": "LogEntry.ProcessName",
"min_doc_count": 10
},
"aggs": {
"histo": {
"date_histogram": {
"field": "@timestamp",
"interval": "4h"
},
"aggs": {
"status": {
"terms": {
"field": "LogEntry.ProcessResult"
}
}
}
}
}
}
}
}

What we need is essentially the success/total for each bucket, if it drops below say 99.9, we want to open a ticket in ServiceNow via Evanios.

[t.Farestad]

Just figured it out after playing around with it again this morning:

GET /applog-*/_search
{
"size": 0,
"query": {
"range": {
"@timestamp": {
"gte": "now-1d"
}
}
},
"aggregations": {
"services": {
"terms": {
"field": "LogEntry.ProcessName",
"min_doc_count": 10
},
"aggs": {
"histo": {
"date_histogram": {
"field": "@timestamp",
"interval": "4h"
},
"aggs": {
"success_filter": {
"filter": {
"term": {
"LogEntry.ProcessResult": "success"
}
}
},
"success_rate" : {
"bucket_script": {
"buckets_path": {
"total_successes": "success_filter>_count",
"total_count": "_count"
},
"script": "total_successes / total_count * 100"
}
}
}
}
}
}
}
}

This gives me my expected results, with a field called "Success_rate" within each bucket that I can watch off of.

Example result:
"histo": {
"buckets": [
{
"key_as_string": "2017-03-05T16:00:00.000Z",
"key": 1488729600000,
"doc_count": 650,
"success_filter": {
"doc_count": 650
},
"success_rate": {
"value": 100
}
}