Aggregations: How to get a only the entries with latest status 'nok' and nothing else


(Sven Beauprez) #1

Suppose I have following mapping for documents
_timestamp: ES timestamp enabled
mod_id : string (a unique ID for a module, not the same as _id field from
ES)
status_code : integer (similar as to HTTP codes where 200 is ok and all
else is nok)

With following aggregation, I get for all modules (buckets) an aggregation
of the status codes, with the latest submitted status code on top:

"aggs": {
"by_module": {
"terms": {
"field": "mod_id"
},
"aggs": {
"by_status": {
"terms": {
"field": "status_code",
"order": {
"max_time": "desc"
}
},
"aggs": {
"max_time": {
"max": {
"field": "_timestamp"
}
}
}
}
}
}
}

result:
"aggregations": {
"by_module": {
"buckets": [
{
"key": "ModuleUniqueID12",
"doc_count": 4,
"by_status": {
"buckets": [
{
"key": 503,
"doc_count": 2,
"max_time": {
"value": 1394750966731
}
},
{
"key": 200,
"doc_count": 2,
"max_time": {
"value": 1394745749862
}
}
]
}
},
{
"key": "ModuleUniqueID1",
"doc_count": 2,
"by_status": {
"buckets": [
{
"key": 200,
"doc_count": 2,
"max_time": {
"value": 1394729958485
}
}
]
}
},

        ... //and so on
    ]
  }

}

What I want now is only the documents where the latest (-> this is the hard
part) entries for a module contains a status_code that is not ok, ie. and
the above resultset I would only get the document with mod_id
"ModuleUniqueID12", because the latest entry added to ES has a status_code
of 503.

Can this be filtered combined with the 'max_time' aggregation metric for
example? Any other ways? How would I use the 'max_time' metric in a script?

thnx!

Sven

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/601181d2-6888-47f6-bf95-6b7708a587b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Sven Beauprez) #2

It might be related to github.com/elasticsearch/elasticsearch/issues/4404but it seems that that is not yet implemented. So a solution that works
with 1.0 version would be welcome.

regards,

Sven

On Friday, March 14, 2014 9:50:57 AM UTC+1, Sven Beauprez wrote:

Suppose I have following mapping for documents
_timestamp: ES timestamp enabled
mod_id : string (a unique ID for a module, not the same as _id field from
ES)
status_code : integer (similar as to HTTP codes where 200 is ok and all
else is nok)

With following aggregation, I get for all modules (buckets) an aggregation
of the status codes, with the latest submitted status code on top:

"aggs": {
"by_module": {
"terms": {
"field": "mod_id"
},
"aggs": {
"by_status": {
"terms": {
"field": "status_code",
"order": {
"max_time": "desc"
}
},
"aggs": {
"max_time": {
"max": {
"field": "_timestamp"
}
}
}
}
}
}
}

result:
"aggregations": {
"by_module": {
"buckets": [
{
"key": "ModuleUniqueID12",
"doc_count": 4,
"by_status": {
"buckets": [
{
"key": 503,
"doc_count": 2,
"max_time": {
"value": 1394750966731
}
},
{
"key": 200,
"doc_count": 2,
"max_time": {
"value": 1394745749862
}
}
]
}
},
{
"key": "ModuleUniqueID1",
"doc_count": 2,
"by_status": {
"buckets": [
{
"key": 200,
"doc_count": 2,
"max_time": {
"value": 1394729958485
}
}
]
}
},

        ... //and so on
    ]
  }

}

What I want now is only the documents where the latest (-> this is the
hard part) entries for a module contains a status_code that is not ok, ie.
and the above resultset I would only get the document with mod_id
"ModuleUniqueID12", because the latest entry added to ES has a status_code
of 503.

Can this be filtered combined with the 'max_time' aggregation metric for
example? Any other ways? How would I use the 'max_time' metric in a script?

thnx!

Sven

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/322f9d73-9743-4380-b5e8-c26c997de5cd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3