Can/should percolator be set up to return when an aggregation value hits a certain threshold?

(Chris) #1

Take the following aggregation query as an example:

  "query": {
    "match_all": {}
  "aggs": {
    "groupBy": {
      "terms": {
        "field": "CustomerName"
      "aggs": {
        "points_sum": {
          "stats": {
            "field": "TransactionAmount"
  "size": 0

I am interested in knowing when any CustomerName has an average TransactionAmount (stats.avg) that is above some threshold for all of that customer's purchases, as soon as I index a document that would put my average above that threshold. It seems like percolator is designed for matching documents to rules, more or less, but I can't find any good examples of using percolator to match rules that are based on aggregation results.

Is this possible? Is percolator the best solution here? Is there another/better solution? Thanks in advance :slight_smile:

(Mark Harwood) #2

Percolator queries are designed to test properties of a single document.
Your example is testing properties of a new document but with respect to some previous history.

One option is to use Watcher to poll your index looking for these anomalies - see [1]
If your logic is complex it may also be worth considering creating an entity-centric index to represent each customer and use a script to re-assess the risk attached to a new transaction in respect of their historic behaviour.



(system) #3