How to parse Watcher/Aggs in JSON format into a table automatically?

Elastic Stack Elasticsearch
1

The output of an aggregation in search (or in watcher), currently comes as JSON and contains key-value. The outcome is different and may contain different fields, so hardcoding them into an html body is not possible. Is there a way I can make the output of WATCHER into an html table (action of email) automatically?

For example

{
    "size": 0,
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-3h"
                    }
                  }
                },
                {
                  "match": {
                    "event.dataset": "sample_web_logs"
                  }
                }
              ]
            }
          },
          "aggs": {
            "group_by_src": {
              "terms": {
                "field": "geo.src",
                "size": 2
              },
              "aggs": {
                "group_by_dest": {
                  "terms": {
                    "field": "geo.dest",
                    "size": 2
                  }
                }
              }
            }
          }
        }

The data output is

{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 7824,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "group_by_src" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 5086,
      "buckets" : [
        {
          "key" : "CN",
          "doc_count" : 1379,
          "group_by_dest" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 921,
            "buckets" : [
              {
                "key" : "CN",
                "doc_count" : 246
              },
              {
                "key" : "IN",
                "doc_count" : 212
              }
            ]
          }
        },
        {
          "key" : "IN",
          "doc_count" : 1359,
          "group_by_dest" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 827,
            "buckets" : [
              {
                "key" : "IN",
                "doc_count" : 307
              },
              {
                "key" : "CN",
                "doc_count" : 225
              }
            ]
          }
        }
      ]
    }
  }
}

I needed the output in the email to be an html table without individually picking up the key-value into a hard-coded html

2

the template language used to write your own emails is mustache. Mustache allows you to loop through a list like the buckets array, and then write out the key in the body.

You may want to take a look at the example watches in the examples repo at https://github.com/elastic/examples/tree/master/Alerting/Sample Watches

3

hi, sorry I may not have been clear. I can work with mustache/HTML, but I have to hard-code lot of these manually. I was mostly asking if the key value pairs can be automatically put into the table (i.e without knowing the key-value field names). So I'm looking for something like as output of aggregation/transform

<mylogic>
| fields [{key1:value1},{key2:value2},{key3:value3}..]

automatically, so in mustache, I can iterate through key/value without hard-coding them

4

You would need to dump the whole map data structure at once (but then it's not formatted), or use a transform to convert the random key/value values to something like {key: 'your_key_alue', value: 'your_value'} and could then access it stable via element.key and element.value

hope that helps.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.