Add the data from a table in a watcher

[Nalin]

Hi All,

We are able to generate data in a tabular format through a watcher

IndexName	StoreSize(in MB)	DocumentCount
index1	23	220
index-2	20	129
index-3	26	178

Is there any way in which we could add the total Store size and total document count separately and print that in the Watcher mail?

Regards,
Nalin

[Nalin]

Hi @richcollier ,

Could you please help on this?

Regards,
Nalin

[richcollier]

Can you provide the content of your current watch?

[Nalin]

Hi Rich,

Please find the current watcher below:

Watcher:
{
"trigger": {
"schedule": {
"weekly": [
{
"on": [
"FRI"
],
"at": [
"06:10"
]
}
]
}
},
"input": {
"http": {
"request": {
"scheme": "https",
"host": "#######",
"port": 9243,
"method": "get",
"path": "_cat/indices/*",
"params": {
"format": "yaml",
"human": "true",
"bytes": "mb"
},
"headers": {},
"auth": {
"basic": {
"username": "elastic",
"password": "#######"
}
}
}
}
},
"condition": {
"script": {
"source": "return true;",
"lang": "painless"
}
},
"actions": {
"displayResults": {
"logging": {
"level": "info",
"text": "the output is: {{ctx.payload.data}}"
}
},
"email_administrator": {
"email": {
"profile": "standard",
"attachments": {
"index.yml": {
"data": {
"format": "yaml"
}
}
},
"to": [
"abc@xyz.com"
],
"subject": "Watcher Alert",
"body": {
"html": """

Index Data SizePlease find the required data below:IndexNameStoreSize(in MB)DocumentCount{{#ctx.payload.data}}{{index}}
{{/ctx.payload.data}}{{#ctx.payload.data}}{{store.size}}
{{/ctx.payload.data}}{{#ctx.payload.data}}{{docs.count}}
{{/ctx.payload.data}}""" } } } } }

My requirement is to add the total StoreSize(in MB) and print it along with the table, in the watcher mail.

It would be of great assistance if you help me with this.

Thanks & Regards,
Nalin

[richcollier]

You're going to have to build a more complicated data structure and also increment a few variables to keep track of the totals...something like:

      "displayResults": {
        "transform": {
          "script": """
           def indices = new HashMap();
           def totals= new HashMap(); 

           def total_docs=0L;
           def total_store=0L;
           for (def data : ctx.payload.data) {
               String key = data.index;
               def info = indices.get(key);
               if (info == null) {
                 info = new HashMap();
                 indices.put(key, info);
               }
               info.put("doc_count",data['docs.count']);
               info.put("store_size",data['store.size']);
               total_docs += Integer.parseInt(data['docs.count']);
               total_store += Integer.parseInt(data['store.size']);
            }
            totals.put("total_docs",total_docs);
            totals.put("total_store",total_store);
            indices.put("totals",totals);
            
            return indices;
          """
        },
        "logging": {
          "text": "{{ctx.payload}}"
        }
      }

So that your payload looks like this:

            "payload": {
              "totals": {
                "total_store": 40468078,
                "total_docs": 31808
              },
              "kibana_sample_data_ecommerce": {
                "doc_count": "4675",
                "store_size": "8888714"
              },
              "kibana_sample_data_logs": {
                "doc_count": "14074",
                "store_size": "18893956"
              },
              "kibana_sample_data_flights": {
                "doc_count": "13059",
                "store_size": "12685408"
              }
            }