Using the payload data (fields) after aggregated to bucket, in Elastic Watcher Action email

Elastic Stack Elasticsearch
1

HI,

I am seeking some help to resolve an issue that I am having in one of my Elastic Watchers. First of all, I must say this is a complex Watcher.

In our environment, we gather some log data from file transferring jobs from various nodes' log files and send them to elastic via Logstash. This part works and we variously use those documents.

I am developing a watcher to find any errors from log files and send alert emails. That is working(This is version one of my solution).

I am running the watcher for every 15 minutes and scan the data for 15 minutes periods back from now and then aggregate the documents to brackets based on the log file name filed. That part also works.

The email action I run for each aggregated buckets ("foreach": "ctx.payload.aggregations.failed_transfers.buckets") and email Subject line I print {{ctx.payload.doc_count}} and {{ctx.payload.key}}. That part also works.

What I am failed to do is, in the email body repeat each document’s-related information.

This means each email will have several lines, one line per document saying what is the error and @timestamp of the error and other information about that error. For this, I am unable to get the correct variable names.

I can see the ctx.payload.hits.hits have all this information, but after aggregating to buckets based on the source of the error (in this case the log file name), I don't know how to access those initial payload field values.

If someone can help me with the proper syntax, highly appreciated it.

Best Regards

Sarath

2

Hey,

you cannot access the global payload within a for loop currently, just the part of that iteration. Feel free to open a feature request in the Elasticsearch repo.

As a workaround, you could use a transform to include the bucket data from your aggregation and the hits in your own list, and then use foreach on that list.

--Alex

3

Hi Alex,

Thanks for the reply.

"As a workaround, you could use a transform to include the bucket data from your aggregation and the hits in your own list, and then use foreach on that list."
How can do this? I am already using 'foreach', for the bucket aggregate

Best Regards
Sarath

4

Transforms allow you to modify your data... see Watcher script payload transform | Elasticsearch Guide [8.1] | Elastic

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.