Watcher - Groovy User Guide

Elastic Stack Elasticsearch
1

We have created a Watcher script to calculate daily summary of details in our log and stored the output in a new index. This newly created index is not readable by Kibana.

After some googling I came to know that we need Groovy script to make Watcher script output index readable.

Could you please provide me a detailed user guide on Watcher with Groovy which will be helpful for us to achieve our requirement.

2

What version are you running?

3

@warkolm we are using 5.4.3 version.

4

I'm sorry, but I do not understand this sentence

After some googling I came to know that we need Groovy script to make Watcher script output index readable.

Can you expand on this? You do not need groovy to make an index readable. Why do you think so?

Side note: We have developed an own scripting language called painless that should be used whenever possible. It might be faster, but more importantly it is more secure than groovy, especially if external users can trigger scripts.

5

@spinscale Thanks for your reply. As per my organization network restrictions I am not able to upload screenshots of the problem, hence attaching only text so kindly bear with me.

I found the below link, where they used Watcher along with Groovy,

So I assumed that I need to use Groovy with Watcher to make it work.

On trying to view my index(created by Watcher) in Discover tab in Kibana, I am getting the below error,

Saved "field" parameter is now invalid. Please select a new field.
Discover: "field" is a required parameter

Watcher Script Used:

{
"trigger": {
"schedule": {
"interval": "24h"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"orbpm_bpmi_metrics-"
],
"types": [],
"body": {
"query": {
"bool": {
"must": [
{
"query_string": {
"analyze_wildcard": true,
"query": ""
}
},
{
"range": {
"@timestamp": {
"gte": 1501512796294,
"lte": 1501513696294,
"format": "epoch_millis"
}
}
}
],
"must_not": []
}
},
"size": 0,
"_source": {
"excludes": []
},
"aggs": {
"4": {
"terms": {
"field": "route",
"exclude": "route1|route2|route3",
"size": 50,
"order": {
"1": "desc"
}
},
"aggs": {
"1": {
"max": {
"field": "max_process_time"
}
},
"5": {
"date_histogram": {
"field": "@timestamp",
"interval": "1d",
"time_zone": "America/New_York",
"min_doc_count": 1
},
"aggs": {
"1": {
"max": {
"field": "max_process_time"
}
},
"2": {
"min": {
"field": "min_process_time"
}
},
"3": {
"avg": {
"field": "avg_process_time"
}
}
}
}
}
}
}
}
}
}
},
"condition": {
"always": {}
},
"actions": {
"index_payload": {
"index": {
"index": "orbpm_daily_summary_bpmi",
"doc_type": "json"
}
}
}
}

orbpm_daily_summary_bpmi INDEX:

name-------type
aggregations.4.buckets.5.buckets.key_as_string string
aggregations.4.buckets.5.buckets.1.value number
aggregations.4.buckets.doc_count number
aggregations.4.buckets.5.buckets.doc_count number
hits.max_score number
aggregations.4.buckets.5.buckets.key number
aggregations.4.buckets.1.value number
took number
hits.total number
aggregations.4.buckets.5.buckets.2.value number
timed_out boolean
message string
@timestamp date
aggregations.4.buckets.key string
aggregations.4.buckets.5.buckets.3.value number
_source _source
aggregations.4.sum_other_doc_count number
aggregations.4.doc_count_error_upper_bound number
_id string
_type string
_index string
_score number

6

Hey, so this a Kibana or an Elasticsearch problem now? I want to make sure I understand this first.

If you use the console/dev tools, can you search in that index?

7

@spinscale Not sure whether it is a Kibana or an Elasticsearch problem. Queried the data contents of the index created by Watcher. It looks like below,

{
"1" : {
"value" : 1487735.375
},
"doc_count" : 3,
"5" : {
"buckets" : [
{
"key_as_string" : "2017-07-31T00:00:00.000-04:00",
"1" : {
"value" : 1487735.375
},
"doc_count" : 3,
"2" : {
"value" : 1373.9156494140625
},
"3" : {
"value" : 1019515.5625
},
"key" : 1501473600000
}
]
},
"key" : "bpmServiceRouteId"
}

Kindly guide me how to fix this problem. It looks like a nested JSON.

8

@spinscale can you please help me with this here.

Kindly let me know the details which you need from my end.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.