Background information:
I'm using Elastic APM to collect API data and send it to Elasticsearch. Now I would like to run a watcher that searches the data for latency. I would like to group all the trace.id of all the documents that pass the 1-second threshold, and then include them in a link that once the user clicks on it, it redirects them to Kibana. I'm having problems with grouping the trace.id, I don't know how it is done in Elasticsearch query or painless script.
Here is the logic I would like to accomplish in python:
Python code:
arr = []
if service_name == 'TEST-B2-UI' and transaction_duration >= 1000000:
arr.append(trace_id)
Current Elasticsearch Code (Fails to execute):
PUT _watcher/watch/prod-b2-ui-latency
{
"trigger": {
"schedule": {
"interval": "6h"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"apm-*"
],
"rest_total_hits_as_int": true,
"body": {
"query": {
"bool": {
"filter": [
{
"script": {
"script": {
"source": "doc['service.name'].value == {{ctx.metadata.service_name}} && doc['transaction.duration.us'].value >= {{ctx.metadata.latency_threshold}}"
}
}
},
{
"range": {
"@timestamp": {
"from": "now-6h/h",
"to": "now"
}
}
}
]
}
},
"script": {
"inline": "doc['tags'].value += doc['trace.id].value",
"lang": "painless"
},
"sort": {
"transaction.duration.us": "desc",
"@timestamp": "desc"
},
"_source": [
"@timestamp",
"transaction.duration.us",
"transaction.name",
"service.name",
"trace.id"
]
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 0
}
}
}
The issue comes from this script code in the above code:
"script": {
"inline": "doc['tags'].value += doc['trace.id].value",
"lang": "painless"
},
You can see I'm trying to execute the exact same logic here, but not sure if this is the way it is done.
Is there any way I can write Java code directly in this search query? Maybe it will be easier, unless you have a better idea in mind. I appreciate any feedback/reference.