I am capturing IBM Tivoli Schedular logs using Elastic stach - so my idea is to retrigger a failed job based on the reasoning, so using query on the specify field i could get only the failure , but i want to try reach out to tivoli via webhook connectors for rerunning the job. How to do that?
You would need to configure your Webhook connector with the endpoint and credentials and then construct a payload in the alert action, this is explained in the documentation.
Also, do you have a paid license? The webhook connector requires a paid license.
Hi,
Most probably you can use a watcher too.
I am now aware of licensing issues with it.
{
"trigger": {
"schedule": {
"interval": "5m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"xxxxxxxxxxxxx*"
],
"rest_total_hits_as_int": true,
"body": {
"size": 0,
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"gte": "now-30m"
}
}
}
]
}
},
"aggs": {
"hostnames": {
"terms": {
"field": "observer.hostname",
"size": 100
},
"aggs": {
"docs_per_5m": {
"date_range": {
"field": "@timestamp",
"ranges": [
{
"to": "now"
},
{
"from": "now-5m"
}
]
},
"aggs": {
"last_bucket": {
"bucket_sort": {
"sort": [
{
"_key": {
"order": "desc"
}
}
],
"size": 1,
"gap_policy": "insert_zeros"
}
}
}
}
}
}
}
}
}
}
},
"condition": {
"script": {
"source": """
for (def host : ctx.payload.aggregations.hostnames.buckets) {
for (def buckt_docs : host.docs_per_5m.buckets) {
if (buckt_docs.doc_count == 0) {
return true;
}
}
}
""",
"lang": "painless"
}
},
"actions": {
"slack_web_hook": {
"transform": {
"script": {
"source": """
def failed_hosts = new ArrayList();
for (def host : ctx.payload.aggregations.hostnames.buckets) {
for (def buckt_docs : host.docs_per_5m.buckets) {
if (buckt_docs.doc_count == 0) {
def failed_host = new HashMap();
failed_host.put("host",host.key);
failed_host.put("doc_count",buckt_docs.doc_count);
failed_host.put("date",buckt_docs.from_as_string);
failed_hosts.add(failed_host);
}
}
}
return failed_hosts;""",
"lang": "painless"
}
},
"webhook": {
"scheme": "https",
"host": "hooks.slack.com",
"port": 443,
"method": "post",
"path": "services/.................",
"params": {},
"headers": {
"Content-Type": "application/json"
},
"body": """{"channel": "#abc-def-pqrst", "username": "webhookbot", "text": "{{#ctx.payload._value}}HaProxy ingest issue {{host}} has {{doc_count}} documents at {{date}}\n!{{/ctx.payload._value}}"}"""
}
}
}
}
Above a watcher with a webhook to a slack channel.
It checks the number of documents ingested per observer.hostname.
If it is 0 then the webhook is executed.
Regards Hans
Hi HansPeterSloot, Thanks for the reply, mine is just a simple check - index - mdss_lineage-mdss_tivoli_job* , log.message contains 'Procedure error' - it needs action - webhook connector to tivoli master to rerun the respective job. for the inputs i would like to provide JobName, JobID, Workstation as parameter to the URL - which TWS Admins are working to get. Also within Kibana UI - Webhook connector trying to connect and test shows the following error.
Test failed to run
The following error was found:
error calling webhook, request failed
Details:
[DEPTH_ZERO_SELF_SIGNED_CERT] self signed certificate
Hello Mohammed
According to https://discuss.elastic.co/t/webhook-https-ignoring-verification-of-certificate/25874/4
you should look at https://www.elastic.co/guide/en/elasticsearch/reference/current/notification-settings.html#ssl-notification-settings
Regards Hans
Will Give it a try and for now - what we did was to include the host into yml - now at the Kibana level when given the webhook connector, with body as { } and run shows as "Test was successful", but then no flow seen to the destination host where the webhook should connect. - Not exactly sure how to move further to this in order to check if an connection calls or any infor was sent from ELK . based on the connector run button.
What I described is a watcher and not a connector.
Those are 2 different things.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.