I am trying to Use Webhook connector from ELK- Need Help

Elastic Stack Kibana
1

I am capturing IBM Tivoli Schedular logs using Elastic stach - so my idea is to retrigger a failed job based on the reasoning, so using query on the specify field i could get only the failure , but i want to try reach out to tivoli via webhook connectors for rerunning the job. How to do that?

2

You would need to configure your Webhook connector with the endpoint and credentials and then construct a payload in the alert action, this is explained in the documentation.

Also, do you have a paid license? The webhook connector requires a paid license.

3

Hi,

Most probably you can use a watcher too.
I am now aware of licensing issues with it.

{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "xxxxxxxxxxxxx*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-30m"
                    }
                  }
                }
              ]
            }
          },
          "aggs": {
            "hostnames": {
              "terms": {
                "field": "observer.hostname",
                "size": 100
              },
              "aggs": {
                "docs_per_5m": {
                  "date_range": {
                    "field": "@timestamp",
                    "ranges": [
                      {
                        "to": "now"
                      },
                      {
                        "from": "now-5m"
                      }
                    ]
                  },
                  "aggs": {
                    "last_bucket": {
                      "bucket_sort": {
                        "sort": [
                          {
                            "_key": {
                              "order": "desc"
                            }
                          }
                        ],
                        "size": 1,
                        "gap_policy": "insert_zeros"
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "script": {
      "source": """
                  for (def host : ctx.payload.aggregations.hostnames.buckets) {
                    for (def buckt_docs : host.docs_per_5m.buckets) {
                      if (buckt_docs.doc_count == 0) {
                        return true;
                      }
                    }
                  }
                  """,
      "lang": "painless"
    }
  },
  "actions": {
    "slack_web_hook": {
      "transform": {
        "script": {
          "source": """
                              def failed_hosts = new ArrayList();
                              for (def host : ctx.payload.aggregations.hostnames.buckets) {
                                for (def buckt_docs : host.docs_per_5m.buckets) {
                                  if (buckt_docs.doc_count == 0) {
                                    def failed_host = new HashMap();
                                    failed_host.put("host",host.key);
                                    failed_host.put("doc_count",buckt_docs.doc_count);
                                    failed_host.put("date",buckt_docs.from_as_string);
                                    failed_hosts.add(failed_host);
                                  }
                                }
                              }
                              return failed_hosts;""",
          "lang": "painless"
        }
      },
      "webhook": {
        "scheme": "https",
        "host": "hooks.slack.com",
        "port": 443,
        "method": "post",
        "path": "services/.................",
        "params": {},
        "headers": {
          "Content-Type": "application/json"
        },
        "body": """{"channel": "#abc-def-pqrst", "username": "webhookbot", "text": "{{#ctx.payload._value}}HaProxy ingest issue {{host}} has {{doc_count}} documents at {{date}}\n!{{/ctx.payload._value}}"}"""
      }
    }
  }
}

Above a watcher with a webhook to a slack channel.
It checks the number of documents ingested per observer.hostname.
If it is 0 then the webhook is executed.

Regards Hans

4

Hi HansPeterSloot, Thanks for the reply, mine is just a simple check - index - mdss_lineage-mdss_tivoli_job* , log.message contains 'Procedure error' - it needs action - webhook connector to tivoli master to rerun the respective job. for the inputs i would like to provide JobName, JobID, Workstation as parameter to the URL - which TWS Admins are working to get. Also within Kibana UI - Webhook connector trying to connect and test shows the following error.

Test failed to run

The following error was found:
error calling webhook, request failed
Details:
[DEPTH_ZERO_SELF_SIGNED_CERT] self signed certificate

5

Hello Mohammed

According to https://discuss.elastic.co/t/webhook-https-ignoring-verification-of-certificate/25874/4
you should look at https://www.elastic.co/guide/en/elasticsearch/reference/current/notification-settings.html#ssl-notification-settings

Regards Hans

6

Will Give it a try and for now - what we did was to include the host into yml - now at the Kibana level when given the webhook connector, with body as { } and run shows as "Test was successful", but then no flow seen to the destination host where the webhook should connect. - Not exactly sure how to move further to this in order to check if an connection calls or any infor was sent from ELK . based on the connector run button.

7

What I described is a watcher and not a connector.
Those are 2 different things.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.