# Transmitting Kibana data through watchers

**URL:** <https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199>\
**Category:** Kibana\
**Tags:** elastic-stack-alerting\
**Created:** [September 12, 2022, 1:56pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199 "2022-09-12T13:56:18Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nalin](https://avatars.discourse-cdn.com/v4/letter/n/ecd19e/32.png) [@Nalin](https://discuss.elastic.co/u/Nalin)\
**Post date:** [September 12, 2022, 1:56pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/1 "2022-09-12T13:56:18Z")

</div>

Hi All,

We are trying to run the API call for "GET /\_cat/indices" through a watcher , but we are not able to receive the data .

Could anyone please help us with the correct script for running the above GET command through the watcher?

Thanks and Regards,  
Nalin Anand

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [September 15, 2022, 2:45pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/2 "2022-09-15T14:45:47Z")

</div>

Example: [compare\_shard\_primary\_and\_replica · GitHub](https://gist.github.com/richcollier/5643e649a2816ed317d0caf3917263b8)

---

<div class="post-metadata">

**Author:** ![Nalin](https://avatars.discourse-cdn.com/v4/letter/n/ecd19e/32.png) [@Nalin](https://discuss.elastic.co/u/Nalin)\
**Post date:** [September 20, 2022, 1:59pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/3 "2022-09-20T13:59:08Z")

</div>

Hi Rich,

The script that is mentioned in the GitHub link is regarding violators count in shards.  
Is there anyway in which we could get the output of "GET /\_cat/indices" in our mails directly through the watcher?

Regards,  
Nalin

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [September 21, 2022, 2:03pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/4 "2022-09-21T14:03:58Z")

</div>

Of course, this was just a representative example of using watcher to hit an HTTP endpoint and format the results. If you want to change the endpoint, then just do that. If you don't want to format the results, you can remove the `transform` sections. The bare-bones example of this would look something like:

```auto
POST _watcher/watch/_execute
{
  "watch": {
    "trigger": {
      "schedule": {
        "interval": "1d"
      }
    },
    "input": {
      "http": {
        "request": {
          "host": "mycluster-name.es.us-central1.gcp.cloud.es.io",
          "port": 9243,
          "scheme": "https",
          "path": "/_cat/indices",
          "params": {
            "format": "json",
            "bytes": "b",
            "human": "true"
          },
          "headers": {},
          "auth": {
            "basic": {
              "username": "elastic",
              "password": "xxx"
            }
          }
        }
      }
    },
    "condition": {
      "script": "return true;"
    },
    "actions": {
      "displayResults": {
        "logging": {
          "text": "the output is: {{ctx.payload}}"
        }
      }
    }
  }
}

```

And the unformatted output would look something like:

```auto
"the output is: {_headers={content-type=[application/json], x-found-handling-cluster=[71bbfab0891a4b02acf6c5a5ff286fd9], x-cloud-request-id=[tPrexYvzRdigrVx5hg3gOw], x-elastic-product=[Elasticsearch], date=[Wed, 21 Sep 2022 13:59:42 GMT], x-found-handling-instance=[instance-0000000000]}, data=[{health=green, status=open, index=kibana_sample_data_logs, uuid=iJw6Xf-JQ5mu_vXPge3y2A, pri=1, rep=0, docs.count=14074, docs.deleted=0, store.size=8778892, pri.store.size=8778892}, {health=yellow, status=open, index=kibana_sample_data_logs_rollup, uuid=b3LG3Gy7Sdm07MyGn1uEbA, pri=1, rep=1, docs.count=1912, docs.deleted=0, store.size=659460, pri.store.size=659460}], _status_code=200}"

```

---

<div class="post-metadata">

**Author:** ![Nalin](https://avatars.discourse-cdn.com/v4/letter/n/ecd19e/32.png) [@Nalin](https://discuss.elastic.co/u/Nalin)\
**Post date:** [September 22, 2022, 12:42pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/5 "2022-09-22T12:42:13Z")

</div>

Hi Rich,

Thanks for the reply.  
We were trying a similar code for the past few days, but getting some other error.  
Now after using your code, we are getting another error:  
**could not parse [http] input for watch [ag4c694d-e6e5-4687-a2c7-a4002673329c]. failed to parse http request template**

When we tried using **schema: "http"** , we are getting below output :

**\_value=Client sent an HTTP request to an HTTPS server.\n, \_status\_code=400**

Could you please suggest that the **host** should be the Elasticsearch endpoint or Kibana endpoint?  
And the **username** should be the Kibana user or Elasticsearch user?

The watcher that we are using currently is below:

{  
"trigger": {  
"schedule": {  
"interval": "1d"  
}  
},  
"input": {  
"http": {  
"request": {  
"scheme": "https",  
"host": "??????",  
"port": 9244,  
"method": "get",  
"path": "/\_cat/indices",  
"params": {  
"format": "json",  
"human": "true",  
"bytes": "b"  
},  
"headers": {},  
"auth": {  
"basic": {  
"username": "Elastic",  
"password": "::es\_redacted::"  
}  
}  
}  
}  
},  
"condition": {  
"script": {  
"source": "return true;",  
"lang": "painless"  
}  
},  
"actions": {  
"displayResults": {  
"logging": {  
"level": "info",  
"text": "the output is: {{ctx.payload}}"  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [September 23, 2022, 1:29pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/6 "2022-09-23T13:29:14Z")

</div>

1. You need to specify the elasticsearch endpoint
2. The user to use should be an elasticsearch user with cluster management privileges (there really is no such thing as a pure "Kibana" user - all users are really elasticsearch users)

---

<div class="post-metadata">

**Author:** ![Nalin](https://avatars.discourse-cdn.com/v4/letter/n/ecd19e/32.png) [@Nalin](https://discuss.elastic.co/u/Nalin)\
**Post date:** [September 28, 2022, 4:25pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/7 "2022-09-28T16:25:01Z")

</div>

Thank you so much for the response, Rich.

Currently, we are getting authentication issue with the elasticsearch endpoint and trying to sort out the User privilege issue.

---

<div class="post-metadata">

**Author:** ![Nalin](https://avatars.discourse-cdn.com/v4/letter/n/ecd19e/32.png) [@Nalin](https://discuss.elastic.co/u/Nalin)\
**Post date:** [October 13, 2022, 7:32am UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/8 "2022-10-13T07:32:48Z")

</div>

Hi Rich,

We are able to get the output as an attachment in the below format:

ctx:  
metadata:  
name: "Watcher\_Test"  
xpack:  
type: "json"  
watch\_id: "bf2c4567d-e6e5-4687-b2d7-##########"  
payload:  
index:  
- health: "green"  
status: "open"  
index: "**"  
uuid: "**\*\*\*\*\*\*\*\*"  
pri: "1"  
rep: "1"  
docs.count: "487"  
docs.deleted: "0"  
store.size: "0"  
pri.store.size: "0"

_Is there any way that we could print only the index, store.size and pri.store.size in the body of the mail itself?_

Watcher:  
{  
"trigger": {  
"schedule": {  
"interval": "2m"  
}  
},  
"input": {  
"http": {  
"request": {  
"scheme": "https",  
"host": "\<\>",  
"port": 9243,  
"method": "get",  
"path": "/\_cat/indices",  
"params": {  
"format": "yaml",  
"human": "true",  
"bytes": "mb"  
},  
"headers": {},  
"auth": {  
"basic": {  
"username": "elastic",  
"password": "\*\*"  
}  
}  
}  
}  
},  
"condition": {  
"script": {  
"source": "return true;",  
"lang": "painless"  
}  
},  
"actions": {  
"displayResults": {  
"logging": {  
"level": "info",  
"text": "the output is: {{ctx.payload.data}}"  
}  
},  
"email\_administrator": {  
"email": {  
"profile": "standard",  
"attachments": {  
"index.yml": {  
"data": {  
"format": "yaml"  
}  
}  
},  
"to": [  
"Nalin.27\*@gmail.com"  
],  
"subject": "Watcher Notification",  
"body": {  
"text": "{{ctx.payload.data}}"  
}  
}  
}

}

---

<div class="post-metadata">

**Author:** ![Nalin](https://avatars.discourse-cdn.com/v4/letter/n/ecd19e/32.png) [@Nalin](https://discuss.elastic.co/u/Nalin)\
**Post date:** [October 18, 2022, 10:33am UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/9 "2022-10-18T10:33:25Z")

</div>

Hi @richcollier ,

Could you please help on the above query?

Regards,  
Nalin

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [October 24, 2022, 3:16pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/10 "2022-10-24T15:16:29Z")

</div>

Just change the actions section to include a `transform` to select only the things you want to print. Something like:

```auto
      "displayResults": {
        "transform": {
          "script": """
          def index_data = new ArrayList();
          for (def index : ctx.payload.data) {
            index_data.add("index=" + index['index'] + " :: pri.store.size=" + index['pri.store.size'] + " :: store.size=" + index['store.size']);
          }
          return index_data;
          """
        },
        "logging": {
          "text": "{{ctx.payload}}"
        }
      }
    }

```

Which would yield something similar to:

```auto
"{_value=[index=kibana_sample_data_logs :: pri.store.size=9054433 :: store.size=18046404, index=mydata-2022.01.01 :: pri.store.size=4253 :: store.size=8506]}"

```

---

<div class="post-metadata">

**Author:** ![Nalin](https://avatars.discourse-cdn.com/v4/letter/n/ecd19e/32.png) [@Nalin](https://discuss.elastic.co/u/Nalin)\
**Post date:** [October 25, 2022, 8:19am UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/11 "2022-10-25T08:19:23Z")

</div>

Thank you so much for the response, Rich.  
We were able to produce the data in a tabular form.

Regards,  
Nalin

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 20, 2022, 9:01pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/12 "2022-11-20T21:01:53Z")

</div>



---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 18, 2022, 9:02pm UTC](https://discuss.elastic.co/t/transmitting-kibana-data-through-watchers/314199/13 "2022-12-18T21:02:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.