# Built in alerts fail to run

**URL:** <https://discuss.elastic.co/t/built-in-alerts-fail-to-run/281207>\
**Category:** Kibana\
**Tags:** elastic-stack-alerting\
**Created:** [August 12, 2021, 1:19pm UTC](https://discuss.elastic.co/t/built-in-alerts-fail-to-run/281207 "2021-08-12T13:19:21Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![radovan](https://avatars.discourse-cdn.com/v4/letter/r/2acd7d/32.png) [@radovan](https://discuss.elastic.co/u/radovan)\
**Post date:** [August 12, 2021, 1:19pm UTC](https://discuss.elastic.co/t/built-in-alerts-fail-to-run/281207/1 "2021-08-12T13:19:22Z")

</div>

Hi,

I decided to create a monitoring cluster and after a few problems at the start all is running now. There are metricbeat clients on the elastic nodes with only elastic/logstash/kibana modules enabled, logs are flowing onto the monitoring cluster to .monitoring-XXX indexes, but built in monitoring rules fail to run. Do I need to do some initialization first or something I need to setup? All of the 14 alerts says only this, when I open them:

```auto
An error occurred when running the rule.
illegal_argument_exception, caused by: ""

```

The version of production and monitoring elastic stack is 7.13.3. Monitoring cluster has 2 nodes, first one is master and second one is ingest/data node.

---

<div class="post-metadata">

**Author:** ![gbeltramelli](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@gbeltramelli](https://discuss.elastic.co/u/gbeltramelli)\
**Post date:** [August 17, 2021, 5:31pm UTC](https://discuss.elastic.co/t/built-in-alerts-fail-to-run/281207/2 "2021-08-17T17:31:16Z")

</div>

same problem!

---

<div class="post-metadata">

**Author:** ![monfera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/monfera/32/26467_2.png) [@monfera](https://discuss.elastic.co/u/monfera)\
**Post date:** [August 20, 2021, 1:41pm UTC](https://discuss.elastic.co/t/built-in-alerts-fail-to-run/281207/3 "2021-08-20T13:41:45Z")

</div>

This may be a problem in Kibana Alerting rules, as by default, it enables all rules associated with remote clusters used in CCR/CCS. If it's the case, all alerting rules created manually should continue to work fine.

Here's a possible fix/workaround:

For 7.7+, add the following in kibana.yml of the "monitoring" cluster:

```auto
monitoring.ui.ccs.enabled: false

```

or add the `remote_cluster_client` role to all of their nodes.

---

<div class="post-metadata">

**Author:** ![radovan](https://avatars.discourse-cdn.com/v4/letter/r/2acd7d/32.png) [@radovan](https://discuss.elastic.co/u/radovan)\
**Post date:** [August 20, 2021, 2:31pm UTC](https://discuss.elastic.co/t/built-in-alerts-fail-to-run/281207/4 "2021-08-20T14:31:30Z")

</div>

I did both, first the kibana setting and after restarting kibana, these 4 remained in error state with same error message:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/6/367941f804da7b37224d73c744e062a8fa38b896.png)

Then I added `remote_cluster_client` role to my 2 nodes that are part of the monitoring cluster, restarted them but the errors still occur.

---

<div class="post-metadata">

**Author:** ![monfera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/monfera/32/26467_2.png) [@monfera](https://discuss.elastic.co/u/monfera)\
**Post date:** [August 20, 2021, 2:48pm UTC](https://discuss.elastic.co/t/built-in-alerts-fail-to-run/281207/5 "2021-08-20T14:48:17Z")

</div>

It's a bug to begin with, and apparently the workarounds didn't work out. We use github for bug reporting. Would you mind putting your specific details into a comment, if you feel it's appropriate for [[Monitoring] `remote_cluster_client` role shouldn't be required to use Monitoring · Issue #93432 · elastic/kibana · GitHub](https://github.com/elastic/kibana/issues/93432) or if not, could you please file a new ticket and link here too?

---

<div class="post-metadata">

**Author:** ![radovan](https://avatars.discourse-cdn.com/v4/letter/r/2acd7d/32.png) [@radovan](https://discuss.elastic.co/u/radovan)\
**Post date:** [August 23, 2021, 8:16am UTC](https://discuss.elastic.co/t/built-in-alerts-fail-to-run/281207/6 "2021-08-23T08:16:03Z")

</div>

Don't know what happened, I just let it be during the weekend and now even the last 4 alerts seem they have no problem now. So my issue is probably fixed, thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 20, 2021, 8:16am UTC](https://discuss.elastic.co/t/built-in-alerts-fail-to-run/281207/7 "2021-09-20T08:16:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
