Endgame Rules in cross cluster search

[Rainiur (David Walden)]

I am trying to create a rule using the endgame metadata in a cross cluster search. Unfortunately I get the following error.

This rule is attempting to query data from Elasticsearch indices listed in the "Index pattern" section of the rule definition, however no index matching: [":endgame-"] was found. This warning will continue to appear until a matching index is created or this rule is de-activated.

Could anyone explain why this is happening and what I can do to fix it?

[spong (Garrett)]

Hey there @Rainiur!

So this warning can occur with remote indices (CCS) even when you have all appropriate privileges as there isn't currently an API for us to determine if a user has read privileges for all specified CCS indices.

This behavior is outlined in this issue:

As a byproduct of elasticsearch/#67798, since there is currently no way to verify if a user has read privileges across all clusters for a CCS index pattern like *:winlogbeat-*, Detection Rules using CCS index patterns will display the following warning even if the user/Rule API Key has read privileges for all matching indices:

This rule may not have the required read privileges to the following indices: ["*:winlogbeat-*"]

If the user has ensured their roles contain all the required read privileges across their clusters, this warning can be noisy or cause unnecessary confusion. In discussion with @peluja1012 & @rylnd, in the interim while elasticsearch/#67798 is worked, we could address this confusion by allowing users to choose whether or not these permission checks are performed via a Check CCS privileges during rule execution checkbox or some similar configuration on the rule.

So if you've ensured the user that has created/modified this rule has the appropriate CCS privileges, it is safe to ignore this message, and you can follow the issue above for future UX enhancements around this.

You may want to verify your CCS index is correct though, as :endgame- will not match all your CCS indices. If you want to query all clusters with that index, you'll want to add a leading wildcard ala: *:endgame-. Docs here.

Hope this helps!
Garrett