Error with Security Rules

[efadeev]

I got an error with Security Rules in my Kibana. Both custom and pre-built rules. " An error occurred when reading the rule. Unauthorized to get a "siem.queryRule" rule for "siem""
Rules added under elastic user

[jamesspi]

Hi @efadeev - could you share rule definition for this rule please?

What version of the stack is this?

Lastly - has this always happened, or did it start suddenly?

Thanks,
James

[spong (Garrett)]

In addition to @jamesspi's questions, this appears to be the Rule Management UI from within Stack Management -> Rules. If you navigate to the dedicated Security Solution UI for Rule Management ( route: app/security/rules), do you see any errors there?

If this occurred post-upgrade to 8.0, even though you're the elastic user, please make sure the privilege pre-reqs have been completed (ensuring the Security Space privileges are All).

[efadeev]

@jamesspi I use Elastic Stack 8.1.2, it happens always with every rule after creation. For example I use Elastic Prebuilt Rule "Hosts File Modified"
@spong from app/security/rules I don't see neither errrors nor alerts (but can see alerts in preview when create the rule). Also can't disable or enable rule, see error: Failed to fetch. But I can enable/disable rule from Stack Management.
It is brand new installation of 8.1.2, not upgrade. Elastic is superuser, has all privilegies.

[spong (Garrett)]

Interesting @efadeev, everything should work out of the box with the elastic user and superuser role, so would be worth it to see if we can get some additional details from the Kibana logs here.

As you can see from this test, that specific error is bubbled up from the RuleClient when the user isn't authorized to view a specific alert type for the given app (siem being the Security Solution app in this context).

Would you be willing to try configuring a dedicated user as per the previous privilege docs and see if that user has the same issues?

Additionally, can you speak more to your deployment configuration -- are you running multiple kibana instances? Is this all within the default space? Does this happen with all Rule Types within Stack Management or only Security Rules? Would be interested to see if you could create non-security rules without error.

Thanks!
Garrett

[efadeev]

1. I've tried to create security rule under specific user due to this docs but got the same issue.
2. It's single node kibana.
3. Have tried to create rules in specific space. In Default space got same issue
4. Got an issue in monitoring rules too
   

Also notices an intersting detail: when I export rule to json file I see that it was created and updated by "kibana" user (but I created it under elastic or specific security user)

"updated_at":"2022-05-20T09:07:11.373Z","updated_by":"kibana","created_at":"2022-05-20T09:07:09.443Z","created_by":"kibana"

[spong (Garrett)]

Thanks for the additional infor @efadeev! So this is looking like it could potentially be a config issue -- would you be willing to share your redacted kibana.yml and also kibana logs from both the time of Rule creation and Rule reading?

That should give us enough information to further debug

[efadeev]

kibana.yml:

server.port: 5601
server.host: "0.0.0.0"
server.name: kibana.example.com
server.publicBaseUrl: https://kibana.example.com
elasticsearch.hosts:
 - https://elk-proxy01.example.com:9200
elasticsearch.pingTimeout: 5000
elasticsearch.requestTimeout: 60000
elasticsearch.requestHeadersWhitelist: []
elasticsearch.customHeaders: {
  "Authorization": "Basic xxxxxxxxxxxxxxxx=="
}
elasticsearch.shardTimeout: 60000
elasticsearch.ssl.certificateAuthorities: [ "/opt/kibana/ssl/elasticsearch-ca.pem" ]
elasticsearch.ssl.verificationMode: certificate
xpack.encryptedSavedObjects.encryptionKey: xxxxxxxxxxxxxxxxxxxxxx
monitoring.ui.ccs.enabled: false

kibana logs (1st line is after creating sec rule):

May 23 10:11:18 kibana kibana[38095]: [2022-05-23T10:11:18.369+03:00][ERROR][plugins.alerting] Executing Rule stable01:siem.eqlRule:87754920-da67-11ec-9965-ffb08f0a6ec9 has resulted in Error: Unauthorized to get a "siem.eqlRule" rule for "siem"
May 23 10:11:48 kibana kibana[38095]: [2022-05-23T10:11:48.351+03:00][ERROR][plugins.alerting] Executing Rule prod01:monitoring_alert_nodes_changed:4fae9f00-bc12-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_alert_nodes_changed" rule for "monitoring"
May 23 10:11:48 kibana kibana[38095]: [2022-05-23T10:11:48.372+03:00][ERROR][plugins.alerting] Executing Rule prod01:monitoring_alert_cluster_health:4fac06f0-bc12-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_alert_cluster_health" rule for "monitoring"
May 23 10:11:48 kibana kibana[38095]: [2022-05-23T10:11:48.374+03:00][ERROR][plugins.alerting] Executing Rule prod01:monitoring_alert_cpu_usage:4fab6ab0-bc12-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_alert_cpu_usage" rule for "monitoring"
May 23 10:11:48 kibana kibana[38095]: [2022-05-23T10:11:48.376+03:00][ERROR][plugins.alerting] Executing Rule prod01:monitoring_alert_missing_monitoring_data:4fadb4a0-bc12-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_alert_missing_monitoring_data" rule for "monitoring"
May 23 10:11:48 kibana kibana[38095]: [2022-05-23T10:11:48.377+03:00][ERROR][plugins.alerting] Executing Rule prod01:monitoring_shard_size:4fabdfe0-bc12-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_shard_size" rule for "monitoring"
May 23 10:11:48 kibana kibana[38095]: [2022-05-23T10:11:48.379+03:00][ERROR][plugins.alerting] Executing Rule prod01:monitoring_alert_jvm_memory_usage:4fad8d90-bc12-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_alert_jvm_memory_usage" rule for "monitoring"
May 23 10:11:48 kibana kibana[38095]: [2022-05-23T10:11:48.380+03:00][ERROR][plugins.alerting] Executing Rule prod01:monitoring_alert_thread_pool_search_rejections:4faeed20-bc12-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_alert_thread_pool_search_rejections" rule for "monitoring"
May 23 10:11:54 kibana kibana[38095]: [2022-05-23T10:11:54.317+03:00][ERROR][plugins.alerting] Executing Rule default:monitoring_shard_size:59040e80-b8cc-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_shard_size" rule for "monitoring"
May 23 10:11:54 kibana kibana[38095]: [2022-05-23T10:11:54.346+03:00][ERROR][plugins.alerting] Executing Rule default:monitoring_alert_cluster_health:59045ca0-b8cc-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_alert_cluster_health" rule for "monitoring"
May 23 10:11:54 kibana kibana[38095]: [2022-05-23T10:11:54.349+03:00][ERROR][plugins.alerting] Executing Rule default:monitoring_alert_thread_pool_write_rejections:5902fd10-b8cc-11ec-aab1-c3343dc0ff5c has resulted in Error: Unauthorized to get a "monitoring_alert_thread_pool_write_rejections" rule for "monitoring"

[spong (Garrett)]

Thanks @efadeev!

So this configuration looks quite suspect:

elasticsearch.customHeaders: {
  "Authorization": "Basic xxxxxxxxxxxxxxxx=="
}

Could you speak to what you're using this configuration to accomplish? Also, what account is the auth value for, the kibana user you're seeing those SO's being updated_by perhaps?

When testing locally with that config it I'm seeing security_exceptions on startup with regards to auth of the user, so I'm thinking this is probably your issue (these rules being created as this user and lacking the correct privileges). Could you verify if things function without this config?

We're getting close -- thanks for hanging in there @efadeev!