Hello guys, I am getting the error below when I go to the security dashboard overview:
Error fetching fields for data view .alerts-security.alerts-default,apm--transaction,auditbeat-,endgame-,filebeat-,logs-,packetbeat-,traces-apm,winlogbeat-*,-elastic-cloud-logs- (ID: security-solution-default)
When I look at: stack management > Kibana > Advanced Settings > Security Solution > Elasticsearch indices, I only have these indices:
If I change (for example, I was trying to include .alerts-security.alerts-default), the error persists, and I don't have this index in my cluster, what can I do to fix this? All my rules in SIEM are getting this error too (I believe is because of the index).
Kibana logs:
"message":"Changing rule status to "partial failure". This rule is attempting to query data from Elasticsearch indices listed in the "Index pattern" section of the rule definition, however no index matching: ["logs-endpoint.alerts-*"] was found. This warning will continue to appear until a matching index is created or this rule is disabled. If you have recently enrolled agents enabled with Endpoint Security through Fleet, this warning should stop once an alert is sent from an agent.
Hi @GustavoPires, apologies for the delay in response. If you are running rules, you should have the index .alerts-security.alerts-default. It only gets added to the security solution data view if it exists, whether or not it is included in the stack management > Kibana > Advanced Settings > Security Solution > Elasticsearch indices. So this tells me maybe your rules aren't running, or if they are they are not detecting any alerts to write to the index. Could you please ensure you have the index and/or that your rules are creating alerts?
Hi @stephmilovic, I have added this index to my Elasticsearch indices, but this error persists, for the test, I have enabled all rules of Elastic, but, without alerts.
I think creating the index yourself vs having the SIEM create it may be the problem. Go ahead and delete the index. Then create and run a rule that detects on something simple that will for sure trigger an alert, for example host.name:*. Let's get some alerts going
@stephmilovic I searched for .alerts-security.alerts-default on dev tools, but I only have this index: .internal.alerts-security.alerts-default-000001. I tried creating with PUT the index .alerts-security.alerts-default but appear this message: "this index already exists". So If I delete the index .alerts-security.alerts-default, do I need to recreate them, right?
Observation:I recreate a new environment with Elasticsearch, and the same message ("...Error fetching fields for data view...") appears. My SIEM rules are OK now, but when I access security dashboard the message persists.
Like @stephmilovic mentioned - the alerts index is create automatically when the very first alert is created. The .internal.alerts-security is the concrete index, and .alerts-security is the alias used. But like Steph said - these are internally managed indices. Trying to create them on your own can definitely cause issues as it may not pick up the right template, etc.
Are all the rules you currently have using data views or are any using index patterns? If you go to rule details you will see that it either says Index pattern: ... or Data view: .... If you could create a rule that specifies an index you know has data you can hit with a very generic query to try to generate an alert like Steph suggested, that should create the alerts index and then maybe this data view error will stop.
Hi @yctercero, I believe is 8.4.1, when I go to Stack Management is 8.4.1 there, and I installed it via apt (Ubuntu Server).
Ok, understood, I won't try to create an index again, because of this I reinstalled all my Elastic Stack environments.
I am using prebuilt rules of the Elastic Stack, with these tags: APM, Endpoint Security, and Linux. When I go to the top of the rule (while I am editing the rules), precisely at "Definition", I can see "Index patterns" at these rules.
Ok, so I created two rules for querying a name host (host.name:, like @stephmilovic said), using the data view and index patterns. The rule is working, I can get the name of my machine through these two queries, but the error persists. When I created a rule using index pattern, I did not see .alerts-security.alerts-default , do I have to put it there?
Unless you want to be alerting on alert events, you do not want the alerts index selected here. This could result in redundant alerts being generated from existing alerts.
Do you get the same error when you go to Stack Management > Kibana > Data Views and click into the security data view? The url: http://your:instance/app/management/kibana/dataViews/dataView/security-solution-default#/?_a=(tab:indexedFields)
What happens when you get that error and click "See the full error", can you please screenshot that?
Ok here is a debugging idea. From the Security Data View Details view, (again http://your:instance/app/management/kibana/dataViews/dataView/security-solution-default#/?_a=(tab:indexedFields)), please hit the "Edit" button.
My goal is to narrow down which index the error is coming from. I'm asking the team for help with what to do with the field error once we identify the index
EDIT: Do not navigate to the Security Solution during this exercise as this will reset the index pattern
At Dataview I'm trying to edit but it's not saving (I save the edit, but nothing changes on the Index pattern). Do I have to perform a specific action to do the HAR file?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
.
