Hello All, I am setting up an ELK stack and auditbeat and filebeat are sending logs to logstash and ingesting them correctly in Elasticsearch. When I go to SIEM, an error is thrown in the host and network section that host.hostname, and process.name are incorrect and can't be displayed because field_data isn't set to true. Of course, you don't want to use a ton of memory, so it's best for these fields to be keywords.
I have done some research on other discussions on this topic and followed the steps needed. It is reported that Logstash converts or makes these fields into text fields and then aggregations can't be created. So, host.hostname should be a keyword. I have taken all the steps to solve this, but I am still having the issue. Here are the details and steps I have taken:
Auditbeat version: 7.4.0
Filebeat version: 7.4.2
-
The Logstash output for config file was set to:
mangage_template => false -
The logstash index for the output was set to:
index => "%{[@metadata][beat]}-%{[@metadata][version]}" -
From other posts, I followed the advice to run auditbeat setup and rebuild the index templates so the can be loaded manually. The curl command was used to output the index template from Auditbeat to JSON and Filebat like so:
auditbeat export template > auditbeat.template.json -
I then uploaded the template to elasticsearch via curl successfully.
-
I restarted logstash and services, deleted index patterns and indices in Kibana and it now has the new indices, index-patterns, and index templates.
-
No dice. Still SIEM error.
- In ES, if I do this I just see Filebeat stuff, no Audibeat pipeline, don't know if that means something
GET /_ingest/pipeline
So, at this point, don't know what to try next. One thing I notice in looking at the index templates, there is a host.name field, and a host.name.keyword field. Not sure what is going on with this. Any help would be appreciated.