Hi everyone,
I am stuck with an issue regarding fielddata aggregation. I spent many days looking for similar issues, but none of them worked.
In a nutshell, I use ELK stack (version 7.3.1) for personal purpose and want to exploit Dashboard & SIEM utilities.
My architecture is the following (all based on Linux OS):
[subnet 1 : various beats on every available machines a own -> kafka] -> Personnal Routeur -> [Subnet 2 : Logstash -> Elasticsearch <- Kibana]
I configured Xpack free functionalities for encryption/Keystores... and I managed to centralize the logs on the Elasticsearch. Then on Kibana interface I created index paterns based on indexes available on the index management page. I used logstash to parse the logs that are not already parsed by default. When I go to the Discover page, everything runs fine. I have all my indexes, and I can search using advanced filters.
The problem is : I have issues when I want to use the SIEM app, only the main overview counts Hosts Events & Network Events. When I click on one of them, nothing displays and I have the following error : [illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [host.name]
As I understand, Logstash has something that alters data type. I firstly checked my Logstash file configuration. Here is the end of it :
output {
elasticsearch {
hosts => "localhost:9200"
**manage_template => "false"**
index => "%{[@metadata][kafka][lc_topic]}-%{+YYYY.MM.dd}"
user => "${ES_USER}"
password => "${ES_PWD}"
}
}
I though it was because I didn't correctly uploaded / install the beats templates. So I did it again, following those steps :
- I deleted all indexes with the Kibana interface
- On each machine :
< beat_name> export template > <beat_name>.template.json
- Checked they were populated, and uploaded them on the ELK server (scp)
- Used the command ```curl -XPUT -H 'Content-Type: application/json'
http://localhost:9200/_template/<beat_name-version> -d@<beat_name>.template.json
5) As the issue was not resolved, I restarted ELK services. No improvement.
I hope I was clear, feel free to ask any further technical information.
Thank you in advance for your help.