I appreciate your help on this.
I also considered the possibility that when changing versions, the beats map differently. On this server I have had versions
7.8, 7.9 and finally
7.9.1. This is why I removed all indexes, templates, and ILM rules linked to each beat. Then I used the
setup command, to force each beat to recreate its indexes, templates and ILM rules. I was looking for a fresh start.
Basically I did the following:
- Stopped the beats
- Removed indexes in Elasticsearch from Index Management
- Removed Legacy index templates from Index Management
- Removed ILM Rules from Index Lifecycle Policies
- Removed Index Templates from Index Patterns
- Uninstalled all beats
- Installed all beats without starting the service
- Started just one beat in one server (I think I started with Auditbeat)
- Started service
- Checked beat logs (no connection nor data shipping errors)
- Confirmed creation of index and index patterns in Elasticsearch
- Checked Discovery and confirmed all data was arriving
I am using Firefox and when inspecting any of the tables below the map in Security/Network or Security/Hosts, I only see a bunch of "
GET" requesting many files to load the page (mostly
.js), but I don't see any errors (or maybe I'm not searching correctly).
As for the indexes patterns, I removed all old indexes and patterns (unless I have missed a system index). I'd like to avoid (if possible) making specific modifications like telling
Security to use other indexes than the generic ones that come by default. I love that ELK works well with most of the default settings. Otherwise I would have to be remembering to update
securitySolution: defaultIndex every time I update ELK or the beats.
I think that a cleaner and more durable solution would be to be able to delete the system indexes that
Security uses (if any, like
.siem-signals-default) to store the data that it then shows. I don't know if what I just said makes much sense.
As for the geolocation, I have a lot of events in
Discovery (from today) including the following fields:
If you need more details or if I did not provide all the information you requested, please let me know. I also ask for your patience and understanding if at times I don't quite understand what you are asking.