(ELK 7.9.1) Security - Hosts and Security - Network missing data

Elastic Security SIEM
6

Hi @ManuelF,

I have seen these before in my custom dashboards and I have been able to fix them by replacing field.data by field.data.keyword

I would be very careful about changes like that and here's why. What might be happening is that you are ingesting data very quickly right as you are deleting your mappings and then Elastic Search is auto creating your indexes by "guessing" at them before you had a chance to run the beats setup:

What you dont want really is ES taking a guess at the indexes before you had a chance to run "setup" from the beats. When that happens because your data ingest is not turned off when you're deleting beat mappings ES is going to make naive and ambiguous choices that will not work out well for most cases. Good examples will be that you would start to notice things like source.ip is now a keyword and text data type rather than an ip data type which is then not going to work with CIDR patterns from detection rules.

I put some links to other times we have helped people out, but I would ensure that your mappings match that of beats and reindex if they do not. If you do delete them, I would ensure you turn off existing beats first so you don't get auto-created indexes or you can set ES to tell it to not auto-create index mappings for a brief moment of time and get dropped messages for a few seconds while you upgrade to your next set of mappings.

fwiw, you can do a manual export of a beats mapping as well to compare them to your current mappings that might or might not be auto-created:

1 Like