Hello,
I am ingesting my PFSense logs and net flow using Filebeat.
Filebeat feeds LogStash and it does the enrichment with select parts of the code from there:
It works pretty well, each data type in its own index.
Netflow data (filebeat net flow) to filebeat-*
PFsense logs to pf-* (so should not be take into account by the SIEM yet)
However, going to the "network" or "host" tab of the SIEM always returns:
"Data Fetch Failure"
The complete message is (or a variation with host.name):
[illegal_argument_exception] Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [source.ip] in order to load field data by uninverting the inverted index. Note that this can use significant memory.
For a reason, it appears that my fields are created as multi-fields with source.ip being text and source.ip.keyword being ... keyword.
I understand that I need source.ip to be keyword for the SIEM.
How can I change this default behavior?
Thanks in advance,