SIEM Hosts/All Hosts Tables Empty

[Dan_Sputnikk]

Hi all,

Configured filebeat 7.9 against ES/Kibana 7.9 and the netflow + cisco modules successfully and without errors, including during template generation.

http s://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-cisco.html
http s://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-netflow.html

However lots of SIEM dashboard remain unpopulated, in particular the "hosts" tab. Reading tells me it depends on host.name being in the documents. My index template contains it, but no document have that field set, even when forcefully adding the host preprocessor. Other host.*'s are populated, such as host.hostname - but they are populated with the hostname of the filebeat ingestor, not its source (an actual firewall sending syslog and netflow).

Looking at filebeat docs 6.8 vs 7.9 however:

http s://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-host-processor.html

vs

http s://www.elastic.co/guide/en/beats/filebeat/6.8/exported-fields-host-processor.html

7.9 does not show host.name as an exported field, whereas 6.8 does. I tried 6.8, but it does not contain the cisco module.

Docs were followed to the letter and even the getting started documentation for 7.9 seems to imply it should work out of the box with the provided example dashboards.

Any ideas?

Edit: just noticed the placeholder in filebeat.yml to not attach host_preproc when tags contain "forwaded". Clearly its there for a reason. Commenting it out resulted in host.* entries but populated with the details of the filebeat server, not ip or hostname details of the log entry.

Is this functionality not expected to work without elastic agents on the machines? In this case, it's an ASA so I can't.

[Mike_Paquette]

Hi Dan,
Thanks for the post, and we're glad you're giving our SIEM/Security app a try!

However lots of SIEM dashboard remain unpopulated, in particular the "hosts" tab.

Generally events from network firewalls and network monitors are not considered host events, so are not expected to populate the SIEM/Security app Hosts view charts and tables (aka widgets).
Instead, these events should be showing up on the SIEM/Security app Network page widgets.
Can you confirm that you're seeing your Cisco and Netflow events in the Network views?

Looking at filebeat docs 6.8 vs 7.9 however:
http s://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-host-processor.html
vs
http s://www.elastic.co/guide/en/beats/filebeat/6.8/exported-fields-host-processor.html
7.9 does not show host.name as an exported field, whereas 6.8 does. I tried 6.8, but it does not contain the cisco module.

This is expected. Early versions of filebeat, such as 6.8, were incorrectly populating the host.name field when filebeat was used to receive events from another device. The filebeat behavior has evolved over time to be consistent with the roles as defined by Elastic Common Schema

For example, in Elastic Common Schema, typical events generated by a network firewall are expected to populate the field sets in the diagram below. You'll see that the host.* fields are not typically included. (There are exceptions for example, when the event is actually coming from the firewall about its own configuration, etc.).

Docs were followed to the letter and even the getting started documentation for 7.9 seems to imply it should work out of the box with the provided example dashboards.

In addition to the SIEM/Security app Network page widgets, the filebeat modules also install one or more Kibana Dashboards into Kibana when the filebeat setup command is first run. For example the NetFlow module provides these.

and the filebeat Cisco module also installs a dashboard:

Can you confirm whether your dashboards and the SIEM/Security app Network page widgets are being populated with your events?

Thanks!

[borna_talebi]

Hi Dan,
If you want to add your switch or firewall to Hosts tab, you can use an ingest pipeline to change host.name field to your firewall device. @timestamp and host.name needs to be available according to docs.
@timestamp field is correct so a simple ingest pipeline that changes host.name field based on source log address is the easiest way I can think of for your problem.