Adding Host fields to configuration

Elastic Stack Beats
1

Hi,

I'm trying to configure filebeat, winlogbeat and metricbeat to send details on their host type (OS type mainly) so I can use this field in Graylog to filter.

Host doesn't seem to be a module and the fields documented in each of the beats documentation aren't sent by default. Here is an example of the data I'm trying to send via my config to Graylog. https://www.elastic.co/guide/en/beats/filebeat/master/exported-fields-host-processor.html

My beats config looks like this at the moment:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

filebeat.inputs:
- input_type: log
  paths:
    - /var/log/*.log
  type: log
output.logstash:
   hosts: ["${user.GraylogHost}:5044"]
    path:
     data: /var/lib/graylog-sidecar/collectors/filebeat/data
     logs: /var/lib/graylog-sidecar/collectors/filebeat/log
2

You configuration seems incorrect. fields_under_root is not a global option. Also, whitespaces are off and input_type is not a keyword we use for anything.

Why are you using fields_under_root? What do you want to configure with it?

To add information about the host Filebeat runs on, you should use add_host_metadata. See more: https://www.elastic.co/guide/en/beats/filebeat/master/add-host-metadata.html

I suggest you rewrite your configuration to this:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log
 
fields:
  collector_node_id: ${sidecar.nodeName}
  gl2_source_collector: ${sidecar.nodeId}

processors:
- add_host_metadata: ~

output.logstash:
   hosts: ["${user.GraylogHost}:5044"]

path.data: /var/lib/graylog-sidecar/collectors/filebeat/data
path.logs: /var/lib/graylog-sidecar/collectors/filebeat/log
3

OK thanks, fields_under_root is required by Graylog.

How would this look for a Windows log?

4

What do you mean fields_under_root is required by Graylog? Does Graylog require the fields collector_node_id and gl1_source_collector on root level? If you use fields option the fields are added to the root.

The configuration is same regardless of the platform Filebeat is running on.

5

https://docs.graylog.org/en/3.3/pages/sidecar.html

I believe it has something to do with Graylog being able to remove the beats type prefix in the logs received (e.g. source -> filebeat_source).

Its part of the Graylog documentation and the example configuration they ship. As are the Filebeat and Winlogbeat configs shipped with Graylog.

The default configs shipped with Graylog are:

Linux Filebeat

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

filebeat.inputs:
- input_type: log
  paths:
    - /var/log/*.log
  type: log
output.logstash:
   hosts: ["192.168.1.1:5044"]
path:
  data: /var/lib/graylog-sidecar/collectors/filebeat/data
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log

Windows Filebeat

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

output.logstash:
   hosts: ["192.168.1.1:5044"]
path:
  data: C:\Program Files\Graylog\sidecar\cache\filebeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - windows
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\logs\log.log

Winlogbeat

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

output.logstash:
   hosts: ["192.168.1.1:5044"]
path:
  data: C:\Program Files\Graylog\sidecar\cache\winlogbeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - windows
winlogbeat:
  event_logs:
   - name: Application
   - name: System
   - name: Security
6

Any comment on the Graylog defaults?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.