Running Auditbeat 6.2.4 on Redhat connecting to ELK 6.2.2 on Ubuntu.
Auditbeat.yml as follows:
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/auditbeat/index.html
#============================ Config Reloading ================================
# Config reloading allows to dynamically load modules. Each file which is
# monitored must contain one or multiple modules as a list.
auditbeat.config.modules:
# Glob pattern for configuration reloading
path: ${path.config}/conf.d/*.yml
# Period on which files under path should be checked for changes
reload.period: 10s
# Set to true to enable config reloading
reload.enabled: false
# Maximum amount of time to randomly delay the start of a metricset. Use 0 to
# disable startup delay.
auditbeat.max_start_delay: 10s
#========================== Modules configuration =============================
auditbeat.modules:
# The kernel metricset collects events from the audit framework in the Linux
# kernel. You need to specify audit rules for the events that you want to audit.
- module: auditd
resolve_ids: true
failure_mode: log
backlog_limit: 8196
rate_limit: 0
include_raw_message: false
include_warnings: false
audit_rules: |
-w /etc/auditbeat/auditbeat.yml -p wa -k auditbeat_issue
-w /etc/passwd -p wa -k passwd_changes
-w /PTC/ -p wr -k ptc_code_access
#================================ General ======================================
# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
# If this options is not defined, the hostname is used.
name: ptc-desk
#================================ Outputs ======================================
# Configure what output to use when sending the data collected by the beat.
#----------------------------- Logstash output ---------------------------------
output.logstash:
hosts: ["soptct62-02.ptc.com:5044"]
protocol: "https"
# output.console:
# pretty: true
#================================= Paths ======================================
# The data path for the auditbeat installation. This is the default base path
# for all the files in which auditbeat needs to store its data. If not set by a
# CLI flag or in the configuration file, the default for the data path is a data
# subdirectory inside the home path.
path.data: /MON/data
# The logs path for a auditbeat installation. This is the default location for
# the Beat's log files. If not set by a CLI flag or in the configuration file,
# the default for the logs path is a logs subdirectory inside the home path.
path.logs: /MON/logs
#============================== Dashboards =====================================
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards are disabled by default and can be enabled either by setting the
# options here, or by using the `-setup` CLI flag or the `setup` command.
setup.dashboards.enabled: true
#============================== Template =====================================
# A template is used to set the mapping in Elasticsearch
# By default template loading is enabled and the template is loaded.
# These settings can be adjusted to load your own template or overwrite existing ones.
# Set to false to disable template loading.
setup.template.enabled: true
# Overwrite existing template
setup.template.overwrite: false
#============================== Kibana =====================================
# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
host: "soptct62-02.ptc.com:5601"
Both the template and the dashboards are deployed, but on going to the [Auditbeat Auditd] Overview dashboard in Kibana I get
I could not replicate the issue you are seeing. I used the same versions (Auditbeat 6.2.4, ES 6.2.2, Kibana 6.2.2) in a clean environment.
After bringing up ES and Kibana, I ran the auditbeat setup command. This installs the index template to Elasticsearch and the dashboards and index pattern to Kibana. I chose to use -E for these examples as an alternative to changing any options in auditbeat.yml.
Sigh. Could not get them to deploy manually so stuck the deploy in the yml file as above, but with template deployment commented out and the dashboards deployed and I get no index errors, but I also get no messages. Only ones showing are the ones from after deploying the template and using the second command above. So frustrating!
Update, using your command it starts sending them. I've been using "sudo service auditbeat start" for which I get
[nhopper@ptc38501-01 ~]$ sudo service auditbeat start
Starting auditbeat: 2018-04-23T12:28:00.458+0100 INFO instance/beat.go:468 Home path: [/usr/share/auditbeat] Config path: [/etc/auditbeat] Data path: [/var/lib/auditbeat] Logs path: [/var/log/auditbeat]
2018-04-23T12:28:00.459+0100 INFO instance/beat.go:475 Beat UUID: 83ea210b-94a7-46c8-9ba4-8a1a04284bce
2018-04-23T12:28:00.459+0100 INFO instance/beat.go:213 Setup Beat: auditbeat; Version: 6.2.4
2018-04-23T12:28:00.461+0100 INFO pipeline/module.go:76 Beat name: ptc-desk
2018-04-23T12:28:00.462+0100 INFO [auditd] auditd/audit_linux.go:65 auditd module is running as euid=0 on kernel=2.6.32-573.18.1.el6.x86_64
2018-04-23T12:28:00.463+0100 INFO [auditd] auditd/audit_linux.go:88 socket_type=unicast will be used.
Config OK
[ OK ]
Will try and figure what the difference is as both seem to be using the same auditbeat.yml file
Since you have the Logstash output enabled in your main config file you'll need to disable the Logstash output temporarily while the setup command runs because our config validation ensures that you have not enabled two different outputs (like ES and LS).
Did you attempt to change the name of the index at all? Someone else reported the same "field" is a required parameter error while customizing the index. https://github.com/elastic/beats/issues/6935
Can you share the Logstash configuration that you are using? This is the recommended input and output config.
Hi, nope no changes to the index. I was happy to use it as it was until I had either found it was missing something or needed changing. But this was still at the very early set up stage of what I was trying. Logstash as follows (Stripped out most of the commented out stuff):
# The # character at the beginning of a line indicates a comment.
input {
beats {
port => "5044"
}
}
# Filters
filter {
# Initially removing nagious events auid might not be properly correct, but is working at the moment
if [user][name_map][auid] == "nagios" {
drop { }
}
# Will output this to ES
output {
elasticsearch {
hosts => ["elkselasticsearch:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
# stdout { codec => rubydebug }
}
We ran into this issue with all the default Beats dashboards and seems to be caused by needing to use 'fieldname.keyword' instead of 'fieldname' for string fields. Updsting the visualisations to do so resolved it for us.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
