Auditbeat default dashboards missing required field

Running Auditbeat 6.2.4 on Redhat connecting to ELK 6.2.2 on Ubuntu.

Auditbeat.yml as follows:

# You can find the full configuration reference here:

#============================  Config Reloading ================================

# Config reloading allows to dynamically load modules. Each file which is
# monitored must contain one or multiple modules as a list.

  # Glob pattern for configuration reloading
  path: ${path.config}/conf.d/*.yml

  # Period on which files under path should be checked for changes
  reload.period: 10s

  # Set to true to enable config reloading
  reload.enabled: false

# Maximum amount of time to randomly delay the start of a metricset. Use 0 to
# disable startup delay.
auditbeat.max_start_delay: 10s

#==========================  Modules configuration =============================

# The kernel metricset collects events from the audit framework in the Linux
# kernel. You need to specify audit rules for the events that you want to audit.
- module: auditd
  resolve_ids: true
  failure_mode: log
  backlog_limit: 8196
  rate_limit: 0
  include_raw_message: false
  include_warnings: false
  audit_rules: |
    -w /etc/auditbeat/auditbeat.yml -p wa -k auditbeat_issue
    -w /etc/passwd -p wa -k passwd_changes
    -w /PTC/ -p wr -k ptc_code_access
#================================ General ======================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
# If this options is not defined, the hostname is used.
name: ptc-desk    

#================================ Outputs ======================================

# Configure what output to use when sending the data collected by the beat.

#----------------------------- Logstash output ---------------------------------
  hosts: [""]
  protocol: "https"

#  output.console:
#  pretty: true

#================================= Paths ======================================
# The data path for the auditbeat installation. This is the default base path
# for all the files in which auditbeat needs to store its data. If not set by a
# CLI flag or in the configuration file, the default for the data path is a data
# subdirectory inside the home path. /MON/data

# The logs path for a auditbeat installation. This is the default location for
# the Beat's log files. If not set by a CLI flag or in the configuration file,
# the default for the logs path is a logs subdirectory inside the home path.
path.logs: /MON/logs

#============================== Dashboards =====================================
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards are disabled by default and can be enabled either by setting the
# options here, or by using the `-setup` CLI flag or the `setup` command.
setup.dashboards.enabled: true
#============================== Template =====================================

# A template is used to set the mapping in Elasticsearch
# By default template loading is enabled and the template is loaded.
# These settings can be adjusted to load your own template or overwrite existing ones.

# Set to false to disable template loading.
setup.template.enabled: true

# Overwrite existing template
setup.template.overwrite: false

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  host: ""

Both the template and the dashboards are deployed, but on going to the [Auditbeat Auditd] Overview dashboard in Kibana I get

In the More Info section I get

Visualize: "field" is a required parameter

Error: "field" is a required parameter

No changes have been made to ELK, the index or any of the dashboards. This is as plain as you can get.

Any help gratefully received

I could not replicate the issue you are seeing. I used the same versions (Auditbeat 6.2.4, ES 6.2.2, Kibana 6.2.2) in a clean environment.

After bringing up ES and Kibana, I ran the auditbeat setup command. This installs the index template to Elasticsearch and the dashboards and index pattern to Kibana. I chose to use -E for these examples as an alternative to changing any options in auditbeat.yml.

auditbeat setup -e -d "*" \
  -E output.logstash.enabled=false \
  -E output.elasticsearch.hosts=['elasticstack:9200'] \
  -E \
  -E setup.dashboards.always_kibana=true

I opened Kibana and loaded the dashboards. They all loaded and said that no auditbeat-* indices existed.

Next I started Auditbeat and sent the data to Logstash.

auditbeat run -e -d "*" \
  -E output.elasticsearch.enabled=false \
  -E output.logstash.hosts=['elasticstack:5044']

I refreshed Kibana and the dashboards started showing data.

I cleared out ELK so starting from scratch. Started ELK.

Unfortunately, this is what I get when I run the first command

[nhopper@ptc38501-01 ~]$ sudo auditbeat setup -e -d "*" \
>   -E output.elasticsearch.hosts=[''] \
>   -E \
>   -E setup.dashboards.always_kibana=true
Exiting: error unpacking config data: more than one namespace configured accessing 'output' (source:'/etc/auditbeat/auditbeat.yml')

I had been using the setup command before and not done it through the yml file and it used to work and now it just hangs.

[nhopper@ptc38501-01 ~]$ auditbeat setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=[""]'`

Just sits on the prompt. Apologies for being such a newbie!

The following worked:

[nhopper@ptc38501-01 ~]$ sudo auditbeat setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=[""]'
Loaded index template

Obviously no dashboards at this stage.

Ran the second command above and it generated messages that when I linked the index, showed correctly.

Now to deploy the dashboards

Sigh. Could not get them to deploy manually so stuck the deploy in the yml file as above, but with template deployment commented out and the dashboards deployed and I get no index errors, but I also get no messages. Only ones showing are the ones from after deploying the template and using the second command above. So frustrating!

Update, using your command it starts sending them. I've been using "sudo service auditbeat start" for which I get

[nhopper@ptc38501-01 ~]$ sudo service auditbeat start
Starting auditbeat: 2018-04-23T12:28:00.458+0100	INFO	instance/beat.go:468	Home path: [/usr/share/auditbeat] Config path: [/etc/auditbeat] Data path: [/var/lib/auditbeat] Logs path: [/var/log/auditbeat]
2018-04-23T12:28:00.459+0100	INFO	instance/beat.go:475	Beat UUID: 83ea210b-94a7-46c8-9ba4-8a1a04284bce
2018-04-23T12:28:00.459+0100	INFO	instance/beat.go:213	Setup Beat: auditbeat; Version: 6.2.4
2018-04-23T12:28:00.461+0100	INFO	pipeline/module.go:76	Beat name: ptc-desk
2018-04-23T12:28:00.462+0100	INFO	[auditd]	auditd/audit_linux.go:65	auditd module is running as euid=0 on kernel=2.6.32-573.18.1.el6.x86_64
2018-04-23T12:28:00.463+0100	INFO	[auditd]	auditd/audit_linux.go:88	socket_type=unicast will be used.
Config OK
                                                           [  OK  ]

Will try and figure what the difference is as both seem to be using the same auditbeat.yml file

Since you have the Logstash output enabled in your main config file you'll need to disable the Logstash output temporarily while the setup command runs because our config validation ensures that you have not enabled two different outputs (like ES and LS).

auditbeat setup -e -d "*" \
  -E output.logstash.enabled=false \
  -E output.elasticsearch.hosts=['elasticstack:9200'] \
  -E \
  -E setup.dashboards.always_kibana=true

Did you attempt to change the name of the index at all? Someone else reported the same "field" is a required parameter error while customizing the index.

Can you share the Logstash configuration that you are using? This is the recommended input and output config.

Hi, nope no changes to the index. I was happy to use it as it was until I had either found it was missing something or needed changing. But this was still at the very early set up stage of what I was trying. Logstash as follows (Stripped out most of the commented out stuff):

# The # character at the beginning of a line indicates a comment.
input {
  beats {
    port => "5044"

# Filters

filter {
# Initially removing nagious events auid might not be properly correct, but is working at the moment
  if [user][name_map][auid] == "nagios" {
    drop { }
# Will output this to ES
output {
  elasticsearch { 
    hosts => ["elkselasticsearch:9200"] 
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
#    stdout { codec => rubydebug }

We ran into this issue with all the default Beats dashboards and seems to be caused by needing to use 'fieldname.keyword' instead of 'fieldname' for string fields. Updsting the visualisations to do so resolved it for us.