Unable to start Auditbeat on LXC container

Elastic Stack Beats
1

Hi,

I've searched here but couldn't find a similar issue. I'm trying to run Auditbeat on CentOS 7 and it fails to start.
Log only has this:
"root@kibana1:~# cat /var/log/auditbeat/auditbeat
2018-02-15T06:23:21.005Z INFO instance/beat.go:468 Home path: [/usr/share/auditbeat] Config path: [/etc/auditbeat] Data path: [/var/lib/auditbeat] Logs path: [/var/log/auditbeat]
2018-02-15T06:23:21.005Z INFO instance/beat.go:475 Beat UUID: deed9a2c-39f1-4fd9-8617-b1a9ae15aa0b
2018-02-15T06:23:21.005Z INFO instance/beat.go:213 Setup Beat: auditbeat; Version: 6.2.1
2018-02-15T06:23:21.006Z INFO pipeline/module.go:76 Beat name: kibana1
2018-02-15T06:23:21.006Z INFO [auditd] auditd/audit_linux.go:65 auditd module is running as euid=0 on kernel=4.13.0-32-generic
2018-02-15T06:23:21.007Z ERROR instance/beat.go:667 Exiting: 1 error: 1 error: failed to create audit client: failed to get audit status: connection refused

When I do auditbeat test output it passes
"root@kibana1:~# auditbeat test output
logstash: 10.221.233.193:5043...
connection...
parse host... OK
dns lookup... OK
addresses: 10.221.233.193
dial up... OK
TLS... WARN secure connection disabled
talk to server... OK

but when I try to do auditbeat test config it fails. I checked config file and there is nothing that stands out. I enabled kibana and logstash output. I did also manually exported indexes to Elasticsearch node.

Any ideas where I'm going wrong?

Thanks

2

Hi @Luq,

Did you define any rules? Is the auditd process running? Depending on your settings it may be interfering with Auditbeat. Check the docs here: https://www.elastic.co/guide/en/beats/auditbeat/6.2/auditbeat-module-auditd.html. Please paste your config if you don't find a fix there :slight_smile:

Best regards

3

Thanks for quick response!

I learned a few things. Firstly, I didn't have auditd installed on the system. After I installed it, I couldn't get it to start. I'm blaming container for that. I'm supposing it's something to do with it being in the container because the error I get when I try to run it is:
root@centos-7 ~]# /sbin/auditd -f
Config file /etc/audit/auditd.conf opened for parsing
local_events_parser called with: yes
write_logs_parser called with: yes
log_file_parser called with: /var/log/audit/audit.log
log_group_parser called with: root
log_format_parser called with: RAW
flush_parser called with: INCREMENTAL_ASYNC
freq_parser called with: 50
max_log_size_parser called with: 8
num_logs_parser called with: 5
priority_boost_parser called with: 4
qos_parser called with: lossy
dispatch_parser called with: /sbin/audispd
name_format_parser called with: NONE
max_log_size_action_parser called with: ROTATE
space_left_parser called with: 75
space_action_parser called with: SYSLOG
action_mail_acct_parser called with: root
admin_space_left_parser called with: 50
admin_space_left_action_parser called with: SUSPEND
disk_full_action_parser called with: SUSPEND
disk_error_action_parser called with: SUSPEND
use_libwrap_parser called with: yes
tcp_listen_queue_parser called with: 5
tcp_max_per_addr_parser called with: 1
tcp_client_max_idle_parser called with: 0
enable_krb5_parser called with: no
krb5_principal_parser called with: auditd
distribute_network_parser called with: no
Cannot change priority (Operation not permitted)
The audit daemon is exiting.

4

So this centos-7 is a container? Could you explain a little bit about your setup?

5

I'm running LXD on Ubuntu. I have 4 containers one for elasticsearch, one for kibana, one for logstash - all running ubuntu, and one for centos system. I got it to work where I can see the logs in Kibana coming from centos filebeat. But when I tried to install and set up auditbeat, I came across that issue. I also tried auditbeat on ubuntu container and it also failed with the same error.

6

It looks like a permissions error, probably related to capabilities? I just checked Auditbeat guide to run it on docker and there are some capabilities you need in place: https://www.elastic.co/guide/en/beats/auditbeat/6.x/running-on-docker.html

I'm not very familiar with LXD, are you dropping any of those?

7

Yes, my understanding is that since lxc containers run as unprivileged it doesn't allow auditd to run. I think case with Docker is a little different.
Also, I'm not sure what you are referring to "are you dropping any of those?"
Well, if I comment out audit module in auditbeat.yml file and leave file integrity module, it works fine. I'll just suffice with Filebeat, I guess.
Thanks for your help!

8

Auditbeat does not have any dependencies on the Linux auditd package. In fact in most cases auditd must be stopped for Auditbeat's auditd module to function correctly. If both are running Auditbeat will log an error and tell you to stop the auditd process.

Based on the logs you are running Auditbeat as root which is good. But the Linux kernel has a few other requirements for user-space processes that try to connect to the kernel's audit framework. This makes it tricky to run any audit deamon inside of a container. One requirement for the process is that it be in the initial PID namespace. The other requirements are that the process have the AUDIT_CONTROL and AUDIT_READ capabilities.

I'm not familiar enough with LXD to be able to provide guidance on how to ensure that the containerized process meets those requirements.

9

I'm having the same issue and I'm afraid docker is not an option... Did you ever find a solution for this?