Hi everyone! I have got no ideas where to find problem in next situation.
When I start/restart container with auditd option socket_type: unicast, I can see in logs "message":"Failure receiving audit events","service.name":"auditbeat","error":{"message":"failed to set audit PID. An audit process is already running (PID 27141)".
So the point is that process with PID=27141 is auditbeat itself trying to run in container - it complains about itself. I mentioned acording to logs that this message comes up when dynamic config starts loading.
So the question is how to correctly run auditbeat module auditd in docker in unicast mode that auditbeat would not complain about itself?
Maybe I just forgot turn smth on or off? I've dug up all documentation and topics and still got no answer.
Given:
- Debian host with no any audit processes running
- auditbeat version 8.6.2 working in docker with module auditd. Args:
docker run --name="AB_test" \
--net "host" \
--pid "host" \
--cap-add "AUDIT_CONTROL" \
--cap-add "AUDIT_READ" \
ab_image:latest
- auditbeat.yml
logging.to_files: true
logging.files:
path: '/var/log/container'
auditbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
reload.period: 10s
auditbeat.modules:
- module: auditd
socket_type: unicast
audit_rules: |
-w /usr/share/auditbeat/auditbeat.yml -p wra -k CFG_ACCESS
-w /usr/share/auditbeat/auditbeat -p x -k AUDITBEAT_EXEC
output.console:
pretty: true
- dynamicaly loaded module from file in ${path.config}/modules.d/auditd.yml
- module: auditd
audit_rules: "-a always,exit -F arch=b64 -S execve,execveat -k SYSCALL"
index: auditbeat_log_syscal-%{+yyyy.MM.dd}
processors:
- drop_fields:
fields:
- auditd.paths
ignore_missing: true
- if:
and:
- or:
- equals:
auditd.data.tty: "(none)"
- not:
has_fields:
- auditd.data.tty
- not:
regexp:
auditd.data.tty: "^pts.*"
then:
- add_tags:
tags: [DUPL]
target: "test_tag"
Logs
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.573Z","log.origin":{"file.name":"instance/beat.go","file.line":724},"message":"Home path: [/usr/share/auditbeat] Config path: [/usr/share/auditbeat] Data path: [/usr/share/auditbeat/data] Logs path: [/usr/share/auditbeat/logs]","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.594Z","log.origin":{"file.name":"instance/beat.go","file.line":732},"message":"Beat ID: a5d5671f-cdf8-441b-ac79-d5a004f6173f","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.594Z","log.origin":{"file.name":"instance/beat.go","file.line":745},"message":"Set max procs limit: 1","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.595Z","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":124},"message":"Syscall filter successfully installed","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.595Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1096},"message":"Beat info","service.name":"auditbeat","system_info":{"beat":{"path":{"config":"/usr/share/auditbeat","data":"/usr/share/auditbeat/data","home":"/usr/share/auditbeat","logs":"/usr/share/auditbeat/logs"},"type":"auditbeat","uuid":"a5d5671f-cdf8-441b-ac79-d5a004f6173f"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.595Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1105},"message":"Build info","service.name":"auditbeat","system_info":{"build":{"commit":"unknown","libbeat":"8.6.2","time":"0001-01-01T00:00:00.000Z","version":"8.6.2"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.595Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1108},"message":"Go runtime info","service.name":"auditbeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":1,"version":"go1.20.4"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.596Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1112},"message":"Host info","service.name":"auditbeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2023-06-20T12:22:41Z","containerized":false,"name":"test_ab_host","ip":["127.0.0.1/8","::1/128","10.22.1.89/24","fe80::c3b:d8ff:fe12:a649/64","172.17.0.1/16","172.18.0.1/24"],"kernel_version":"4.19.0-24-cloud-amd64","mac":["0e:3b:d8:12:a6:49","02:42:3f:54:3d:e7","02:42:0f:bd:ba:26"],"os":{"type":"linux","family":"debian","platform":"debian","name":"Debian GNU/Linux","version":"11 (bullseye)","major":11,"minor":0,"patch":0,"codename":"bullseye"},"timezone":"UTC","timezone_offset_sec":0},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.596Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1141},"message":"Process info","service.name":"auditbeat","system_info":{"process":{"capabilities":{"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","audit_control","setfcap","audit_read"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","audit_control","setfcap","audit_read"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","audit_control","setfcap","audit_read"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","audit_control","setfcap","audit_read"],"ambient":null},"cwd":"/usr/share/auditbeat","exe":"/usr/share/auditbeat/auditbeat","name":"auditbeat","pid":27141,"ppid":27124,"seccomp":{"mode":"filter","no_new_privs":true},"start_time":"2023-06-25T15:45:35.180Z"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.597Z","log.origin":{"file.name":"instance/beat.go","file.line":296},"message":"Setup Beat: auditbeat; Version: 8.6.2","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.604Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: test_ab_host","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.605Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":107},"message":"auditd module is running as euid=0 on kernel=4.19.0-24-cloud-amd64","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.656Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":134},"message":"socket_type=unicast will be used.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.656Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":145},"message":"Starting metrics logging every 30s","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.656Z","log.origin":{"file.name":"instance/beat.go","file.line":486},"message":"auditbeat start running.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.656Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":125},"message":"Using path /usr/share/auditbeat/modules.d/*.yml and pathType file for configuration","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:35.656Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":175},"message":"Config file reloader started","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.576Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":287},"message":"Deleted 7 pre-existing audit rules.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.576Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":306},"message":"Successfully added 3 of 3 audit rules.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.627Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":333},"message":"audit status from kernel at start","service.name":"auditbeat","audit_status":{"Mask":0,"Enabled":1,"Failure":0,"PID":0,"RateLimit":0,"BacklogLimit":8192,"Lost":0,"Backlog":10,"FeatureBitmap":127,"BacklogWaitTime":0,"BacklogWaitTimeActual":0},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.628Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":360},"message":"Setting kernel backlog wait time to prevent backpressure propagating to the kernel.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.658Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":107},"message":"auditd module is running as euid=0 on kernel=4.19.0-24-cloud-amd64","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.708Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":134},"message":"socket_type=unicast will be used.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-06-25T15:45:45.710Z","log.logger":"system","log.origin":{"file.name":"system/system.go","file.line":65},"message":"Could not get host ID, will not fill entity_id fields.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.813Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":287},"message":"Deleted 3 pre-existing audit rules.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.814Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":306},"message":"Successfully added 7 of 7 audit rules.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.865Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":333},"message":"audit status from kernel at start","service.name":"auditbeat","audit_status":{"Mask":0,"Enabled":1,"Failure":0,"PID":27141,"RateLimit":0,"BacklogLimit":8192,"Lost":0,"Backlog":0,"FeatureBitmap":127,"BacklogWaitTime":0,"BacklogWaitTimeActual":0},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-06-25T15:45:45.865Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":360},"message":"Setting kernel backlog wait time to prevent backpressure propagating to the kernel.","service.name":"auditbeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-06-25T15:45:45.871Z","log.logger":"auditd","log.origin":{"file.name":"auditd/audit_linux.go","file.line":192},"message":"Failure receiving audit events","service.name":"auditbeat","error":{"message":"failed to set audit PID. An audit process is already running (PID 27141)"},"ecs.version":"1.6.0"}