Hi all,
I have something weird since upgrade to 7.8 of auditbeat, below the log that produce by auditbeat
I use Centos 7.6 with kernel
Linux sync-02 3.10.0-1062.18.1.el7.x86_64 #1 SMP Tue Mar 17 23:49:17 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
My config of auditbeat like below for auditbeat.yml
#========================== Modules configuration =============================
auditbeat.modules:
- module: auditd
# Load audit rules from separate files. Same format as audit.rules(7).
audit_rule_files: [ '${path.config}/audit.rules.d/*.conf' ]
audit_rules: |
## Define audit rules here.
## Create file watches (-w) or syscall audits (-a or -A). Uncomment these
## examples or add your own rules.
## If you are on a 64 bit platform, everything should be running
## in 64 bit mode. This rule will detect any use of the 32 bit syscalls
## because this might be a sign of someone exploiting a hole in the 32
## bit API.
#-a always,exit -F arch=b32 -S all -F key=32bit-abi
## Executions.
#-a always,exit -F arch=b64 -S execve,execveat -k exec
## External access (warning: these can be expensive to audit).
#-a always,exit -F arch=b64 -S accept,bind,connect -F key=external-access
## Identity changes.
#-w /etc/group -p wa -k identity
#-w /etc/passwd -p wa -k identity
#-w /etc/gshadow -p wa -k identity
## Unauthorized access attempts.
#-a always,exit -F arch=b64 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EACCES -k access
#-a always,exit -F arch=b64 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EPERM -k access
- module: file_integrity
paths:
- /bin
- /usr/bin
- /sbin
- /usr/sbin
- /etc
exclude_files:
- '(?i)\.sw[nop]$'
- '~$'
- '/\.git($|/)'
include_files: []
scan_at_start: true
scan_rate_per_sec: 50 MiB
max_file_size: 100 MiB
hash_types: [sha1]
recursive: false
#- module: system
# datasets:
# - host # General host information, e.g. uptime, IPs
# - login # User logins, logouts, and system boots.
# #- package # Installed, updated, and removed packages
# - process # Started and stopped processes
# - socket # Opened and closed sockets
# - user # User information
# How often datasets send state updates with the
# current state of the system (e.g. all currently
# running processes, all open sockets).
# socket.enable_ipv6: false
# period: 20s
# state.period: 12h
# Enabled by default. Auditbeat will read password fields in
# /etc/passwd and /etc/shadow and store a hash locally to
# detect any changes.
#user.detect_password_changes: true
- module: system
datasets:
- host
- login
- package
- user
period: 1m
user.detect_password_changes: true
- module: system
datasets:
- process
- socket
period: 1s
# File patterns of the login record files.
login.wtmp_file_pattern: /var/log/wtmp*
login.btmp_file_pattern: /var/log/btmp*
The rule.conf in /etc/auditbeat/audit.rules.d/rules.conf as
## If you are on a 64 bit platform, everything should be running
## in 64 bit mode. This rule will detect any use of the 32 bit syscalls
## because this might be a sign of someone exploiting a hole in the 32
## bit API.
-a always,exit -F arch=b32 -S all -F key=32bit-abi
## Executions.
-a always,exit -F arch=b64 -S execve,execveat -k exec
## Identity changes.
-w /etc/group -p wa -k identity
-w /etc/passwd -p wa -k identity
-w /etc/gshadow -p wa -k identity
## Unauthorized access attempts.
#-a always,exit -F arch=b64 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EACCES -k access
#-a always,exit -F arch=b64 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EPERM -k access
# Unauthorized access attempts to files (unsuccessful).
-a always,exit -F arch=b32 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EACCES -F key=access
-a always,exit -F arch=b32 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EPERM -F key=access
-a always,exit -F arch=b64 -S open,truncate,ftruncate,creat,openat,open_by_handle_at -F exit=-EACCES -F key=access
-a always,exit -F arch=b64 -S open,truncate,ftruncate,creat,openat,open_by_handle_at -F exit=-EPERM -F key=access
-a always,exit -F arch=b64 -S socket -F a0=2 -k socket
-a always,exit -F arch=b64 -S socket -F a0=10 -k socket
-a always,exit -F arch=b32 -S socket -F a0=2 -k socket
-a always,exit -F arch=b32 -S socket -F a0=10 -k socket
My ipv6 disabled by network-scripts startup
TYPE="Ethernet"
BOOTPROTO="dhcp"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="no"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_PEERDNS="yes"
IPV6_PEERROUTES="yes"
IPV6_FAILURE_FATAL="no"
NAME="ens160"
UUID="b5cb9065-0222-40f4-b116-fc8df7cd27a8"
ONBOOT="yes"
HWADDR="02:00:27:4b:00:14"
PEERDNS="yes"
PEERROUTES="yes"
My problem is the socket event don't send the event 2 minutes after auditbeat start and the result in auditbeat index for event.category: network_traffic is just 2 minutes range then empty. If i restart the auditbeat, the socket event send again, only 2 minutes then disappeared.
Below the result of the auditbeat log.
Jun 30 05:36:31 sync-02 auditbeat: 2020-06-30T05:36:31.119+0700#011INFO#011[monitoring]#011log/log.go:145#011Non-zero metrics in the last 30s#011{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":4990,"time":{"ms":4992}},"total":{"ticks":11070,"time":{"ms":11081},"value":11070},"user":{"ticks":6080,"time":{"ms":6089}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":219},"info":{"ephemeral_id":"3eb85d0c-3e51-4f8e-b852-fce81d262d5a","uptime":{"ms":34867}},"memstats":{"gc_next":14937248,"memory_alloc":8154400,"memory_total":425490624,"rss":103575552},"runtime":{"goroutines":76}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":1053,"batches":32,"total":1053},"type":"elasticsearch"},"pipeline":{"clients":8,"events":{"active":0,"published":1053,"retry":2,"total":1053},"queue":{"acked":1053}}},"metricbeat":{"file_integrity":{"file":{"events":1,"success":1}},"system":{"host":{"events":2,"success":2},"process":{"events":324,"success":324},"socket":{"events":725,"success":725}}},"system":{"cpu":{"cores":12},"load":{"1":20.1,"15":21.24,"5":20.33,"norm":{"1":1.675,"15":1.77,"5":1.6942}}}}}}
Jun 30 05:37:01 sync-02 auditbeat: 2020-06-30T05:37:01.114+0700#011INFO#011[monitoring]#011log/log.go:145#011Non-zero metrics in the last 30s#011{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":7410,"time":{"ms":2426}},"total":{"ticks":16710,"time":{"ms":5646},"value":16710},"user":{"ticks":9300,"time":{"ms":3220}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":219},"info":{"ephemeral_id":"3eb85d0c-3e51-4f8e-b852-fce81d262d5a","uptime":{"ms":64867}},"memstats":{"gc_next":17701680,"memory_alloc":14282880,"memory_total":803499392,"rss":1572864},"runtime":{"goroutines":76}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":3289,"batches":76,"total":3289}},"pipeline":{"clients":8,"events":{"active":75,"published":3364,"total":3364},"queue":{"acked":3289}}},"metricbeat":{"system":{"process":{"events":120,"success":120},"socket":{"events":3244,"success":3244}}},"system":{"load":{"1":19.92,"15":21.23,"5":20.37,"norm":{"1":1.66,"15":1.7692,"5":1.6975}}}}}}
Jun 30 05:37:31 sync-02 auditbeat: 2020-06-30T05:37:31.113+0700#011INFO#011[monitoring]#011log/log.go:145#011Non-zero metrics in the last 30s#011{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":9770,"time":{"ms":2358}},"total":{"ticks":30480,"time":{"ms":13765},"value":30480},"user":{"ticks":20710,"time":{"ms":11407}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":219},"info":{"ephemeral_id":"3eb85d0c-3e51-4f8e-b852-fce81d262d5a","uptime":{"ms":94871}},"memstats":{"gc_next":18110736,"memory_alloc":16534464,"memory_total":1135415656,"rss":1163264},"runtime":{"goroutines":76}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":2483,"batches":60,"total":2483}},"pipeline":{"clients":8,"events":{"active":0,"published":2408,"total":2408},"queue":{"acked":2483}}},"metricbeat":{"system":{"process":{"events":100,"success":100},"socket":{"events":2308,"success":2308}}},"system":{"load":{"1":16.85,"15":20.96,"5":19.64,"norm":{"1":1.4042,"15":1.7467,"5":1.6367}}}}}}
Jun 30 05:38:01 sync-02 auditbeat: 2020-06-30T05:38:01.117+0700#011INFO#011[monitoring]#011log/log.go:145#011Non-zero metrics in the last 30s#011{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":10890,"time":{"ms":1124}},"total":{"ticks":61980,"time":{"ms":31507},"value":61980},"user":{"ticks":51090,"time":{"ms":30383}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":219},"info":{"ephemeral_id":"3eb85d0c-3e51-4f8e-b852-fce81d262d5a","uptime":{"ms":124871}},"memstats":{"gc_next":21779136,"memory_alloc":16074832,"memory_total":1370155424,"rss":61440},"runtime":{"goroutines":76}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":92,"batches":7,"total":92}},"pipeline":{"clients":8,"events":{"active":0,"published":92,"total":92},"queue":{"acked":92}}},"metricbeat":{"system":{"process":{"events":92,"success":92}}},"system":{"load":{"1":21.92,"15":21.24,"5":20.61,"norm":{"1":1.8267,"15":1.77,"5":1.7175}}}}}}
Jun 30 05:38:31 sync-02 auditbeat: 2020-06-30T05:38:31.115+0700#011INFO#011[monitoring]#011log/log.go:145#011Non-zero metrics in the last 30s#011{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":12160,"time":{"ms":1265}},"total":{"ticks":95040,"time":{"ms":33053},"value":95040},"user":{"ticks":82880,"time":{"ms":31788}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":219},"info":{"ephemeral_id":"3eb85d0c-3e51-4f8e-b852-fce81d262d5a","uptime":{"ms":154869}},"memstats":{"gc_next":19968240,"memory_alloc":9970312,"memory_total":1604663712,"rss":581632},"runtime":{"goroutines":76}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":100,"batches":11,"total":100}},"pipeline":{"clients":8,"events":{"active":0,"published":100,"total":100},"queue":{"acked":100}}},"metricbeat":{"system":{"process":{"events":100,"success":100}}},"system":{"load":{"1":22.79,"15":21.29,"5":20.83,"norm":{"1":1.8992,"15":1.7742,"5":1.7358}}}}}
In other node, with kernel
Linux asdpsync 3.10.0-957.10.1.el7.x86_64 #1 SMP Mon Mar 18 15:06:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
and use auditbeat 7.7.1, there's no issue at all, using standard installation and configuration.
I already downgrade to 7.7.1 for the problem node, but no luck. Upgrade again to 7.8.0 also still have this problem.
Is there any workaround to make the socket event stream come back again?
Regards,
Fadjar Tandabawana