Grok Patterns: (truncated to matching patterns only)
## Fixing AVC
## Example:
## type=AVC msg=audit(1499716867.329:3287727): avc: denied { execute } for pid=29362 comm=\"ld-linux-x86-64\" path=\"/usr/libexec/hal-system-sonypic\" dev=sda1 ino=529238
## scontext=system_u:system_r:prelink_mask_t:s0-s0:c0.c1023 tcontext=system_u:object_r:hald_sonypic_exec_t:s0 tclass=file
AUDITAVCTYPE %{WORD:audit_avc_type}
AUDITAVCINFO \{ %{DATA:audit_avc_info} \}
AUDITAVC type=%{WORD:audit_type} msg=audit\(%{NUMBER:audit_epoch}:%{NUMBER:audit_counter}\): avc: denied \{ %{DATA:audit_avc_info} \} %{WORD} pid=%{NUMBER:parent_pid} comm=\\"%{GREEDYDATA:command}\\" name=\\"%{GREEDYDATA:path_file}\\" dev=%{WORD:hdd_name} ino=%{NUMBER:inode_number} scontext=%{GREEDYDATA:scontext_data} tcontext=%{GREEDYDATA:scontext_data} tclass=%{WORD:target_class_type}
AUDITAVC_OTHER type=%{WORD:audit_type} msg=audit\(%{NUMBER:audit_epoch}:%{NUMBER:audit_counter}\): avc: denied \{ %{DATA:audit_avc_info} \} %{WORD} pid=%{NUMBER:parent_pid} comm=\\"%{GREEDYDATA:command}\\" path=\\"%{GREEDYDATA:path_file}\\" dev=%{WORD:hdd_name} ino=%{NUMBER:inode_number} scontext=%{GREEDYDATA:scontext_data} tcontext=%{GREEDYDATA:scontext_data} tclass=%{WORD:target_class_type}
Logstash Configuration:
filter {
if "java-logs" in [ktopic] {
grok {
match => ["message", "%{JAVASTACKTRACEPART}"]
}
grok {
match => ["message", "%{TIMESTAMP_ISO8601:timestamp}"]
}
date {
# example: 2017-06-08 06:08:41,393
locale => "en"
#match => ["timestamp", "yyyy-MM-dd HH:mm:ss,SSS"]
match => ["timestamp", "ISO8601"]
timezone => "UTC"
remove_field => ["timestamp"]
}
}
elseif "audit" in [ktopic] {
grok {
patterns_dir => ["/etc/logstash/pattern.d/"]
match => {
"message" => [
"%{AUDITPAMLOGIN}",
"%{AUDITSYSCALL}",
"%{AUDITEXECVE}",
"%{AUDITCWD}",
"%{AUDITPATHLONG}",
"%{AUDITPATHSHORT}",
"%{AUDITPAMLOGIN}",
"%{AUDITCONFIG}",
"%{AUDITAVC}",
"%{AUDITAVC_OTHER}"
]
}
}
mutate {
add_tag => ["inside_audit_grok"]
}
date {
match => [ 'audit_epoch', 'UNIX' ]
}
}
else {
grok {
match => ["message", "%{TIMESTAMP_ISO8601:timestamp}"]
}
date {
# example: 2017-06-26T23:38:31.535488+00:00
locale => "en"
match => ["timestamp", "ISO8601"]
timezone => "UTC"
remove_field => ["timestamp"]
}
}
}
Regex Validator: https://grokdebug.herokuapp.com/
Logstash STDOut:
{
"cluster" => "xxxx",
"offset" => 4371620,
"input_type" => "log",
"ktopic" => "audit",
"source" => "/var/log/audit/audit.log",
"message" => "type=AVC msg=audit(1499722201.270:1276981): avc: denied { getattr } for pid=55358 comm=\"rsyslogd\" path=\"/var/log/audit/audit.log\" dev=sda2 ino=1179653 scontext=system_u:system_r:syslogd_t:s0 tcontext=system_u:object_r:auditd_log_t:s0 tclass=file",
"type" => "log",
"tags" => [
[0] "beats_input_codec_plain_applied",
[1] "_grokparsefailure",
[2] "inside_audit_grok"
],
"@timestamp" => 2017-07-10T21:30:03.774Z,
"node_type" => "slave",
"fwder" => "filebeat",
"@version" => "1",
"beat" => {
"hostname" => "xxxxxx",
"name" => "xxxxx",
"version" => "5.4.1"
},
"host" => "xxxxxx"
}