Splitting a field that looks like an array

Elastic Stack Logstash
1

I'm getting audit data from my linux hosts that will sometimes repeat a field name, so I get what looks like an array of data in that field.
I've tried various forms of 'split' to see if I can separate the data into two different fields, but so far no luck.

Here's the data:
"auditd.log": {
"subj": "system_u:system_r:sshd_t:s0-s0:c0.c1023",
"UID": "root",
"AUID": "haliburtonj",
"SUID": "haliburtonj",
"auid": "609443",
"uid": "0",
"msg": [
"audit(1728476586.530:1812440):",
"op=destroy kind=session fp=? direction=from-server spid=475218 suid=609443 rport=52032 laddr=X.X.X.Xlport=22 exe="/usr/sbin/sshd" hostname=? addr=X.X.X.X terminal=? res=success"
],
"type": "CRYPTO_KEY_USER",
"pid": "475148",
"ses": "406"
},

I want to separate the auditd.log.msg field. The second field I'd like to be able to run thru KV.

I've tried this -
mutate { convert => { "[auditd][log][auditmsg]" => "string" } split => { field => "[auditd][log][msg]" } add_field => { "[auditd][log][auditid]" => "%{[auditd][log][msg][0]}" "[auditd][log][auditmesg]" => "%{[auditd][log][msg][1]}" } }
but all this does is assign audltd.log.auditid = "%{[auditd][log][msg][0]}" literally.

Any suggestions?

2

Shoudn't that be "[auditd][log][msg]"?

A mutate filter does operations in a fixed order. Luckily convert comes before split and add_field happens at the end, so using one mutate for all of these operations should work.

3

Typo on my part. Yes, auditmsg should just be msg. Still has same output.

4

The mutate+convert converts the msg field from an array to a string. The mutate+split then converts it back to an array. It doesn't do anything.

Perhaps the underlying issue is that the field you have is actually called [audit.log][msg], in which case the convert and split are currently no-ops (the fields they operate on do not exist) and the add_field will not modify the references to non-existent fields. Try

mutate {
    add_field => {
        "[auditd.log][auditid]" => "%{[auditd.log][msg][0]}"
        "[auditd.log][auditmesg]" => "%{[auditd.log][msg][1]}" }
    }
}
5

Still getting -
auditd.log.auditid literally = "%{[auditd.log][msg][0]}"
auditd.log.auditmesg literally = "%{[auditd.log][msg][1]}"

6

Try using

output { stdout { codec => rubydebug } }

and show us what the event looks like.

7

"auditd.log" => {
"a1" => "c0006be10c",
"exe" => "/usr/bin/podman",
"sgid" => "70000",
"ARCH" => "x86_64",
"ses" => "7",
"SUID" => "yoosr",
"a3" => "0",
"FSGID" => "engineering",
"tty" => "none",
"egid" => "70000",
"suid" => "11638518",
"auditid" => "%{[auditd][log][msg][0]}",
"UID" => "yoosr",
"node" => "sweetwater.u.l3",
"type" => "SYSCALL",
"arch" => "c000003e",
"items" => "2",
"syscall" => "263",
"msg" => "audit(1728489445.008:50936455):",
"auid" => "11638518",
"uid" => "11638518",
"euid" => "11638518",
"exit" => "0",
"ppid" => "69476",
"comm" => "podman",
"a0" => "a",
"subj" => "unconfined_u:unconfined_r:container_runtime_t:s0-s0:c0.c1023",
"a2" => "0",
"SYSCALL" => "unlinkat",
"key" => "delete",
"gid" => "70000",
"fsgid" => "70000",
"AUID" => "yoosr",
"GID" => "engineering",
"EUID" => "yoosr",
"success" => "yes",
"FSUID" => "yoosr",
"pid" => "2837330",
"EGID" => "engineering",
"SGID" => "engineering",
"fsuid" => "11638518",
"auditmesg" => "%{[auditd][log][msg][1]}"

8

Found the problem, too many square brackets under add_field.

"[auditd][log][msg]" should be "[audit.log][msg]"
"[auditd][log][auditid]" should be "[auditd.log][auditid]"
"[auditd][log][auditmesg]" should be '"[auditd.log][auditmesg]"