Extracting some fields from an array item and renaming them with Auditbeat's processors

[alanoe (Alan Evangelista)]

I'm trying to use Auditbeat to create a filesystem tracker. When I touch a file in the directory watched by my audit rules, I get the following event from Auditbeat saved to Elasticsearch:

    {
      "_index":"auditbeat-7.11.1-2021.02.18-000001",
      "_type":"_doc",
      "_id":"CWxI1HcBqGJqLMcXlYSQ",
      "_source":{
        "@timestamp":"2021-02-24T13:44:21.638Z",
        "agent": {
          "type":"auditbeat",
          "version":"7.11.1",
          "hostname":"localhost.localdomain",
          "ephemeral_id":"bbc6fad6-5b21-42ca-8240-13478adc07e8",
          "id":"705df1a2-a033-4f1c-bfb1-d78865741025",
          "name":"localhost.localdomain"
        },
       (...)
      "tags":["filesystem_op"],
      "auditd": {
        "data": {
          "arch":"x86_64",
          "a0":"7ffc3678f67c",
          "tty":"pts0",
          "exit":"3",
          "a1":"941",
          "syscall":"open",
          "a2":"1b6",
          "a3":"7ffc3678d320"
        },
        "session":"25",
        "summary":{
          "how":"/usr/bin/touch",
          "actor":{
            "secondary":"root","primary":"vagrant"
          },
          "object":{"primary":"/data/","type":"file"}
        },
        "paths":[
          {
            "item":"0",
            "objtype":"PARENT",
            "name":"/data/", 
            "mode":"040755",
            "ouid":"0"
            "ogid":"0",
            "inode":"723959",
            "cap_fe":"0",
            "cap_fi":"0000000000000000",
            "cap_fp":"0000000000000000",
            "cap_fver":"0",
            "rdev":"00:00",
            "dev":"08:01",
          },
          {
            "item":"1",
            "objtype":"CREATE",
            "name":"/data/test3",
            "mode":"0100644",
            "ouid":"0",
            "ogid":"0"
            "inode":"6304070",
            "cap_fe":"0",
            "cap_fi":"0000000000000000",
            "cap_fp":"0000000000000000",
            "cap_fver":"0",
            "rdev":"00:00",
            "dev":"08:01",
          }
        ],
        (...)

I want to convert this record to:

{
  "_source":{
    "@timestamp":"2021-02-24T13:44:21.638Z",
    "agent": {
      "type":"auditbeat",
      "version":"7.11.1"
    }
  },
  "file_path": "/data/test3",
  "file_op": "create",
  "mode": "0100644",
  "owner_uid": "0",
  "owner_gid": "0",
  tags: ["filesystem_op"]
}

As you can see, I want to extract the file path, mode, owner user ID, owner group ID of the path which has objtype == "CREATE" (alternatively, I think I can rely on the elements order in "arrays" and always get the last element) and rename them. I have checked Auditbeat processors, but I have not found a way to extract some fields of the last item of an array OR some fields of an item of an array with a specific key/value. Is that possible with the extract_array processor or any other processor? If not, I guess a possible solution would be to create a custom processor ?

[andrewkroh]

I guess you could use something like below. You can test it with Beats Playground.

processors:
- decode_json_fields:
    fields: message
    target: ""
- script:
    lang: javascript
    source: |
      function process(evt) {
          var files = evt.Get('auditd.paths');
          if (files) {
              for (var i = 0; i < files.length; i++) {
                  var file = files[i];
                  if (file.objtype === 'CREATE') {
                      evt.Put("file_op", file.objtype.toLowerCase());
                      evt.Put("file_path", file.name);
                      evt.Put("file_mode", file.mode);
                      evt.Put("owner_uid", file.ouid);
                      evt.Put("owner_gid", file.ogid);
                      break;
                  }
              }
          }
      }

Take a look at ECS. You might want to follow it in order to be able to take advantage of some parts of Kibana. File Fields | Elastic Common Schema (ECS) Reference [1.8] | Elastic

[alanoe (Alan Evangelista)]

Thanks! Is this script processor documented somewhere? I can't find it anywhere in the processors documentation at Define processors | Auditbeat Reference [7.11] | Elastic. Shouldn't it appear in the list of available processors on the right side?

Where can I find the list of supported languages (besides Javascript) ?

[andrewkroh]

I didn't test it with Auditbeat, my bad. It might not be part of Auditbeat (originally I think it was included only with Filebeat and Journalbeat b/c they process logs that need transformed). You can try it to see.

You can see its docs if you look at Filebeat. Script Processor | Filebeat Reference [7.11] | Elastic

If it's not there then you can use Ingest Node with Auditbeat. It's easy to setup. See Processors | Elasticsearch Reference [master] | Elastic and Configure the Elasticsearch output | Auditbeat Reference [7.11] | Elastic.