# Extracting some fields from an array item and renaming them with Auditbeat's processors

**URL:** <https://discuss.elastic.co/t/extracting-some-fields-from-an-array-item-and-renaming-them-with-auditbeats-processors/265408>\
**Category:** Beats\
**Tags:** auditbeat\
**Created:** [February 24, 2021, 8:34pm UTC](https://discuss.elastic.co/t/extracting-some-fields-from-an-array-item-and-renaming-them-with-auditbeats-processors/265408 "2021-02-24T20:34:13Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![alanoe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alanoe/32/84505_2.png) [@alanoe](https://discuss.elastic.co/u/alanoe)\
**Post date:** [February 24, 2021, 8:34pm UTC](https://discuss.elastic.co/t/extracting-some-fields-from-an-array-item-and-renaming-them-with-auditbeats-processors/265408/1 "2021-02-24T20:34:13Z")

</div>

I'm trying to use Auditbeat to create a filesystem tracker. When I touch a file in the directory watched by my audit rules, I get the following event from Auditbeat saved to Elasticsearch:

```auto
    {
      "_index":"auditbeat-7.11.1-2021.02.18-000001",
      "_type":"_doc",
      "_id":"CWxI1HcBqGJqLMcXlYSQ",
      "_source":{
        "@timestamp":"2021-02-24T13:44:21.638Z",
        "agent": {
          "type":"auditbeat",
          "version":"7.11.1",
          "hostname":"localhost.localdomain",
          "ephemeral_id":"bbc6fad6-5b21-42ca-8240-13478adc07e8",
          "id":"705df1a2-a033-4f1c-bfb1-d78865741025",
          "name":"localhost.localdomain"
        },
       (...)
      "tags":["filesystem_op"],
      "auditd": {
        "data": {
          "arch":"x86_64",
          "a0":"7ffc3678f67c",
          "tty":"pts0",
          "exit":"3",
          "a1":"941",
          "syscall":"open",
          "a2":"1b6",
          "a3":"7ffc3678d320"
        },
        "session":"25",
        "summary":{
          "how":"/usr/bin/touch",
          "actor":{
            "secondary":"root","primary":"vagrant"
          },
          "object":{"primary":"/data/","type":"file"}
        },
        "paths":[
          {
            "item":"0",
            "objtype":"PARENT",
            "name":"/data/", 
            "mode":"040755",
            "ouid":"0"
            "ogid":"0",
            "inode":"723959",
            "cap_fe":"0",
            "cap_fi":"0000000000000000",
            "cap_fp":"0000000000000000",
            "cap_fver":"0",
            "rdev":"00:00",
            "dev":"08:01",
          },
          {
            "item":"1",
            "objtype":"CREATE",
            "name":"/data/test3",
            "mode":"0100644",
            "ouid":"0",
            "ogid":"0"
            "inode":"6304070",
            "cap_fe":"0",
            "cap_fi":"0000000000000000",
            "cap_fp":"0000000000000000",
            "cap_fver":"0",
            "rdev":"00:00",
            "dev":"08:01",
          }
        ],
        (...)

```

I want to convert this record to:

```auto
{
  "_source":{
    "@timestamp":"2021-02-24T13:44:21.638Z",
    "agent": {
      "type":"auditbeat",
      "version":"7.11.1"
    }
  },
  "file_path": "/data/test3",
  "file_op": "create",
  "mode": "0100644",
  "owner_uid": "0",
  "owner_gid": "0",
  tags: ["filesystem_op"]
}

```

As you can see, I want to extract the file path, mode, owner user ID, owner group ID of the path which has objtype == "CREATE" (alternatively, I think I can rely on the elements order in "arrays" and always get the last element) and rename them. I have checked Auditbeat processors, but I have not found a way to extract some fields of the last item of an array OR some fields of an item of an array with a specific key/value. Is that possible with the extract\_array processor or any other processor? If not, I guess a possible solution would be to create a custom processor ?

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [February 25, 2021, 9:48pm UTC](https://discuss.elastic.co/t/extracting-some-fields-from-an-array-item-and-renaming-them-with-auditbeats-processors/265408/2 "2021-02-25T21:48:27Z")

</div>

I guess you could use something like below. You can test it with [Beats Playground](https://andrewkroh.github.io/beats-playground/#?load_logs=https://gist.githubusercontent.com/andrewkroh/0beeb595cc9ed83cf53a410e49da40af/raw/8c23f84a0a8a13948772e832f97dfc9ced6bccef/auditd.txt).

```auto
processors:
- decode_json_fields:
    fields: message
    target: ""
- script:
    lang: javascript
    source: |
      function process(evt) {
          var files = evt.Get('auditd.paths');
          if (files) {
              for (var i = 0; i < files.length; i++) {
                  var file = files[i];
                  if (file.objtype === 'CREATE') {
                      evt.Put("file_op", file.objtype.toLowerCase());
                      evt.Put("file_path", file.name);
                      evt.Put("file_mode", file.mode);
                      evt.Put("owner_uid", file.ouid);
                      evt.Put("owner_gid", file.ogid);
                      break;
                  }
              }
          }
      }

```

Take a look at ECS. You might want to follow it in order to be able to take advantage of some parts of Kibana. [File Fields | Elastic Common Schema (ECS) Reference [1.8] | Elastic](https://www.elastic.co/guide/en/ecs/current/ecs-file.html)

---

<div class="post-metadata">

**Author:** ![alanoe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alanoe/32/84505_2.png) [@alanoe](https://discuss.elastic.co/u/alanoe)\
**Post date:** [February 26, 2021, 6:28pm UTC](https://discuss.elastic.co/t/extracting-some-fields-from-an-array-item-and-renaming-them-with-auditbeats-processors/265408/3 "2021-02-26T18:28:09Z")

</div>

Thanks! Is this script processor documented somewhere? I can't find it anywhere in the processors documentation at [Define processors | Auditbeat Reference [7.11] | Elastic](https://www.elastic.co/guide/en/beats/auditbeat/current/defining-processors.html). Shouldn't it appear in the list of available processors on the right side?

Where can I find the list of supported languages (besides Javascript) ?

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [February 27, 2021, 12:12am UTC](https://discuss.elastic.co/t/extracting-some-fields-from-an-array-item-and-renaming-them-with-auditbeats-processors/265408/4 "2021-02-27T00:12:54Z")

</div>

I didn't test it with Auditbeat, my bad. It might not be part of Auditbeat (originally I think it was included only with Filebeat and Journalbeat b/c they process logs that need transformed). You can try it to see.

You can see its docs if you look at Filebeat. [Script Processor | Filebeat Reference [7.11] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/processor-script.html)

If it's not there then you can use Ingest Node with Auditbeat. It's easy to setup. See [Processors | Elasticsearch Reference [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest-processors.html) and [Configure the Elasticsearch output | Auditbeat Reference [7.11] | Elastic](https://www.elastic.co/guide/en/beats/auditbeat/current/elasticsearch-output.html#pipeline-option-es).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 27, 2021, 2:13am UTC](https://discuss.elastic.co/t/extracting-some-fields-from-an-array-item-and-renaming-them-with-auditbeats-processors/265408/5 "2021-03-27T02:13:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.