# The perennial auditd event correlation problem

**URL:** <https://discuss.elastic.co/t/the-perennial-auditd-event-correlation-problem/131202>\
**Category:** Logstash\
**Created:** [May 9, 2018, 4:06pm UTC](https://discuss.elastic.co/t/the-perennial-auditd-event-correlation-problem/131202 "2018-05-09T16:06:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![0xeb0de](https://avatars.discourse-cdn.com/v4/letter/0/82dd89/32.png) [@0xeb0de](https://discuss.elastic.co/u/0xeb0de)\
**Post date:** [May 9, 2018, 4:06pm UTC](https://discuss.elastic.co/t/the-perennial-auditd-event-correlation-problem/131202/1 "2018-05-09T16:06:38Z")

</div>

Many, many people over the years have tried to find a sensible way of reducing the several events produced by Linux auditd to a single, meaningful one. audisp, for example, will produce syslog messages of type SYSCALL, CWD, PATH, PATH, PROCTITLE and EOE for a single file open, correlated by a msg=audit(1521726040.236:659) key-value pair in each message. [Here's an example.](https://discuss.elastic.co/t/logstash-aggregate-filter-on-auditd-logs/98480)

auditbeat seems to do a really good job of taking these internal events directly from the kernel uni/multicast and producing a single "someone opened this file" event which is transmitted to one of several outputs; Elasticsearch, Logstash, Kafka, Redis, File or Console. I'd **love** to use it, but alas I have an environment where I **have** to transmit syslog from all of the devices on a network.

I'm really excited about the whole concept of an Elastic Common Schema, too - I'd really like our cluster to end up storing data in this format if we can (we may have to store data somewhere in its original format, too for evidential purposes...)

My options, I guess, are ::

1. Fork auditbeat to add a syslog output type
2. Correlate from syslog in Logstash using the msg kv pair and an in-memory data store
3. Trap the syslog messages somewhere in between using a custom syslog client and some sort of message bus.

Has anyone else had to jump through these hoops and found a way to provide SOC analysts with easily searchable, accessible data from SYSCALL ?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 9, 2018, 6:07pm UTC](https://discuss.elastic.co/t/the-perennial-auditd-event-correlation-problem/131202/2 "2018-05-09T18:07:03Z")

</div>

```auto
filter {
  dissect { mapping => { "message" => "type=%{type} msg=audit(%{timestamp}.%{millifraction}:%{something}): %{restofline}" } }
  mutate { add_field => { "auditID" => "%{timestamp}.%{millifraction}:%{something}" } }
  date { match => ["timestamp", "UNIX"] target => "audittime" }
  kv { source => "restofline" target => "kvps" }

  if [type] in ["SYSCALL", "EXECVE", "CWD", "PATH"] {
    aggregate {
      task_id => "%{auditID}"
      code => "
        t = event.get('type')
        if t == 'SYSCALL'
          map['timestamp'] = event.get('audittime');
          map['SYSCALL'] = event.get('kvps');
          map['PATH'] = Array.new
        end
        if t == 'EXECVE'
          map['EXECVE'] = event.get('kvps')
        end
        if t == 'CWD'
          map['CWD'] = event.get('kvps')
        end
        if t == 'PATH'
          map['PATH'] << event.get('kvps')
        end"
      push_map_as_event_on_timeout => true
      timeout_task_id_field => "auditID"
      timeout => 10 # 10 seconds
      timeout_code => "
        event.set('SYSCALL', map['SYSCALL']);
        event.set('EXECVE', map['EXECVE']);
        event.set('CWD', map['CWD']);
        event.set('PATH', map['PATH'])"
   }
   mutate { remove_field => ["message", "kvps"] }
# drop {}
  }
}

```

---

<div class="post-metadata">

**Author:** ![0xeb0de](https://avatars.discourse-cdn.com/v4/letter/0/82dd89/32.png) [@0xeb0de](https://discuss.elastic.co/u/0xeb0de)\
**Post date:** [May 10, 2018, 8:48am UTC](https://discuss.elastic.co/t/the-perennial-auditd-event-correlation-problem/131202/3 "2018-05-10T08:48:35Z")

</div>

Thanks, Badger,

I'll have a play around with that.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 7, 2018, 8:48am UTC](https://discuss.elastic.co/t/the-perennial-auditd-event-correlation-problem/131202/4 "2018-06-07T08:48:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.