# Logstash-output-s3 not working with auditbeat

**URL:** <https://discuss.elastic.co/t/logstash-output-s3-not-working-with-auditbeat/140194>\
**Category:** Logstash\
**Created:** [July 16, 2018, 4:30pm UTC](https://discuss.elastic.co/t/logstash-output-s3-not-working-with-auditbeat/140194 "2018-07-16T16:30:13Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![computermaster0101](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/computermaster0101/32/33389_2.png) [@computermaster0101](https://discuss.elastic.co/u/computermaster0101)\
**Post date:** [July 16, 2018, 4:30pm UTC](https://discuss.elastic.co/t/logstash-output-s3-not-working-with-auditbeat/140194/1 "2018-07-16T16:30:13Z")

</div>

I have setup auditbeat on a centos6 server. it ships to logstash which ships to s3. Below is my config

input {  
beats {  
port =\> 9210  
}  
}  
output {  
s3{  
region =\> "us-east-1"  
bucket =\> "NAME-REMOVED"  
prefix =\> "auditLogs/%{+YYYY}/%{[fields][env]}/%{[fields][stack]}/%{[beat][hostname]}/%{+MM}/%{+dd}/"  
time\_file =\> 60  
}  
}

An object is created in the correct location in the s3 bucket but it contains the following

2018-07-16T05:48:58.036Z {name=SERVER-NAME-REMOVED} %{message}

I've setup WinLogBeat on Windows server and it ships to the same logstash and the messages are coming through. After doing in-depth troubleshooting, i believe the issue is that auditbeat is sending a messages object instead of a message object. Has anyone else experienced this? Should this be a new ticket in github?

ps. I've removed PII from the above

---

<div class="post-metadata">

**Author:** ![computermaster0101](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/computermaster0101/32/33389_2.png) [@computermaster0101](https://discuss.elastic.co/u/computermaster0101)\
**Post date:** [July 23, 2018, 1:54pm UTC](https://discuss.elastic.co/t/logstash-output-s3-not-working-with-auditbeat/140194/2 "2018-07-23T13:54:05Z")

</div>

Good morning community!

I wanted to check in on this as it has been a week. Is there any other information i can supply or any other validations I should complete before opening a ticket for a bug?

---

<div class="post-metadata">

**Author:** ![computermaster0101](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/computermaster0101/32/33389_2.png) [@computermaster0101](https://discuss.elastic.co/u/computermaster0101)\
**Post date:** [July 23, 2018, 8:32pm UTC](https://discuss.elastic.co/t/logstash-output-s3-not-working-with-auditbeat/140194/3 "2018-07-23T20:32:45Z")

</div>

Hey Everyone!

Here is another update. I was able to setup and configure auditd and filebeat. Going this route everything shows up in the s3 bucket as expected. Im not sure if this is a bug in the way auditbeat ships vs how all the other beats ship but i did not change anything in the logstash config.

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [July 24, 2018, 12:33am UTC](https://discuss.elastic.co/t/logstash-output-s3-not-working-with-auditbeat/140194/4 "2018-07-24T00:33:39Z")

</div>

The default format string for the S3 output plugin includes a reference to the `message` field, but if the event being pushed does not have a `message` field, the format string is not expanded.

I'm not familiar with the exact format of AuditBeat, but if you add a Stdout Output Plugin, you may be able to determine the "shape" of the output and coerce it into place:

```auto
output {
  if [message]
    s3 {
      # ...
    }
  } else {
    stdout {
      codec => rubydebug
    }
  }

```

Once you know the "shape" of the events, you can add a _filter_ that will compose your `message` using the [sprintf syntax](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#sprintf):

```auto
filter {
  if not [message] {
    mutate => {
      add_field => {
        "message" => ""
      }
    }
  }
}

```

OR: if auditbeat is sending `messages`, and you want to split these out to be individual `message` objects, you can use the [Split Filter Plugin](https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html):

```auto
filter {
  if [messages] and not [message] {
    split {
      field => "messages"
      target => "message"
      remove_field => "messages"
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 21, 2018, 12:33am UTC](https://discuss.elastic.co/t/logstash-output-s3-not-working-with-auditbeat/140194/5 "2018-08-21T00:33:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.