# S3 input stack trace and multi-pipeline config question

**URL:** <https://discuss.elastic.co/t/s3-input-stack-trace-and-multi-pipeline-config-question/158022>\
**Category:** Logstash\
**Created:** [November 23, 2018, 2:08pm UTC](https://discuss.elastic.co/t/s3-input-stack-trace-and-multi-pipeline-config-question/158022 "2018-11-23T14:08:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![shed7](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@shed7](https://discuss.elastic.co/u/shed7)\
**Post date:** [November 23, 2018, 2:08pm UTC](https://discuss.elastic.co/t/s3-input-stack-trace-and-multi-pipeline-config-question/158022/1 "2018-11-23T14:08:19Z")

</div>

Hello

Had a stack trace appear when using the following configs, there are a few more configs I couldn't fit in this post, but are very similar to the one below. I found including more than 1 config in a single pipeline resulted in events ending up in the wrong output, so I defined a pipeline per AWS flow log bucket.

Does this look like a sane way of managing multiple s3 inputs going to their own syslog output? Pipelines seem well suited otherwise I'd be doing lots of tagging and conditionals in one config. Stack trace at the bottom. Grateful for any thoughts.

pipelines.yml:

```
- pipeline.id: 25da1c4c-pipline
   path.config: "/etc/logstash/conf.d/aws-flowlogs-eu-central-1-25da1c4c.conf"
 - pipeline.id: 06f2f234a6d3b142a-pipline
   path.config: "/etc/logstash/conf.d/aws-flowlogs-eu-central-1-06f2f234a6d3b142a.conf"
 - pipeline.id: 2b47e44e-pipline
   path.config: "/etc/logstash/conf.d/aws-flowlogs-eu-west-1-2b47e44e.conf"
 - pipeline.id: cd8909a9-pipline
   path.config: "/etc/logstash/conf.d/aws-flowlogs-ap-southeast-1-cd8909a9.conf"

```

conf.d/\*

```
input {
    s3 {
	bucket => "flowlogs-ireland-2b47e44e"
	sincedb_path => "/var/lib/logstash/sincedb-flowlogs-ireland-2b47e44e"
	type => "s3"
	region => "eu-west-1"
	exclude_pattern => "(?:(2017\/[0-1])|2018\/0)"
	interval => 300
    }
}

output {
  syslog {
    host => "mylogserver"
    protocol => "ssl-tcp"
    port => 6514
    facility => "local0"
    severity => "informational"
    appname => "aws-flowlogs-eu-west-1-2b47e44e"
    ssl_cacert => "/etc/syslog-ng/ca.d/cacert.pem"
    ssl_cert => "/etc/pki/tls/certs/hostcert.pem"
    ssl_key => "/etc/pki/tls/certs/hostkey.pem"
    sourcehost => "mylogforwarder"
  }
}

logstash[29981]: [8,955][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin.
logstash[29981]: Pipeline_id:2b47e44e-pipline
logstash[29981]: Plugin: <LogStash::Inputs::S3 bucket=>"flowlogs-ireland-2b47e44e", exclude_pattern=>"(?:(2017\\/[0-1])|2018\\/0)", interval=>300, id=
>"c1fbe0cf05e8080b6b81f1190cc700636aa18c7d9bfe8a75b061ab4a6e8a0e72", sincedb_path=>"/var/lib/logstash/sincedb-flowlogs-ireland-2b47e44e", type=>"s3", region=>"eu-west-1", enable_metric=>true, codec=>
<LogStash::Codecs::Plain id=>"plain_93dca249-a400-4c97-9968-b68e8693e255", enable_metric=>true, charset=>"UTF-8">, role_session_name=>"logstash", delete=>false, temporary_directory=>"/tmp/logstash", include_obje
ct_properties=>false>
logstash[29981]: Error:
logstash[29981]: Exception: Java::JavaLang::NullPointerException
logstash[29981]: Stack: org.jruby.RubyString.getStringForPattern(RubyString.java:3741)
logstash[29981]: org.jruby.RubyString.asRegexpArg(RubyString.java:2405)
logstash[29981]: org.jruby.RubyString.subBangNoIter(RubyString.java:2445)
logstash[29981]: org.jruby.RubyString.sub_bang(RubyString.java:2398)
logstash[29981]: org.jruby.RubyString$INVOKER$i$sub_bang.call(RubyString$INVOKER$i$sub_bang.gen)
logstash[29981]: org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrTwoOrNBlock.call(JavaMethod.java:402)
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 21, 2018, 2:11pm UTC](https://discuss.elastic.co/t/s3-input-stack-trace-and-multi-pipeline-config-question/158022/2 "2018-12-21T14:11:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.