# Logstash output not working when using logstash-s3-plugin

**URL:** <https://discuss.elastic.co/t/logstash-output-not-working-when-using-logstash-s3-plugin/101404>\
**Category:** Logstash\
**Created:** [September 21, 2017, 11:18pm UTC](https://discuss.elastic.co/t/logstash-output-not-working-when-using-logstash-s3-plugin/101404 "2017-09-21T23:18:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![dwdw](https://avatars.discourse-cdn.com/v4/letter/d/b5e925/32.png) [@dwdw](https://discuss.elastic.co/u/dwdw)\
**Post date:** [September 21, 2017, 11:18pm UTC](https://discuss.elastic.co/t/logstash-output-not-working-when-using-logstash-s3-plugin/101404/1 "2017-09-21T23:18:40Z")

</div>

Configuration with input file from local file system writes to output file and output elasticsearch as expected. When using logtash-input-s3 input{s3{ etc }} the output to elasticsearch and file only write one of the files/output from the s3 bucket. I have run the configuration in --debug mode and see that all the files in the s3 bucket with the given prefix are in fact parsed and processed but I only get one line in the debug trace that indicates "received output" and only one line much later in the trace that the output event is written to file (same with elasticsearch config).

It appears that what ever triggers the event to write is not being called, not sure why the s3 plugin would have anything to do with this . I have tried with logstash 5.2.1 and 5.4.x and logstash-input-s3 v 3.1.2 and 3.1.4.

Any good reason that would happen?

Thanks!

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 21, 2017, 11:57pm UTC](https://discuss.elastic.co/t/logstash-output-not-working-when-using-logstash-s3-plugin/101404/2 "2017-09-21T23:57:49Z")

</div>

Please show your config.

---

<div class="post-metadata">

**Author:** ![dwdw](https://avatars.discourse-cdn.com/v4/letter/d/b5e925/32.png) [@dwdw](https://discuss.elastic.co/u/dwdw)\
**Post date:** [September 22, 2017, 1:21pm UTC](https://discuss.elastic.co/t/logstash-output-not-working-when-using-logstash-s3-plugin/101404/3 "2017-09-22T13:21:03Z")

</div>

Input{  
S3{  
Region=\>"myRegion"  
Bucket=\>"myBucket"  
Prefix=\>"my prefix"  
Codec=\>multiline{  
Pattern=\>"mypattern"  
What=\>"previous"  
Negate=\>"true"  
Auto\_flush\_interval=\>1  
}  
Sincedb\_path=\>"/dev/null"  
}  
}  
Filter{  
Xml{  
Store\_xml=\>false  
Source=\>"message"  
Remove\_namespace=\>true  
Force\_array=\>false  
Xpath=\>[....works]  
Suppress\_empty=\>true  
}  
Mutate{  
Remove\_field=\>["fld1"]  
Replace=\>{"myfld"=\> ....this works}  
}  
}  
Output{  
File{  
Path=\>"/path/my file.txt"  
}  
}

Please ignore the case...  
Input from file with same filters output as expected.

---

<div class="post-metadata">

**Author:** ![dwdw](https://avatars.discourse-cdn.com/v4/letter/d/b5e925/32.png) [@dwdw](https://discuss.elastic.co/u/dwdw)\
**Post date:** [September 22, 2017, 5:31pm UTC](https://discuss.elastic.co/t/logstash-output-not-working-when-using-logstash-s3-plugin/101404/4 "2017-09-22T17:31:05Z")

</div>

Withdrawn...s3 content was not the same as local filesystem content and were parsed as specified.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 20, 2017, 5:31pm UTC](https://discuss.elastic.co/t/logstash-output-not-working-when-using-logstash-s3-plugin/101404/5 "2017-10-20T17:31:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.