# Ask for a help. Does the logstash output s3 plugin support getting data from Kafka?

**URL:** <https://discuss.elastic.co/t/ask-for-a-help-does-the-logstash-output-s3-plugin-support-getting-data-from-kafka/292084>\
**Category:** Logstash\
**Created:** [December 16, 2021, 6:45am UTC](https://discuss.elastic.co/t/ask-for-a-help-does-the-logstash-output-s3-plugin-support-getting-data-from-kafka/292084 "2021-12-16T06:45:41Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![shaojielinux](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaojielinux/32/99095_2.png) [@shaojielinux](https://discuss.elastic.co/u/shaojielinux)\
**Post date:** [December 16, 2021, 6:45am UTC](https://discuss.elastic.co/t/ask-for-a-help-does-the-logstash-output-s3-plugin-support-getting-data-from-kafka/292084/1 "2021-12-16T06:45:41Z")

</div>

Ask for a help. Does the logstash output s3 plugin support getting data from Kafka? Below is my configuration

input {  
kafka {  
bootstrap\_servers =\> "10.88.14.172:9092,10.88.6.9:9092,10.88.10.166:9092"  
#topics =\> ["k8s"]  
#group\_id =\> "k8s1"  
topics =\> ["fluent-log"]  
#topics\_pattern =\> ".\*"  
group\_id =\> "k8s2"  
#如果使用元数据就不能使用下面的byte字节序列化，否则会报错  
#key\_deserializer\_class =\> "org.apache.kafka.common.serialization.ByteArrayDeserializer"  
#value\_deserializer\_class =\> "org.apache.kafka.common.serialization.ByteArrayDeserializer"  
consumer\_threads =\> 4  
#默认为false，只有为true的时候才会获取到元数据  
decorate\_events =\> true  
auto\_offset\_reset =\> "latest"  
codec =\> "json"  
}  
}  
output {  
s3{  
access\_key\_id =\> ""  
secret\_access\_key =\> ""  
region =\> ""  
bucket =\> ""  
prefix =\> "%{[container\_name]}/%{+YYYY}/%{+MM}/%{+dd}"  
temporary\_directory =\> "/tmp/logstash\_s3"  
encoding =\> "gzip"  
codec =\> "plain"  
rotation\_strategy =\> "time"  
time\_file =\> 5  
}  
}  
[root@ip-10-88-126-115 bin]#

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [December 16, 2021, 3:53pm UTC](https://discuss.elastic.co/t/ask-for-a-help-does-the-logstash-output-s3-plugin-support-getting-data-from-kafka/292084/2 "2021-12-16T15:53:15Z")

</div>

The Kafka input plugin can read events from Kafka, placing the events into the pipeline's queue. The pipeline workers pick up those events and process them as instructed, including routing to output plugins like the s3 output.

No plugin has knowledge of the other plugins in the pipeline.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 13, 2022, 3:53pm UTC](https://discuss.elastic.co/t/ask-for-a-help-does-the-logstash-output-s3-plugin-support-getting-data-from-kafka/292084/3 "2022-01-13T15:53:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.