# Need help for parsing s3 logs to ELK dashboard

**URL:** <https://discuss.elastic.co/t/need-help-for-parsing-s3-logs-to-elk-dashboard/131120>\
**Category:** Logstash\
**Created:** [May 9, 2018, 7:51am UTC](https://discuss.elastic.co/t/need-help-for-parsing-s3-logs-to-elk-dashboard/131120 "2018-05-09T07:51:18Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abdur\_Raqeeb1](https://avatars.discourse-cdn.com/v4/letter/a/fbc32d/32.png) [@Abdur\_Raqeeb1](https://discuss.elastic.co/u/Abdur_Raqeeb1)\
**Post date:** [May 9, 2018, 7:51am UTC](https://discuss.elastic.co/t/need-help-for-parsing-s3-logs-to-elk-dashboard/131120/1 "2018-05-09T07:51:19Z")

</div>

please let us know how can i configure the logstash for multiple buckets and also please let me know for parsing the logs from multiple directories of buckets

Below is a configuration file for particular bucket and particular directories.

input {  
s3 {  
bucket =\> "file-service-access-logs"  
prefix =\> "access-logs/AWSLogs/227866648348/elasticloadbalancing/ap-south-1/2018/04/21/"  
region =\> "ap-south-1"  
type =\> "elblogs"  
codec =\> plain  
secret\_access\_key =\> "myaccesskey"  
access\_key\_id =\> "muid"  
}  
}  
filter {  
if [type] == "elblogs" {  
grok {  
match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} %{NOTSPACE:elb\_name} %{IP:elb\_client\_ip}:%{INT:elb\_client\_port:int} (?:%{IP:elb\_backend\_ip}:%{NUMBER:elb\_backend\_port:int}|-) %{NUMBER:request\_processing\_time:float} %{NUMBER:backend\_processing\_time:float} %{NUMBER:response\_processing\_time:float} (?:%{INT:elb\_status\_code:int}|-) (?:%{INT:backend\_status\_code:int}|-) %{INT:elb\_received\_bytes:int} %{INT:elb\_sent\_bytes:int} "(?:%{GREEDYDATA:elb\_request}|-)" "(?:%{GREEDYDATA:userAgent}|-)" %{NOTSPACE:elb\_sslcipher} %{NOTSPACE:elb\_sslprotocol}"]  
match =\> ["message", "%{GREEDYDATA:event\_name} for ELB: %{NOTSPACE:elb\_name} at %{TIMESTAMP\_ISO8601:timestamp}"]  
}  
if [elb\_request] =~ /.+/ {  
grok {  
match =\> ["elb\_request", "(?:%{WORD:http\_method}) (?:%{DATA:http\_path})? (?:%{DATA:http\_type}/%{NUMBER:http\_version:float})?|%{GREEDYDATA:rawrequest}"]  
}  
}  
if [http\_path] =~ /.+/ {  
grok {  
match =\> ["http\_path", "(?:%{WORD:http\_path\_protocol}://)?(%{NOTSPACE:http\_path\_site}:)?(?:%{NUMBER:http\_path\_port:int})?(?:%{GREEDYDATA:http\_path\_url})?"]  
}  
}  
geoip {  
source =\> "elb\_client\_ip"  
}  
}  
date {  
match =\> ["timestamp", "ISO8601"]  
}  
}  
output {  
stdout { codec =\> rubydebug }  
elasticsearch {  
hosts =\> ["localhost:9200"]  
index =\> "elb"  
}  
}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2018, 7:51am UTC](https://discuss.elastic.co/t/need-help-for-parsing-s3-logs-to-elk-dashboard/131120/2 "2018-06-06T07:51:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.