# Have logstash output AWS s3 ELB log to multiple elastic indexes

**URL:** <https://discuss.elastic.co/t/have-logstash-output-aws-s3-elb-log-to-multiple-elastic-indexes/62108>\
**Category:** Logstash\
**Created:** [October 3, 2016, 9:20pm UTC](https://discuss.elastic.co/t/have-logstash-output-aws-s3-elb-log-to-multiple-elastic-indexes/62108 "2016-10-03T21:20:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![meathouse](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/meathouse/32/10343_2.png) [@meathouse](https://discuss.elastic.co/u/meathouse)\
**Post date:** [October 3, 2016, 9:20pm UTC](https://discuss.elastic.co/t/have-logstash-output-aws-s3-elb-log-to-multiple-elastic-indexes/62108/1 "2016-10-03T21:20:29Z")

</div>

I have logstash properly working to send ELB logs in s3 to a single elasticsearch index. Relevant configs are shown below. This works great. But now I'd like to filter some of the ELB logs to specific indexes. Let's say part of the "message" string contains the phrase "company-A", and I want those to go to an index called "elb-company-a-%{+YYYY.MM.dd}", while everything else simply goes to "elb-%{+YYYY.MM.dd}".

Do I need to do this in the filter, or in the output, or both?

IN

> input {  
> s3 {  
> type =\> "elb"  
> bucket =\> "MY-elb-logs"  
> prefix =\> "MY-S3-BUCKET"  
> region =\> "us-east-1"  
> use\_ssl =\> "false"  
> delete =\> "false"  
> interval =\> "120"  
> temporary\_directory =\> "/storage/elk-storage/logstash/"  
> sincedb\_path =\> "/var/log/logstash/my\_last\_elb\_s3\_file"  
> codec =\> plain {charset =\> "US-ASCII"}  
> }  
> }

FILTER

> filter {  
> if [type] == "elb" {  
> grok {  
> match =\> ['message', '%{TIMESTAMP\_ISO8601:timestamp} %{NOTSPACE:elb\_name} %{IP:client\_ip}:%{NUMBER:client\_port} %{IP:backend\_ip}:%{NUMBER:backend\_port} %{NUMBER:request\_processing\_time} %{NUMBER:backend\_processing\_time} %{NUMBER:response\_processing\_time} (?:%{NUMBER:elb\_status\_code}|-) (?:%{NUMBER:backend\_status\_code}|-) %{NUMBER:elb\_received\_bytes} %{NUMBER:elb\_sent\_bytes} (?:%{QS:elb\_request}|-) (?:%{QS:userAgent}|-) (?:%{NOTSPACE:elb\_sslcipher}|-) (?:%{NOTSPACE:elb\_sslprotocol}|-)']  
> }  
> date {  
> match =\> ["timestamp", "ISO8601"]  
> }  
> # Add geolocalization attributes based on ip.  
> geoip {  
> source =\> "client\_ip"  
> target =\> "geoip"  
> database =\> "/etc/logstash/GeoLiteCity.dat"  
> add\_field =\> ["[geoip][coordinates]", "%{[geoip][longitude]}" ]  
> add\_field =\> ["[geoip][coordinates]", "%{[geoip][latitude]}" ]  
> }  
> mutate {  
> convert =\> ["[geoip][coordinates]", "float"]  
> }  
> }  
> }

OUT

> output {  
> if [type] == "elb" {  
> elasticsearch {  
> hosts =\> ["\<% @elk\_nodes.each do |elk\_node| -%\>\<%= elk\_node['ipaddress'] -%\>\<% end -%\>:9200"]  
> sniffing =\> true  
> manage\_template =\> false  
> index =\> "elb-%{+YYYY.MM.dd}"  
> document\_type =\> "%{[@metadata][type]}"  
> }  
> }

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 4, 2016, 3:13am UTC](https://discuss.elastic.co/t/have-logstash-output-aws-s3-elb-log-to-multiple-elastic-indexes/62108/2 "2016-10-04T03:13:01Z")

</div>

In the ouput, as you have there.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:35am UTC](https://discuss.elastic.co/t/have-logstash-output-aws-s3-elb-log-to-multiple-elastic-indexes/62108/3 "2017-07-06T04:35:59Z")

</div>