# \[resolved\] No AWS ELB logs in the elasticsearch

**URL:** <https://discuss.elastic.co/t/resolved-no-aws-elb-logs-in-the-elasticsearch/170266>\
**Category:** Logstash\
**Created:** [February 28, 2019, 2:13am UTC](https://discuss.elastic.co/t/resolved-no-aws-elb-logs-in-the-elasticsearch/170266 "2019-02-28T02:13:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![divyakaushik](https://avatars.discourse-cdn.com/v4/letter/d/dec6dc/32.png) [@divyakaushik](https://discuss.elastic.co/u/divyakaushik)\
**Post date:** [February 28, 2019, 2:13am UTC](https://discuss.elastic.co/t/resolved-no-aws-elb-logs-in-the-elasticsearch/170266/1 "2019-02-28T02:13:44Z")

</div>

Stack Version: 6.6

I am not able to see logs in elasticsearch/kibana, even though they get successfully pasred in logstash.

In the logs, I can see that the logstash is able to get logs from the ELB. It also prints output received but I don't see it in the elasticsearch. First, I added template for filebeat and I am able to get data from syslogs. After adding ELB logs, I didn't see the logs in the ES so I created another index named elb-6.6.1 using same index\_template as filebeat (thinking that may it needs new index, still no luck). It will be of great help if you can tell me what I am doing wrong. Does ELB needs new index or it should be able to use filebeat index?

logs removing actual data:  
S3 input: Found key {:key=\>"buket/prefix/s3\_file.gz"}  
Using default generated file for the sincedb  
S3 input: Adding to objects {:key=\>"buket/prefix/s3\_file.gz"}  
objects length is: {:length=\>1}  
...  
...  
S3 input: Download remote file {:remote\_key=\>"bucket/prefix/file.log", :local\_filename=\>"tmp\_file.log"}  
Processing file {:filename=\>"tmp.log"}  
filter received {"event"=\>{"message"=\>"log\_string"}}  
Running grok filter {:event=\>#LogStash::Event:0x7c3d00e}  
Event now: {:event=\>#LogStash::Event:0x7c3d00e}  
output received {"event"=\>key,values}

Below are my configs:

input {  
s3 {  
bucket =\> "bucket"  
prefix =\> "prefix"  
region =\> "region"  
type =\> "elblogs"  
codec =\> plain  
add\_field =\> {  
"[@metadata][type]" =\> "elb"  
}  
}  
}

filter {  
if [type] == "elblogs" {  
grok {  
match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} %{NOTSPACE:elb\_name} %{IP:elb\_client\_ip}:%{NUMBER:elb\_client\_port:int} (?:%{IP:elb\_backend\_ip}:%{NUMBER:elb\_backend\_port:int}|-) %{NUMBER:request\_processing\_time:float} %{NUMBER:backend\_processing\_time:float} %{NUMBER:response\_processing\_time:float} (?:%{NUMBER:elb\_status\_code:int}|-) (?:%{NUMBER:backend\_status\_code:int}|-) %{NUMBER:elb\_received\_bytes:int} %{NUMBER:elb\_sent\_bytes:int} "(?:%{WORD:verb}|-) (?:%{GREEDYDATA:request}|-) (?:HTTP/%{NUMBER:httpversion}|-( )?)" "%{DATA:userAgent}"( %{NOTSPACE:ssl\_cipher} %{NOTSPACE:ssl\_protocol})?"]  
match =\> ["message", "%{GREEDYDATA:event\_name} for ELB: %{NOTSPACE:elb\_name} at %{TIMESTAMP\_ISO8601:timestamp}"]  
}  
if [elb\_request] =~ /.+/ {  
grok {  
match =\> ["elb\_request", "(?:%{WORD:http\_method}) (?:%{DATA:http\_path})? (?:%{DATA:http\_type}/%{NUMBER:http\_version:float})?|%{GREEDYDATA:rawrequest}"]  
}  
}  
if [http\_path] =~ /.+/ {  
grok {  
match =\> ["http\_path", "(?:%{WORD:http\_path\_protocol}://)?(%{NOTSPACE:http\_path\_site}:)?(?:%{NUMBER:http\_path\_port:int})?(?:%{GREEDYDATA:http\_path\_url})?"]  
}  
}  
geoip {  
source =\> "elb\_client\_ip"  
}  
}  
date {  
match =\> ["timestamp", "ISO8601"]  
}  
}

output {  
output {  
elasticsearch {  
hosts =\> ["ip:port"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "elb-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}

---

<div class="post-metadata">

**Author:** ![divyakaushik](https://avatars.discourse-cdn.com/v4/letter/d/dec6dc/32.png) [@divyakaushik](https://discuss.elastic.co/u/divyakaushik)\
**Post date:** [February 28, 2019, 8:13pm UTC](https://discuss.elastic.co/t/resolved-no-aws-elb-logs-in-the-elasticsearch/170266/2 "2019-02-28T20:13:20Z")

</div>

This is resolved. I didn't see that data in the current timestamp as the elb logs were old and they were indexed at the log date. Now I can see the data in the kibana.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 28, 2019, 8:13pm UTC](https://discuss.elastic.co/t/resolved-no-aws-elb-logs-in-the-elasticsearch/170266/3 "2019-03-28T20:13:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.