# ELb logs from s3 to ELK server
**URL:** https://discuss.elastic.co/t/elb-logs-from-s3-to-elk-server/133168
**Category:** Logstash
**Created:** [May 24, 2018, 2:14pm UTC](https://discuss.elastic.co/t/elb-logs-from-s3-to-elk-server/133168 "2018-05-24T14:14:11Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@Suresh\_Pal](https://discuss.elastic.co/u/Suresh_Pal)
#### Post date: [May 24, 2018, 2:14pm UTC](https://discuss.elastic.co/t/elb-logs-from-s3-to-elk-server/133168/1 "2018-05-24T14:14:12Z")
I'm trying to ship my elb logs from the s3 bucket to ELK server. I'm getting following error
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/s3\_elb\_logs.conf
WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO] 2018-05-24 09:37:20.484 [main] scaffold - Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"}
[INFO] 2018-05-24 09:37:20.489 [main] scaffold - Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"}
[WARN] 2018-05-24 09:37:20.883 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO] 2018-05-24 09:37:21.069 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=\>"6.2.4"}
[INFO] 2018-05-24 09:37:21.204 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=\>9600}
[ERROR] 2018-05-24 09:37:21.294 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, ", ', -, [, { at line 2, column 12 (byte 24) after input {s3 {\n bucket =\> ", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in`compile\_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in`map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in`initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in`execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in`with\_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in`block in converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in`converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in`block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
.conf file is
input {s3 {
bucket =\> “fcalb-logs”
prefix =\> "/"
region =\> “us-east-1”
type =\> “elblogs”
codec =\> plain
secret\_access\_key =\> “”
access\_key\_id =\> “”
}
}
filter {
grok {
match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} %{NOTSPACE:elb} %{IP:clientip}:%{INT:clientport:int} (?:(%{IP:backendip}:?:%{INT:backendport:int})|-) %{NUMBER:request\_processing\_time:float} %{NUMBER:backend\_processing\_time:float} %{NUMBER:response\_processing\_time:float} (?:-|%{INT:elb\_status\_code:int}) (?:-|%{INT:backend\_status\_code:int}) %{INT:received\_bytes:int} %{INT:sent\_bytes:int} "%{ELB\_REQUEST\_LINE}" "(?:-|%{DATA:user\_agent})" (?:-|%{NOTSPACE:ssl\_cipher}) (?:-|%{NOTSPACE:ssl\_protocol})"]
}
date {
match =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
}
}
output {
elasticsearch { hosts =\> ["localhost:9200"] }
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [May 27, 2018, 8:00pm UTC](https://discuss.elastic.co/t/elb-logs-from-s3-to-elk-server/133168/2 "2018-05-27T20:00:43Z")
If you look carefully you'll see that the quotes around e.g. "fcalb-logs" aren't straight but curly.
[https://practicaltypography.com/straight-and-curly-quotes.html](https://practicaltypography.com/straight-and-curly-quotes.html)
---
### Author:  [@Suresh\_Pal](https://discuss.elastic.co/u/Suresh_Pal)
#### Post date: [May 29, 2018, 6:29am UTC](https://discuss.elastic.co/t/elb-logs-from-s3-to-elk-server/133168/3 "2018-05-29T06:29:23Z")
Thanks @magnusbaeck , It works fine now.
I have one more question,
I'm not able to create index in my kibana though i have that index in my elasticsearch. Below are the steps i'm performing while creating index in kibana.
Step 1 of 2: Define index pattern
Here my index is showing correctly.
Step 2 of 2: Configure settings
after configuring setting and clicking on create index, its not creating index. Please help
 
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [May 29, 2018, 7:45am UTC](https://discuss.elastic.co/t/elb-logs-from-s3-to-elk-server/133168/4 "2018-05-29T07:45:21Z")
For new questions I suggest you create new topics instead of continuing old ones.
> after configuring setting and clicking on create index, its not creating index.
That screen creates **index patterns**. It does not create **indexes**. I'm not sure whether you actually have an issue or if it's just a case of incorrect expectations.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 26, 2018, 7:45am UTC](https://discuss.elastic.co/t/elb-logs-from-s3-to-elk-server/133168/5 "2018-06-26T07:45:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.