# S3 output debug

**URL:** <https://discuss.elastic.co/t/s3-output-debug/131141>\
**Category:** Logstash\
**Created:** [May 9, 2018, 9:22am UTC](https://discuss.elastic.co/t/s3-output-debug/131141 "2018-05-09T09:22:41Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mememe](https://avatars.discourse-cdn.com/v4/letter/m/977dab/32.png) [@mememe](https://discuss.elastic.co/u/mememe)\
**Post date:** [May 9, 2018, 9:22am UTC](https://discuss.elastic.co/t/s3-output-debug/131141/1 "2018-05-09T09:22:41Z")

</div>

Hi, I am quite new to logstash and do run logstash 5 on AWS Linux using s3 output and elasticsearch output like in below snippet from logstash.conf  
The problem I face is that data is not put in s3. Still, the s3 test file can be put in the bucket and sometimes data are put, too, but most of the times it does not work.  
The log file does not show anything critical.  
Now I want to switch logs to debug. I am new to linux and using 'initctl start logstash --log.level error' wont work...  
Can anybody provide some guidance?

here my logstash conf:  
output {  
s3{  
region =\> "us-east-1"  
bucket =\> "aws-xxx"  
codec =\> "json\_lines"  
}  
elasticsearch {  
hosts =\> "search-xxx:443"  
ssl =\> true  
index =\> "dummy"  
}  
}

another thing i noted - although test file is put in s3 the log says nothing about it:

[ec2-user@ip-10-213-72-21 ~]$ more /var/log/logstash/log\*  
[2018-05-09T10:53:43,376][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"}  
[2018-05-09T10:53:43,380][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"}  
[2018-05-09T10:53:43,391][INFO][logstash.setting.writabledirectory] Creating directory {:setting=\>"path.queue", :path=\>"/var/lib/logstash/queue"}  
[2018-05-09T10:53:43,392][INFO][logstash.setting.writabledirectory] Creating directory {:setting=\>"path.dead\_letter\_queue", :path=\>"/var/lib/logstash/dead\_letter\_queue"}  
[2018-05-09T10:53:43,414][INFO][logstash.agent] No persistent UUID file found. Generating new UUID {:uuid=\>"44672d84-c9a0-461b-91f5-a071888ef75e", :path=\>"/var/lib/logstash/uuid"}  
[2018-05-09T10:54:35,893][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>[], :added=\>xxx:443/]}}  
[2018-05-09T10:54:35,896][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>xxx:443/, :path=\>"/"}  
[2018-05-09T10:54:36,061][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"xxx:443/"  
}  
[2018-05-09T10:54:36,121][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil}  
[2018-05-09T10:54:36,137][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"  
=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"  
=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type  
"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"ty  
pe"=\>"half\_float"}}}}}}}}  
[2018-05-09T10:54:36,148][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["//xxx:443"]}  
[2018-05-09T10:54:36,152][INFO][logstash.pipeline] Starting pipeline {"id"=\>"main", "pipeline.workers"=\>2, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>5, "pipeline.max\_inflight"=\>2  
50}  
[2018-05-09T10:54:36,814][INFO][logstash.inputs.beats] Beats inputs: Starting input listener {:address=\>"0.0.0.0:5044"}  
[2018-05-09T10:54:36,817][INFO][logstash.pipeline] Pipeline main started  
[2018-05-09T10:54:36,851][INFO][org.logstash.beats.Server] Starting server on port: 5044  
[2018-05-09T10:54:36,891][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 9, 2018, 5:54pm UTC](https://discuss.elastic.co/t/s3-output-debug/131141/2 "2018-05-09T17:54:19Z")

</div>

> Now I want to switch logs to debug. I am new to linux and using 'initctl start logstash --log.level error' wont work...

You can set the loglevel in logstash.yml.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2018, 5:54pm UTC](https://discuss.elastic.co/t/s3-output-debug/131141/3 "2018-06-06T17:54:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.