# Input S3 with debug mode (BIG POST)

**URL:** <https://discuss.elastic.co/t/input-s3-with-debug-mode-big-post/124127>\
**Category:** Logstash\
**Created:** [March 15, 2018, 3:14pm UTC](https://discuss.elastic.co/t/input-s3-with-debug-mode-big-post/124127 "2018-03-15T15:14:04Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Juan\_Andres\_Ramirez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_andres_ramirez/32/9467_2.png) [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\
**Post date:** [March 15, 2018, 3:14pm UTC](https://discuss.elastic.co/t/input-s3-with-debug-mode-big-post/124127/1 "2018-03-15T15:14:04Z")

</div>

Hello ,  
For a couple days I have been working with logstash 6.2 with docker, using the official docker's image for try get S3 file with logs, I'll try describe all the process with lot details, because I can't do get logs with logstash.

1- The following config I have been using to format files, basically I'm using grok filter, previously check it, in the grok tester online ([http://grokconstructor.appspot.com](http://grokconstructor.appspot.com/do/match))

pipeline.yml

```
input {
      s3 {
        bucket => "e-api-logs"
        prefix => "api/test/*.txt"
        interval => 30
        access_key_id => "xxxxxxx"
        secret_access_key => "xxxxx"
      }
    }

output {
      if "-grokparsefailure" not in [tags] {
        elasticsearch {
          hosts => ["http://elastic-svc:9200"]
          index => "e-api-%{+YYYY.MM.dd}"
        }
        stdout { codec => rubydebug }
      }
    }

```

`

Is not necessary show the grok filter because I don't have errors with it.

I ran the docker with logstash in debug mode:

1. 
  - Starting logstash:

2. 
  - S3 plugin:

3. 
  - Connecting to S3

Everything looks good for now, but I have 2 problems, the first I don't have any output (remember it has rubydebug activated) and I have an error in debug mode:

```
[DEBUG][logstash.instrument.periodicpoller.cgroup] Error, cannot retrieve cgroups information {:exception=>"Errno::ENOENT", :message=>"No such file or directory - /sys/fs/cgroup/cpuacct/kubepods/besteffort/pod00ce1eec-2858-11e8-bc5c-0297d68e3126/5e404648a476ab9f4667f6d09bee669bc939bd8318a227ee6737d8a48c8f5cb8/cpuacct.usage"}
[2018-03-15T13:52:35,257][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2018-03-15T13:52:35,259][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2018-03-15T13:52:37,063][DEBUG][logstash.pipeline] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x2d382e65 sleep>"}

```

It has repeat at the end of logs.

Any help I'll appreciate, thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 12, 2018, 3:14pm UTC](https://discuss.elastic.co/t/input-s3-with-debug-mode-big-post/124127/2 "2018-04-12T15:14:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.