# Problem with beats/logstash parsing

**URL:** <https://discuss.elastic.co/t/problem-with-beats-logstash-parsing/43168>\
**Category:** Logstash\
**Created:** [March 1, 2016, 10:26pm UTC](https://discuss.elastic.co/t/problem-with-beats-logstash-parsing/43168 "2016-03-01T22:26:44Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![corey\_jones](https://avatars.discourse-cdn.com/v4/letter/c/958977/32.png) [@corey\_jones](https://discuss.elastic.co/u/corey_jones)\
**Post date:** [March 1, 2016, 10:26pm UTC](https://discuss.elastic.co/t/problem-with-beats-logstash-parsing/43168/1 "2016-03-01T22:26:45Z")

</div>

Hi guys. I have a server running beats and another server that has a docker container running logstash. Both are at aws. It seems that the connection is being made. From my debug.log from logstash I see this.

> {:timestamp=\>"2016-03-01T22:14:56.545000+0000", :message=\>"Beats input: decoding this event with the codec", :target\_field\_value=\>"{"sql":"SELECT \"public\".\"apiv3\_authorizations\".\* FROM \"public\".\"apiv3\_authorizations\" WHERE \"public\".\"apiv3\_authorizations\".\"project\_id\" IS NULL AND \"public\".\"apiv3\_authorizations\".\"user\_id\" IS NULL LIMIT 1","name":"Apiv3::Authorization Load","connection\_id":47308764521820,"statement\_name":null,"binds":,"duration":0.6,"request\_id":"437da433-eaf9-4653-9ddb-e6b5dab1da50","source":"unknown","tags":["request"],"@timestamp":"2016-03-01T22:14:49.395Z","@version":"1"}", :level=\>:debug, :file=\>"logstash/inputs/beats\_support/connection\_handler.rb", :line=\>"55", :method=\>"process"}

This is definitely from my beats server. Although the output never is generated. Nothing ends up in kibana and nothing gets output to a file on the logstash server. Any help would be great.

Here is my logstash config

```
input {
  beats {
    port => 5044
    codec => json_lines
  }
}
output {
  file {
    codec => plain
    path => "/var/log/logstash/logstash.log"
  }
  elasticsearch {
    codec => json_lines
    hosts => ["awsURL:80"]
    ssl => false
    sniffing => false
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

```

and my beats config

```
filebeat:
  # List of prospectors to fetch data.
  prospectors:
    # Each - is a prospector. Below are the prospector specific configurations
    -
      paths:
        #- /var/log/*.log
        - /var/app/current/log/logstasher.log
      encoding: plain
      input_type: log
      scan_frequency: 2s
    
  registry_file: /var/lib/filebeat/registry
output:
  ### Logstash as output
  logstash:
    hosts: ["myip:5044"]
    bulk_max_size: 1024
    path: "/tmp/filebeat"
    filename: filebeat
    rotate_every_kb: 10000
    pretty: true
logging:
  to_files: true
  # To enable logging to files, to_files option has to be set to true
  files:
    # The directory where the log files will written to.
    path: /var/log/filebeat
    # The name of the files where the logs are written to.
    name: filebeat.log
    # Configure log file size limit. If limit is reached, log file will be
    # automatically rotated
    rotateeverybytes: 10485760 # = 10MB
    # Number of rotated log files to keep. Oldest files will be deleted first.
    keepfiles: 5
  # Enable debug output for selected components. To enable all selectors use ["*"]
  # Other available selectors are beat, publish, service
  # Multiple selectors can be chained.
  #selectors: []
  # Sets log level. The default log level is error.
  # Available log levels are: critical, error, warning, info, debug
  level: debug

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 2, 2016, 5:40am UTC](https://discuss.elastic.co/t/problem-with-beats-logstash-parsing/43168/2 "2016-03-02T05:40:49Z")

</div>

> [@corey\_jones](#):
>
> and yes. The normal yml is formatted correctly. Im just having trouble getting it to follow the formatting on here

Use the `</>` formatting button 🙂

---

<div class="post-metadata">

**Author:** ![corey\_jones](https://avatars.discourse-cdn.com/v4/letter/c/958977/32.png) [@corey\_jones](https://discuss.elastic.co/u/corey_jones)\
**Post date:** [March 2, 2016, 1:33pm UTC](https://discuss.elastic.co/t/problem-with-beats-logstash-parsing/43168/3 "2016-03-02T13:33:46Z")

</div>

Thanks fixed. Figured out my problem as well. I did not need "codec =\> json\_lines" in my input.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:08am UTC](https://discuss.elastic.co/t/problem-with-beats-logstash-parsing/43168/4 "2017-07-06T05:08:47Z")

</div>