# Logstash/Beats Issue - Logs not being published

**URL:** <https://discuss.elastic.co/t/logstash-beats-issue-logs-not-being-published/64601>\
**Category:** Beats\
**Created:** [November 1, 2016, 6:11pm UTC](https://discuss.elastic.co/t/logstash-beats-issue-logs-not-being-published/64601 "2016-11-01T18:11:47Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Danxx6](https://avatars.discourse-cdn.com/v4/letter/d/6bbea6/32.png) [@Danxx6](https://discuss.elastic.co/u/Danxx6)\
**Post date:** [November 1, 2016, 6:11pm UTC](https://discuss.elastic.co/t/logstash-beats-issue-logs-not-being-published/64601/1 "2016-11-01T18:11:47Z")

</div>

Hi

I've recently followed a couple of guides to get a ELK stack up and working however I'm having some issues transporting my logs via filebeat to logstash.

Errors:  
2016-11-01T18:11:19Z ERR Failed to publish events caused by: EOF  
2016-11-01T18:11:19Z INFO Error publishing events (retrying): EOF

Filebeat Config:

```
filebeat:
  prospectors:
    -
      paths:
        - /var/log/auth.log
        - /var/log/syslog
      # - /var/log/*.log

      input_type: log

      document_type: syslog

  registry_file: /var/lib/filebeat/registry

output:
  logstash:
    hosts: ["elk:5044"]
    bulk_max_size: 1024

    tls:
      certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

shipper:

logging:
  files:
    rotateeverybytes: 10485760 # = 10MB

```

Logstash Config:

```
input {
  lumberjack {
    port => 5044
    type => "logs"
    ssl_certificate => "/etc/logstash/logstash-forwarder.crt"
    ssl_key => "/etc/logstash/logstash-forwarder.key"
  }
}

```

Does anyone know what's causing this?

Thanks

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [November 1, 2016, 6:39pm UTC](https://discuss.elastic.co/t/logstash-beats-issue-logs-not-being-published/64601/2 "2016-11-01T18:39:14Z")

</div>

What version of Filebeat are you using?

Did you follow the official [Getting Started guide (5.X)](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-getting-started.html) provided by Elastic?

The Logstash input configuration you are using is not correct for use with Beats. See the [Logstash Setup](https://www.elastic.co/guide/en/beats/libbeat/5.0/logstash-installation.html#logstash-setup) docs.

---

<div class="post-metadata">

**Author:** ![Danxx6](https://avatars.discourse-cdn.com/v4/letter/d/6bbea6/32.png) [@Danxx6](https://discuss.elastic.co/u/Danxx6)\
**Post date:** [November 1, 2016, 7:38pm UTC](https://discuss.elastic.co/t/logstash-beats-issue-logs-not-being-published/64601/3 "2016-11-01T19:38:04Z")

</div>

Hi Andrew

Thanks for that, I've now figured it out and got it working. Just need to add the SSL now.

One question, in the tutorial i was following a separate file was created in the /etc/logstash/conf.d/ folder for the syslog filter, do I need to use a separate file or does this go into the logstash.conf file now?

Thanks

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [November 1, 2016, 11:30pm UTC](https://discuss.elastic.co/t/logstash-beats-issue-logs-not-being-published/64601/4 "2016-11-01T23:30:36Z")

</div>

> [@Danxx6](#):
>
> One question, in the tutorial i was following a separate file was created in the /etc/logstash/conf.d/ folder for the syslog filter, do I need to use a separate file or does this go into the logstash.conf file now?

All of the config files in the conf.d directory are concatenated in lexicographical order and then parsed as a single config file. So you can split them up or just have one large file.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 22, 2016, 6:12pm UTC](https://discuss.elastic.co/t/logstash-beats-issue-logs-not-being-published/64601/5 "2016-11-22T18:12:22Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.