# Filebeat log sending problem

**URL:** https://discuss.elastic.co/t/filebeat-log-sending-problem/62244
**Category:** Beats
**Tags:** filebeat
**Created:** [October 5, 2016, 8:42am UTC](https://discuss.elastic.co/t/filebeat-log-sending-problem/62244 "2016-10-05T08:42:15Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![abu.sayeed](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)
#### Post date: [October 5, 2016, 8:42am UTC](https://discuss.elastic.co/t/filebeat-log-sending-problem/62244/1 "2016-10-05T08:42:15Z")

</div>

**Server Side**  
#java -version  
java version "1.8.0\_101"

#elasticsearch version

"5.0.0-beta1",

#bin/kibana --version  
5.0.0-beta1

#bin/logstash --version  
logstash 5.0.0-beta1

#/etc/logstash/conf.d/02-beats-input.conf

input {  
beats {  
port =\> 5044  
ssl =\> true  
ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
}  
}

#/etc/logstash/conf.d/10-syslog-filter.conf

filter {  
if [type] == "syslog" {  
grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" }  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]  
}  
syslog\_pri { }  
date {  
match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
}  
}  
}

#/etc/logstash/conf.d/30-elasticsearch-output.conf

output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

#bin/logstash -f /etc/logstash/conf.d/ --config.test\_and\_exit  
Configuration OK

#bin/logstash-plugin install logstash-input-beats  
Installation successful

#bin/logstash-plugin update logstash-input-beats  
Updated logstash-input-beats 3.1.4 to 3.1.6

#bin/logstash-plugin install logstash-output-elasticsearch  
Installation successful

#bin/logstash-plugin update logstash-output-elasticsearch  
Updated logstash-output-elasticsearch 5.1.1 to 5.1.2

#/var/log/logstash/logstash-plain.log

[2016-10-05T14:19:50,717][WARN][logstash.outputs.elasticsearch] Elasticsearch output attempted to sniff for new connections but cannot. No living connections are detected. Pool contains the following current URLs {:url\_info=\>{}}  
[2016-10-05T14:19:51,952]**[ERROR][org.logstash.beats.BeatsHandler] Exception: not an SSL/TLS record:** 325700000001324300000..................f7000000ffffbf7b794e  
[2016-10-05T14:19:55,718][WARN][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=\>#\<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections\>}

**client side**

#bin/filebeat --version  
filebeat version 5.0.0-beta1 (amd64), libbeat 5.0.0-beta1

#/etc/filebeat/filebeat.yml  
filebeat:

```
      prospectors:
  
  
        -
  
  
          paths:
  
  
            - /var/log/auth.log
  
  
            - /var/log/syslog
  
  
          # - /var/log/*.log
  
  
  
          input_type: log
  
  
  
          document_type: syslog
  
  
  
      registry_file: /var/lib/filebeat/registry
  
 
  
    output:
  
  
      logstash:
  
  
        hosts: ["my_elk_server_ip:5044"]
  
  
        bulk_max_size: 1024
  
  
  
        tls:
  
  
          certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
  
  
  
    shipper:
  
  
    logging:
  
  
      files:
  
  
        rotateeverybytes: 10485760 # = 10MB

```

#bin/filebeat -c /etc/filebeat/filebeat.yml -e -v

2016/10/05 08:24:23.710760 beat.go:204: INFO filebeat start running.  
2016/10/05 08:24:23.710783 registrar.go:66: INFO Registry file set to: /var/lib/filebeat/registry  
2016/10/05 08:24:23.710824 registrar.go:99: INFO Loading registrar data from /var/lib/filebeat/registry  
2016/10/05 08:24:23.711077 prospector.go:106: INFO Starting prospector of type: log  
2016/10/05 08:24:23.711312 log.go:60: INFO Harvester started for file: /var/log/syslog  
2016/10/05 08:24:23.711406 spooler.go:64: INFO Starting spooler: spool\_size: 2048; idle\_timeout: 5s  
2016/10/05 08:24:23.711432 registrar.go:178: INFO Starting Registrar  
2016/10/05 08:24:23.711473 sync.go:41: INFO Start sending events to output  
2016/10/05 08:24:23.716637 log.go:60: INFO Harvester started for file: /var/log/auth.log  
2016/10/05 08:24:23.747492 **sync.go:85: ERR Failed to publish events caused by: EOF**  
2016/10/05 08:24:23.747737 **single.go:91: INFO Error publishing events (retrying): EOF**  
2016/10/05 08:24:53.710713 **logp.go:230: INFO Non-zero metrics in the last 30s: filebeat.harvester.running=2 filebeat.harvester.open\_files=2 filebeat.harvester.started=2 libbeat.logstash.published\_but\_not\_acked\_events=5120 libbeat.logstash.call\_count.PublishEvents=5 libbeat.logstash.publish.write\_bytes=2305 libbeat.publisher.published\_events=2046 libbeat.logstash.publish.read\_errors=5**

Can anybody help me for this problem ?  
Thanks in advance.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [October 5, 2016, 10:45am UTC](https://discuss.elastic.co/t/filebeat-log-sending-problem/62244/2 "2016-10-05T10:45:58Z")

</div>

post is quite hard to read. Can you properly format it? beats config files need proper indentation. Without formatting it's hard to see if something is wrong there.

Nothing to see in filebeat logs besides EOF. But check your logstash logs. It's also complaining about elasticsearch output not working. Could it be the logstash pipeline is blocked due blocking outputs?

---

<div class="post-metadata">

### Author: ![abu.sayeed](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)
#### Post date: [October 6, 2016, 5:14am UTC](https://discuss.elastic.co/t/filebeat-log-sending-problem/62244/3 "2016-10-06T05:14:10Z")

</div>

[WARN][logstash.outputs.elasticsearch] Elasticsearch output attempted to sniff for new connections but cannot. No living connections are detected. Pool contains the following current URLs {:url\_info=\>{}}  
[WARN][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=\>#\<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections\>}

[ERROR][org.logstash.beats.BeatsHandler] Exception: not an SSL/TLS record: 32570000000132430000010a785e5c90c14ac3401086d33759fe739aec26da968542bd7a160f1a916dbb491792ddb0333196d27797d428e871be19be99f98bc724491649b2385db0eb2c9ba361037d019f7b0b0d3a531b1aa4d85bc3d0a85d6bf7d630ae29425d93656855aa14ffe79def077e9f697b537496c83493f5a18fa2d808b5d1b2d0b210c37ef03c88f8bdeda8c56b88ae715e50a87934d16e2bfc342b081a9f6d2417fcb6c22653ab4c56109fcbde1d27a0ee6e95f375d8563831f73acfc771cc664376085d8537416c22ff7e768137dd74dba175d64ffc1488ffb26b8a1dbbce129bae874621d56aa9e452de3fc9b52ea52ecb6cb52e5e9082c2100f932eff30316f4393cf495ebf020000ffffc7b57586

/var/log/elasticsearch/elasticsearch.log  
[INFO][o.e.n.Node] [rTbwjik] started  
[INFO][o.e.g.GatewayService] [rTbwjik] recovered [1] indices into cluster\_state  
[INFO][o.e.c.r.a.AllocationService] [rTbwjik] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.kibana][3]] ...]).

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [October 6, 2016, 10:34am UTC](https://discuss.elastic.co/t/filebeat-log-sending-problem/62244/4 "2016-10-06T10:34:35Z")

</div>

Looks like tls / ssl issue. tls is not called ssl in the 5.0 release. You can use the migration script in your download package under scripts to migrate the config.

Can you please format your posts with 3 ticks ` before and after to make them more readable as @steffens requested before ?

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [October 6, 2016, 1:26pm UTC](https://discuss.elastic.co/t/filebeat-log-sending-problem/62244/5 "2016-10-06T13:26:56Z")

</div>

with 5.0.0 beta1 the `tls` section has been renamed to `ssl` for consistency with other projects in elastic stack.

---

<div class="post-metadata">

### Author: ![abu.sayeed](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)
#### Post date: [October 6, 2016, 1:56pm UTC](https://discuss.elastic.co/t/filebeat-log-sending-problem/62244/6 "2016-10-06T13:56:47Z")

</div>

how can I use migration script?

**#cd /usr/share/filebeat/scripts**

**#./migrate\_beat\_config\_1\_x\_to\_5\_0.py**

usage: migrate\_beat\_config\_1\_x\_to\_5\_0.py [-h] [--dry] file  
migrate\_beat\_config\_1\_x\_to\_5\_0.py: error: too few arguments

THanks all for replying

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [October 6, 2016, 1:58pm UTC](https://discuss.elastic.co/t/filebeat-log-sending-problem/62244/7 "2016-10-06T13:58:12Z")

</div>

You just printed out the usage docs.

I can only repeat: Please format your posts.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 26, 2016, 8:42am UTC](https://discuss.elastic.co/t/filebeat-log-sending-problem/62244/8 "2016-10-26T08:42:51Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.