# Error publishing events (retrying): EOF **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [November 8, 2016, 4:24pm UTC](https://discuss.elastic.co/t/error-publishing-events-retrying-eof/65378 "2016-11-08T16:24:14Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![zoram](https://avatars.discourse-cdn.com/v4/letter/z/ebca7d/32.png) [@zoram](https://discuss.elastic.co/u/zoram)\ **Post date:** [November 8, 2016, 4:24pm UTC](https://discuss.elastic.co/t/error-publishing-events-retrying-eof/65378/1 "2016-11-08T16:24:14Z")
Dear All, File beat is not sending events to logstash after starting my this command. ``` sudo ./filebeat -e -c filebeat-logstash.yml -d "publish" ``` It thrown an error messages.. ``` single.go:77: INFO Error publishing events (retrying): EOF 2016/11/08 16:09:04.700334 single.go:154: INFO send fail ``` I'm using Filebeat -\> Logstash -\> Elasticsearch -\> Kibana to have an overview of my glassfish log file. Logstash+Elasticsearch+Kibana is in ELK Docker Container. Here is my logstash output [http://pastebin.com/FP10pdq2](http://pastebin.com/FP10pdq2) My Filebeat configuration [http://pastebin.com/kcHf2h1H](http://pastebin.com/kcHf2h1H) My Logstash configurations in ELK docker container. ``` root@4569cf1f66ab:/etc/logstash/conf.d# 01-lumberjack-input.conf.backup 02-beats-input.conf 10-syslog.conf 11- nginx.backup 30-output.conf ``` 02-beats-input.conf ``` input { beats { port => 5044 ssl => false #ssl_certificate => "/etc/pki/tls/certs/logstash-beats.crt" #ssl_key => "/etc/pki/tls/private/logstash-beats.key"}} ``` 10-syslog.conf ``` filter { if [type] == "log" { grok { match => { "message", "(?m)\[\#\|%{TIMESTAMP_ISO8601:timestamp}\|%{LOGLEVEL:Log Level}\|% {DATA:server_version}\|%{JAVACLASS:Class}\|%{DATA:thread}\|%{DATA:message_detail}\|\#\]" } add_field => ["Log level", "%{LOGLEVEL:Log Level}"] } } syslog_pri { } date { match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } ``` Any idea where i'm going wrong? Thank you for your help. Best Regards, Thomas ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [November 8, 2016, 5:20pm UTC](https://discuss.elastic.co/t/error-publishing-events-retrying-eof/65378/2 "2016-11-08T17:20:24Z")
```auto { port => 5044 ssl => false #ssl_certificate => "/etc/pki/tls/certs/logstash-beats.crt" #ssl_key => "/etc/pki/tls/private/logstash-beats.key"}} ``` Is your logstash even running? The input config is clearly invalid (as suggested by logstash logs), because you accidentaly commented out the closing `}` by commenting out `ssl_key`. Better try ```auto input { beats { port => 5044 ssl => false #ssl_certificate => "/etc/pki/tls/certs/logstash-beats.crt" #ssl_key => "/etc/pki/tls/private/logstash-beats.key" } } ``` While taking some more space, it's always a good practice to write clean configs. ---
**Author:** ![zoram](https://avatars.discourse-cdn.com/v4/letter/z/ebca7d/32.png) [@zoram](https://discuss.elastic.co/u/zoram)\ **Post date:** [November 11, 2016, 11:04am UTC](https://discuss.elastic.co/t/error-publishing-events-retrying-eof/65378/3 "2016-11-11T11:04:38Z")
Hi @steffens, my input config look like now ``` input { beats { port => 5044 ssl => false #ssl_certificate => "/etc/pki/tls/certs/logstash-beats.crt" #ssl_key => "/etc/pki/tls/private/logstash-beats.key" } } ``` and 10-syslog.conf ``` filter { if [type] == "log" { grok { match => { "message", "(?m)\[\#\|%{TIMESTAMP_ISO8601:timestamp}\|%{LOGLEVEL:Log Level}\|% {DATA:server_version}\|%{JAVACLASS:Class}\|%{DATA:thread}\|%{DATA:message_detail}\|\#\]" } add_field => ["Log level", "%{LOGLEVEL:Log Level}"] } } syslog_pri { } date { match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } ``` I still have the same error.. ``` 2016/11/11 10:57:52.972269 output.go:87: DBG output worker: publish 2048 events 2016/11/11 10:57:52.973354 single.go:77: INFO Error publishing events (retrying): EOF 2016/11/11 10:57:52.973364 single.go:154: INFO send fail ``` After using with filebeat-5.0 version... ``` 2016/11/11 13:44:45.911161 single.go:91: INFO Error publishing events (retrying): EOF 2016/11/11 13:45:00.752791 logp.go:230: INFO Non-zero metrics in the last 30s: filebeat.harvester.started=1 libbeat.logstash.publish.read_errors=5 libbeat.logstash.publish.write_bytes=5547 libbeat.logstash.call_count.PublishEvents=5 libbeat.logstash.published_but_not_acked_events=10235 libbeat.publisher.published_events=2047 filebeat.harvester.open_files=1 filebeat.harvester.running=1 2016/11/11 13:45:01.913866 sync.go:85: ERR Failed to publish events caused by: EOF ``` Yes, my logstash is running. but still the same error messages [http://pastebin.com/Hh9ECFjd](http://pastebin.com/Hh9ECFjd) ``` ==> /var/log/logstash/logstash.log <== elk_1 | {:timestamp=>"2016-11-11T10:56:43.166000+0000", :message=>"fetched an invalid config", :config=>"input {\n lumberjack {\n port => 5000\n type => \"log\"\n ssl => false\n #ssl_certificate => \"/etc/pki/tls/certs/logstash-forwarder.crt\"\n #ssl_key => \"/etc/pki/tls/private/logstash-forwarder.key\"\n }\n}\n\ninput {\n beats {\n port => 5044\n ssl => false\n #ssl_certificate => \"/etc/pki/tls/certs/logstash-beats.crt\"\n #ssl_key => \"/etc/pki/tls/private/logstash-beats.key\"\n }\n}\n\nfilter {\n if [type] == \"log\" {\n grok {\n match => { \"message\", \"(?m)\\[\\#\\|%{TIMESTAMP_ISO8601:timestamp}\\|%{LOGLEVEL:Log Level}\\|%{DATA:server_version}\\|%{JAVACLASS:Class}\\|%{DATA:thread}\\|%{DATA:message_detail}\\|\\#\\]\"}\n add_field => [\"Log level\", \"%{LOGLEVEL:Log Level}\"]\n }\n }\n syslog_pri { }\n date {\n match => [\"timestamp\", \"MMM d HH:mm:ss\", \"MMM dd HH:mm:ss\"]\n }\n}\n\n\nfilter {\n if [type] == \"nginx-access\" {\n grok {\n match => { \"message\" => \"%{NGINXACCESS}\" }\n }\n }\n}\n\noutput {\n elasticsearch {\n hosts => [\"http://localhost:9200\"]\n sniffing => true\n manage_template => false\n index => \"%{[@metadata][beat]}-%{+YYYY.MM.dd}\"\n document_type => \"%{[@metadata][type]}\"\n }\n}\n\n", :reason=>"Expected one of #, => at line 23, column 27 (byte 459) after filter {\n if [type] == \"log\" {\n grok {\n match => { \"message\"", :level=>:error} ``` I have check also my "10-syslog.conf" filter file, seem to be ok? Cheers, ThomasK ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [November 14, 2016, 1:07pm UTC](https://discuss.elastic.co/t/error-publishing-events-retrying-eof/65378/4 "2016-11-14T13:07:18Z")
I think a closing } is missing in your filter config. But didn't validate. As the logs tell there seems to be something wrong with the logstash config. ---
**Author:** ![zoram](https://avatars.discourse-cdn.com/v4/letter/z/ebca7d/32.png) [@zoram](https://discuss.elastic.co/u/zoram)\ **Post date:** [November 15, 2016, 11:44am UTC](https://discuss.elastic.co/t/error-publishing-events-retrying-eof/65378/5 "2016-11-15T11:44:26Z")
it work now. After changing the filter with the following ``` match => { "message", ``` to ``` match => { "message" => ``` and removed "curly brackets in the date match". ``` filter { if [type] == "log" { grok { match => { "message" => "(?m)\[\#\|%{TIMESTAMP_ISO8601:timestamp}\|%{LOGLEVEL:log_level}\|% {DATA:server_version}\|%{JAVACLASS:Class}\|%{DATA:thread}\|%{DATA:message_detail}\|\#\]" } } } syslog_pri { } date { match => ["timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 13, 2016, 11:44am UTC](https://discuss.elastic.co/t/error-publishing-events-retrying-eof/65378/6 "2016-12-13T11:44:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.