# "Failed to publish events...i/o timeout" (but telnet is OK) **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [October 1, 2019, 1:07pm UTC](https://discuss.elastic.co/t/failed-to-publish-events-i-o-timeout-but-telnet-is-ok/201802 "2019-10-01T13:07:09Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![zebu14](https://avatars.discourse-cdn.com/v4/letter/z/aca169/32.png) [@zebu14](https://discuss.elastic.co/u/zebu14)\ **Post date:** [October 1, 2019, 1:07pm UTC](https://discuss.elastic.co/t/failed-to-publish-events-i-o-timeout-but-telnet-is-ok/201802/1 "2019-10-01T13:07:09Z")
Hello, I try to push some log files from filebeat to logstash. Everything is working from the server A (filebeat) to Logstash Then I tried to push same logs from server B (filebeat) to Logstash and I have these errors : ``` 2019-10-01T11:04:26.802+0200 INFO pipeline/output.go:95 Connecting to backoff(async(tcp://10.118.120.65:1762)) 2019-10-01T11:04:26.806+0200 INFO pipeline/output.go:105 Connection to backoff(async(tcp://10.118.120.65:1762)) established 2019-10-01T11:04:53.764+0200 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":10,"time":{"ms":13}},"total":{"ticks":180,"time":{"ms":184},"value":0},"user":{"ticks":170,"time":{"ms":171}}},"handles":{"limit":{"hard":65535,"soft":65535},"open":8},"info":{"ephemeral_id":"26170285-e93b-4c2b-a210-5ba6fc1f7576","uptime":{"ms":30018}},"memstats":{"gc_next":33498480,"memory_alloc":16922056,"memory_total":40137952,"rss":44019712},"runtime":{"goroutines":25}},"filebeat":{"events":{"active":4117,"added":4119,"done":2},"harvester":{"open_files":1,"running":1,"started":1}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"events":{"active":2048,"batches":1,"total":2048},"type":"logstash"},"pipeline":{"clients":1,"events":{"active":4117,"filtered":2,"published":4116,"retry":2048,"total":4119}}},"registrar":{"states":{"current":1,"update":2},"writes":{"success":2,"total":2}},"system":{"cpu":{"cores":2},"load":{"1":0,"15":0.05,"5":0,"norm":{"1":0,"15":0.025,"5":0}}}}}} 2019-10-01T11:04:56.846+0200 ERROR logstash/async.go:256 Failed to publish events caused by: write tcp 10.117.56.140:26856->10.118.120.65:1762: i/o timeout 2019-10-01T11:04:58.341+0200 ERROR pipeline/output.go:121 Failed to publish events: write tcp 10.117.56.140:26856->10.118.120.65:1762: i/o timeout 2019-10-01T11:04:58.341+0200 INFO pipeline/output.go:95 Connecting to backoff(async(tcp://10.118.120.65:1762)) 2019-10-01T11:04:58.345+0200 INFO pipeline/output.go:105 Connection to backoff(async(tcp://10.118.120.65:1762)) established ``` A telnet from the server B to Logstash is OK and if I try to push some random data to logstash in the telnet session, I obtain some logs to the logstash side : ``` [2019-10-01T11:19:56,575][INFO][org.logstash.beats.BeatsHandler] [local: 10.118.120.65:1762, remote: 10.117.56.140:27276] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 117 [2019-10-01T11:19:56,576][WARN][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception. ``` Logstash and Filebeat are the same version (7.2.0) The filebeat configuration between Server A and Server B is exactly the same (copy paste from A to B) Here is the filebeat conf : ``` # =========================== Filebeat inputs ============================= filebeat.inputs: - type: log enabled: true paths: - /var/opt/data/flat/squid/log/daily-access.log fields: logtype: squid_access_log_dev fields_under_root: true #============================= Filebeat modules =============================== filebeat.config.modules: # Glob pattern for configuration loading path: ${path.config}/modules.d/*.yml # Set to true to enable config reloading reload.enabled: false # Period on which files under path should be checked for changes #reload.period: 10s #==================== Elasticsearch template setting ========================== setup.template.settings: index.number_of_shards: 1 #index.codec: best_compression #_source.enabled: false #================================ General ===================================== # ... #================================ Outputs ===================================== # Configure what output to use when sending the data collected by the beat. #-------------------------- Elasticsearch output ------------------------------ #output.elasticsearch: # Array of hosts to connect to. # hosts: ["localhost:9200"] # Optional protocol and basic auth credentials. #protocol: "https" #username: "elastic" #password: "changeme" #----------------------------- Logstash output -------------------------------- output.logstash: # The Logstash hosts hosts: ["10.118.120.65:1762"] # Optional SSL. By default is off. # List of root certificates for HTTPS server verifications #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] # Certificate for SSL client authentication #ssl.certificate: "/etc/pki/client/cert.pem" # Client Certificate Key #ssl.key: "/etc/pki/client/cert.key" #================================ Processors ===================================== # Configure processors to enhance or manipulate events generated by the beat. processors: - add_host_metadata: ~ - add_cloud_metadata: ~ #================================ Logging ===================================== #... #================================= Migration ================================== #... ``` Logstash conf is pretty simple too : ``` input { beats { port => "1762" } } filter { dissect { mapping => { "message" => "%{month->} %{} %{} %{host} %{} %{timestamp->} %{} %{user_id} %{req_status} %{status_code} %{user_ip} %{user_req_size} %{method} %{url} %{mime_type} %{reply_size} %{hierarchy} %{fwd_ip} %{srv_ip} %{srv_port} %{}" } convert_datatype => { "status_code" => "int" "user_req_size" => "int" "reply_size" => "int" "srv_port" => "int" } } grok { match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp}.*%{NUMBER:duration:int}" } #remove_field => ["message"] } date { match => ["syslog_timestamp", "MMM d HH:mm:ss"] timezone => "Europe/Paris" } } output { elasticsearch { hosts => ["10.118.123.226:1761", "10.118.123.227:1761", "10.118.123.229:1761"] index => "squid-%{+YYYY.MM.dd}" manage_template => true template => "/etc/logstash/conf.d/squid_access_log_mapping.json" template_name => "squid_template" } } ``` Any idea ? Thank you ---
**Author:** ![zebu14](https://avatars.discourse-cdn.com/v4/letter/z/aca169/32.png) [@zebu14](https://discuss.elastic.co/u/zebu14)\ **Post date:** [October 3, 2019, 9:32am UTC](https://discuss.elastic.co/t/failed-to-publish-events-i-o-timeout-but-telnet-is-ok/201802/2 "2019-10-03T09:32:29Z")
Tried changing bulk\_max\_size to 1024 on filebeat side ==\> not better Tried adding more pipeline workers on logstash side ==\> not better Tried to add client\_inactivity\_timeout =\> "842" in beats input paragraph (on logstash side) ==\> not better.... I am quite stuck actually... I any of you have an idea, don't be scared to propose 🙂 Thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 31, 2019, 9:32am UTC](https://discuss.elastic.co/t/failed-to-publish-events-i-o-timeout-but-telnet-is-ok/201802/3 "2019-10-31T09:32:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.