# Filebeat shows i/o timeout when pushing file

**URL:** <https://discuss.elastic.co/t/filebeat-shows-i-o-timeout-when-pushing-file/170976>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 5, 2019, 7:10pm UTC](https://discuss.elastic.co/t/filebeat-shows-i-o-timeout-when-pushing-file/170976 "2019-03-05T19:10:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![vladiulianbogdan](https://avatars.discourse-cdn.com/v4/letter/v/a3d4f5/32.png) [@vladiulianbogdan](https://discuss.elastic.co/u/vladiulianbogdan)\
**Post date:** [March 5, 2019, 7:10pm UTC](https://discuss.elastic.co/t/filebeat-shows-i-o-timeout-when-pushing-file/170976/1 "2019-03-05T19:10:29Z")

</div>

Hi,

I have a very simple setup. I have the ELK stack in a docker container. I am using the latest image from here [https://hub.docker.com/r/sebp/elk/](https://hub.docker.com/r/sebp/elk/).

I've installed Filebeat (6.6.1) on the machine. The configuration file that I am using for Filebeat is this one:

```
output:
  logstash:
    enabled: true
    hosts:
      - localhost:5044
    ssl:
      certificate_authorities:
        - /etc/pki/tls/certs/logstash-beats.crt
    timeout: 15

filebeat.prospectors:
- type: log
  paths:
    - /tmp/*.json
  json.keys_under_root: true
  json.overwrite_keys: true

setup.template.name: "mylogs"
setup.template.pattern: "mylogs"

```

I have put the .crt file in this path "/etc/pki/tls/certs/logstash-beats.crt".

After I start the whole environment and I put a "json" file in /tmp folder everything works fine. The logs arrive in log stash and I can see them in Kibana. The problem appears if after a few minutes, I add a second file in that folder. I start seeing this.

```
2019-03-05T10:51:26.163-0800	INFO	log/harvester.go:255	Harvester started for file: /tmp/qqq1.json
2019-03-05T10:51:26.279-0800	ERROR	logstash/async.go:256	Failed to publish events caused by: EOF
2019-03-05T10:51:26.318-0800	ERROR	logstash/async.go:256	Failed to publish events caused by: client is not connected
2019-03-05T10:51:28.127-0800	ERROR	pipeline/output.go:121	Failed to publish events: client is not connected
2019-03-05T10:51:28.127-0800	INFO	pipeline/output.go:95	Connecting to backoff(async(tcp://localhost:5044))
2019-03-05T10:51:35.924-0800	INFO	[monitoring]	log/log.go:144	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":910,"time":{"ms":64}},"total":{"ticks":6556,"time":{"ms":346},"value":6556},"user":{"ticks":5646,"time":{"ms":282}}},"info":{"ephemeral_id":"cbe29447-e7c6-491a-9dca-5ec204b49de0","uptime":{"ms":270201}},"memstats":{"gc_next":23260592,"memory_alloc":13159008,"memory_total":650281088,"rss":20602880}},"filebeat":{"events":{"active":4117,"added":4118,"done":1},"harvester":{"open_files":3,"running":3,"started":1}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"batches":2,"failed":4096,"total":4096},"read":{"bytes":31},"write":{"bytes":173242}},"pipeline":{"clients":1,"events":{"active":4117,"filtered":1,"published":4116,"retry":2048,"total":4118}}},"registrar":{"states":{"current":3,"update":1},"writes":{"success":1,"total":1}},"system":{"load":{"1":1.9077,"15":1.7837,"5":2.021,"norm":{"1":0.4769,"15":0.4459,"5":0.5052}}}}}}
2019-03-05T10:51:46.554-0800	ERROR	pipeline/output.go:100	Failed to connect to backoff(async(tcp://localhost:5044)): read tcp [::1]:50768->[::1]:5044: i/o timeout
2019-03-05T10:51:46.564-0800	INFO	pipeline/output.go:93	Attempting to reconnect to backoff(async(tcp://localhost:5044)) with 1 reconnect attempt(s)
2019-03-05T10:52:05.925-0800	INFO	[monitoring]	log/log.go:144	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":914,"time":{"ms":4}},"total":{"ticks":6562,"time":{"ms":6},"value":6562},"user":{"ticks":5648,"time":{"ms":2}}},"info":{"ephemeral_id":"cbe29447-e7c6-491a-9dca-5ec204b49de0","uptime":{"ms":300201}},"memstats":{"gc_next":23260592,"memory_alloc":13779312,"memory_total":650901392,"rss":544768}},"filebeat":{"harvester":{"open_files":3,"running":3}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"errors":2},"write":{"bytes":160}},"pipeline":{"clients":1,"events":{"active":4117,"retry":2048}}},"registrar":{"states":{"current":3}},"system":{"load":{"1":1.7744,"15":1.7778,"5":1.9839,"norm":{"1":0.4436,"15":0.4445,"5":0.496}}}}}}
2019-03-05T10:52:06.989-0800	ERROR	pipeline/output.go:100	Failed to connect to backoff(async(tcp://localhost:5044)): read tcp 127.0.0.1:50771->127.0.0.1:5044: i/o timeout
2019-03-05T10:52:06.989-0800	INFO	pipeline/output.go:93	Attempting to reconnect to backoff(async(tcp://localhost:5044)) with 2 reconnect attempt(s)

```

After this happens, if I try to access Kibana from browser it won't work anymore. On the elk stack side, I don't see any relevant logs.

I've tried to telnet into localhost 5044 and I manage to connect successfully.

For debugging purposes, I've switched the input in filebeat to "stdin" and it is working well.

I am out of ideas what I could try. Any help is welcomed.

LE: I've hosted the ELK stack on a different machine without Docker and it seems like it is working fine.

Thanks,  
Bogdan

---

<div class="post-metadata">

**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\
**Post date:** [March 6, 2019, 3:22pm UTC](https://discuss.elastic.co/t/filebeat-shows-i-o-timeout-when-pushing-file/170976/2 "2019-03-06T15:22:49Z")

</div>

This is weird, I haven't used that specific docker images, but I would check Logstash's logs to see if there is any errors there.

FYI we also have our official dockers images available at [https://www.docker.elastic.co/](https://www.docker.elastic.co/)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 3, 2019, 3:22pm UTC](https://discuss.elastic.co/t/filebeat-shows-i-o-timeout-when-pushing-file/170976/3 "2019-04-03T15:22:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
