# Logstash doesn't open files

**URL:** <https://discuss.elastic.co/t/logstash-doesnt-open-files/287100>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [October 19, 2021, 2:00pm UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100 "2021-10-19T14:00:42Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![mrdiogon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mrdiogon/32/78248_2.png) [@mrdiogon](https://discuss.elastic.co/u/mrdiogon)\
**Post date:** [October 19, 2021, 2:00pm UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/1 "2021-10-19T14:00:42Z")

</div>

Hi everyone,  
I tried to install filebeat on a windows server which send logs to a logstash server on docker. But no indexes are created and it seems logstash doesn't get any input from logstash. Here's my configuration file:

```auto
filebeat.inputs:

- type: log
  enabled: false
  paths:
    - C:\Server\Log\batch\*.json
    
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

output.logstash:
  enabled: true
  hosts: ["logstash.tools.lan:80"]

```

Does anyone have an idea what's happening here? For what i see, I think filebeat is ignoring the log files.

Thanks

---

<div class="post-metadata">

**Author:** ![mrdiogon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mrdiogon/32/78248_2.png) [@mrdiogon](https://discuss.elastic.co/u/mrdiogon)\
**Post date:** [October 19, 2021, 2:02pm UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/2 "2021-10-19T14:02:21Z")

</div>

Here's my log:

```auto
2021-10-19T15:32:10.536+0200	INFO	[monitoring]	log/log.go:142	Starting metrics logging every 30s
2021-10-19T15:32:10.538+0200	INFO	memlog/store.go:119	Loading data file of 'C:\ProgramData\filebeat\registry\filebeat' succeeded. Active transaction id=0
2021-10-19T15:32:10.538+0200	INFO	memlog/store.go:124	Finished loading transaction log file for 'C:\ProgramData\filebeat\registry\filebeat'. Active transaction id=0
2021-10-19T15:32:10.538+0200	WARN	beater/filebeat.go:381	Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2021-10-19T15:32:10.538+0200	INFO	[registrar]	registrar/registrar.go:109	States Loaded from registrar: 0
2021-10-19T15:32:10.538+0200	INFO	[crawler]	beater/crawler.go:71	Loading Inputs: 1
2021-10-19T15:32:10.539+0200	INFO	[crawler]	beater/crawler.go:108	Loading and starting Inputs completed. Enabled inputs: 0
2021-10-19T15:32:10.539+0200	INFO	cfgfile/reload.go:164	Config reloader started
2021-10-19T15:32:10.540+0200	INFO	cfgfile/reload.go:224	Loading of config files completed.
2021-10-19T15:32:40.563+0200	INFO	[monitoring]	log/log.go:184	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":156,"time":{"ms":156}},"total":{"ticks":312,"time":{"ms":312},"value":312},"user":{"ticks":156,"time":{"ms":156}}},"handles":{"open":217},"info":{"ephemeral_id":"e6df9357-db57-4be9-bd18-7f11aa20cbf1","uptime":{"ms":30135},"version":"7.15.1"},"memstats":{"gc_next":18571648,"memory_alloc":16198416,"memory_sys":32021720,"memory_total":55095984,"rss":55422976},"runtime":{"goroutines":24}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"events":{"active":0},"type":"logstash"},"pipeline":{"clients":0,"events":{"active":0},"queue":{"max_events":4096}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":4}}}}}
2021-10-19T15:33:10.573+0200	INFO	[monitoring]	log/log.go:184	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":156},"total":{"ticks":327,"time":{"ms":15},"value":327},"user":{"ticks":171,"time":{"ms":15}}},"handles":{"open":221},"info":{"ephemeral_id":"e6df9357-db57-4be9-bd18-7f11aa20cbf1","uptime":{"ms":60147},"version":"7.15.1"},"memstats":{"gc_next":18571648,"memory_alloc":16397880,"memory_total":55295448,"rss":55451648},"runtime":{"goroutines":24}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}}}}}
2021-10-19T15:33:40.578+0200	INFO	[monitoring]	log/log.go:184	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":156},"total":{"ticks":327,"value":327},"user":{"ticks":171}},"handles":{"open":223},"info":{"ephemeral_id":"e6df9357-db57-4be9-bd18-7f11aa20cbf1","uptime":{"ms":90142},"version":"7.15.1"},"memstats":{"gc_next":18571648,"memory_alloc":16592064,"memory_total":55489632,"rss":55484416},"runtime":{"goroutines":24}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}}}}}
2021-10-19T15:34:10.579+0200	INFO	[monitoring]	log/log.go:184	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":156},"total":{"ticks":327,"value":327},"user":{"ticks":171}},"handles":{"open":228},"info":{"ephemeral_id":"e6df9357-db57-4be9-bd18-7f11aa20cbf1","uptime":{"ms":120165},"version":"7.15.1"},"memstats":{"gc_next":19420848,"memory_alloc":9768376,"memory_total":55759960,"rss":55717888},"runtime":{"goroutines":24}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}}}}}
2021-10-19T15:34:40.556+0200	INFO	[monitoring]	log/log.go:184	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":171,"time":{"ms":15}},"total":{"ticks":358,"time":{"ms":31},"value":358},"user":{"ticks":187,"time":{"ms":16}}},"handles":{"open":230},"info":{"ephemeral_id":"e6df9357-db57-4be9-bd18-7f11aa20cbf1","uptime":{"ms":150141},"version":"7.15.1"},"memstats":{"gc_next":19420848,"memory_alloc":9881000,"memory_total":55872584,"rss":55173120},"runtime":{"goroutines":24}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}}}}}
2021-10-19T15:35:10.568+0200	INFO	[monitoring]	log/log.go:184	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":171},"total":{"ticks":358,"value":358},"user":{"ticks":187}},"handles":{"open":232},"info":{"ephemeral_id":"e6df9357-db57-4be9-bd18-7f11aa20cbf1","uptime":{"ms":180145},"version":"7.15.1"},"memstats":{"gc_next":19420848,"memory_alloc":10077832,"memory_total":56069416,"rss":55193600},"runtime":{"goroutines":24}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}}}}}
2021-10-19T15:41:11.192+0200	INFO	beater/filebeat.go:515	Stopping filebeat
2021-10-19T15:41:11.192+0200	INFO	beater/crawler.go:148	Stopping Crawler
2021-10-19T15:41:11.192+0200	INFO	beater/crawler.go:158	Stopping 0 inputs
2021-10-19T15:41:11.192+0200	INFO	cfgfile/reload.go:227	Dynamic config reloader stopped
2021-10-19T15:41:11.192+0200	INFO	beater/crawler.go:178	Crawler stopped

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 19, 2021, 2:12pm UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/3 "2021-10-19T14:12:07Z")

</div>

> [@mrdiogon](#):
>
> ```auto
> - type: log
> enabled: false
> 
> ```

You didn't enabled the input, try to use `enabled: true`.

---

<div class="post-metadata">

**Author:** ![mrdiogon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mrdiogon/32/78248_2.png) [@mrdiogon](https://discuss.elastic.co/u/mrdiogon)\
**Post date:** [October 21, 2021, 6:09am UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/4 "2021-10-21T06:09:29Z")

</div>

Thanks, i forgot. But it still does not work...

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [October 21, 2021, 9:30am UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/5 "2021-10-21T09:30:59Z")

</div>

Hi,

Can you show logs ?

---

<div class="post-metadata">

**Author:** ![mrdiogon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mrdiogon/32/78248_2.png) [@mrdiogon](https://discuss.elastic.co/u/mrdiogon)\
**Post date:** [October 21, 2021, 9:41am UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/6 "2021-10-21T09:41:02Z")

</div>

At firts it didn't change but now ther's an explicit error. I'm not sure what's causing it...

```auto
2021-10-21T11:33:33.469+0200	INFO	[publisher_pipeline_output]	pipeline/output.go:151	Connection to backoff(async(tcp://logstash.tools.lan:80)) established
2021-10-21T11:33:58.271+0200	INFO	[monitoring]	log/log.go:184	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":8953,"time":{"ms":16}},"total":{"ticks":59843,"time":{"ms":188},"value":59843},"user":{"ticks":50890,"time":{"ms":172}}},"handles":{"open":300},"info":{"ephemeral_id":"b80b724c-d4c8-4611-bfd6-8c1658105122","uptime":{"ms":13260237},"version":"7.15.1"},"memstats":{"gc_next":40082192,"memory_alloc":22078496,"memory_total":2234106352,"rss":84209664},"runtime":{"goroutines":236}},"filebeat":{"harvester":{"open_files":33,"running":78}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":4096,"batches":3,"failed":6144,"total":6144},"read":{"errors":1},"write":{"bytes":263843}},"pipeline":{"clients":1,"events":{"active":4117,"retry":8192}}},"registrar":{"states":{"current":3}}}}}
2021-10-21T11:34:03.515+0200	ERROR	[logstash]	logstash/async.go:280	Failed to publish events caused by: read tcp 10.90.101.37:62964->10.100.153.4:80: i/o timeout
2021-10-21T11:34:03.515+0200	ERROR	[logstash]	logstash/async.go:280	Failed to publish events caused by: read tcp 10.90.101.37:62964->10.100.153.4:80: i/o timeout
2021-10-21T11:34:03.515+0200	INFO	[publisher]	pipeline/retry.go:219	retryer: send unwait signal to consumer
2021-10-21T11:34:03.515+0200	INFO	[publisher]	pipeline/retry.go:223 done
2021-10-21T11:34:03.515+0200	INFO	[publisher]	pipeline/retry.go:219	retryer: send unwait signal to consumer
2021-10-21T11:34:03.515+0200	INFO	[publisher]	pipeline/retry.go:223 done
2021-10-21T11:34:03.556+0200	ERROR	[logstash]	logstash/async.go:280	Failed to publish events caused by: client is not connected
2021-10-21T11:34:03.556+0200	INFO	[publisher]	pipeline/retry.go:219	retryer: send unwait signal to consumer
2021-10-21T11:34:03.556+0200	INFO	[publisher]	pipeline/retry.go:223 done
2021-10-21T11:34:05.442+0200	ERROR	[publisher_pipeline_output]	pipeline/output.go:180	failed to publish events: client is not connected

```

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [October 21, 2021, 9:45am UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/7 "2021-10-21T09:45:05Z")

</div>

Can you please double check network configuration ? i believe your running it on docker are you able to telnet or curl to the host ?

You can easily check on linux port connectivity with a simple trick 😉

```auto
echo > /dev/tcp/[host]/[port] && echo "Port is open"
```

---

<div class="post-metadata">

**Author:** ![mrdiogon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mrdiogon/32/78248_2.png) [@mrdiogon](https://discuss.elastic.co/u/mrdiogon)\
**Post date:** [October 21, 2021, 10:03am UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/8 "2021-10-21T10:03:08Z")

</div>

I tried to test de connection from my windows computer with the Powershell command:

```auto
 Test-NetConnection logstash.tools.lan -Port 80

```

and it succeed.

However, i do use swarm AND traefik for my elk stack. So, with swarm, i redirect the port 80 to the port 5044 of my logstash service...

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [October 21, 2021, 11:32am UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/9 "2021-10-21T11:32:04Z")

</div>

Hum,

I'm sorry i'm not familiar with the stack your using, maybe traefik logs could help on this ?

I'm really not sure what could be your problem here 🤔

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 18, 2021, 11:32am UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100/10 "2021-11-18T11:32:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.