# Filebeat and Logstash logdata not passing **URL:** https://discuss.elastic.co/t/filebeat-and-logstash-logdata-not-passing/211459 **Category:** Beats **Tags:** filebeat **Created:** [December 11, 2019, 12:02pm UTC](https://discuss.elastic.co/t/filebeat-and-logstash-logdata-not-passing/211459 "2019-12-11T12:02:39Z") **Posts on this page:** 5 **Page:** 1
### Author: ![katara](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/katara/32/60143_2.png) [@katara](https://discuss.elastic.co/u/katara) #### Post date: [December 11, 2019, 12:02pm UTC](https://discuss.elastic.co/t/filebeat-and-logstash-logdata-not-passing/211459/1 "2019-12-11T12:02:39Z")
Hi I have 2 servers in which one has Logstash 7.2.0 and another has filbeat 7.2.0. The below is my logstash conf file: \> input { > ``` > beats { > port => 5044 > ssl => false > } > } > output { > elasticsearch { > hosts => ["10.226.51.24:9200"] > index => "oasis" > } > stdout { codec => rubydebug } > } > > ``` below is my filebeat yml config: filebeat.inputs: ``` # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. - type: log # Change to true to enable this input configuration. enabled: false # Paths that should be crawled and fetched. Glob based paths. paths: - D:\Oasis\Logs\InetPub_Pearson_k12internet\Ordering_Log\* - D:\OasisServices\Log\REST\DigitalOrders\* #----------------------------- Logstash output -------------------------------- output.logstash: # The Logstash hosts hosts: ["110.226.51.22:5044"] ``` **Logstash logs: I restarted the service as well.** > [2019-12-11T06:41:53,380][WARN][logstash.runner] SIGTERM received. Shutting down. > [2019-12-11T06:41:58,598][WARN][org.logstash.execution.ShutdownWatcherExt] {"inflight\_count"=\>0, "stalling\_threads\_info"=\>{"other"=\>[{"thread\_id"=\>30, "name"=\>"[oasislogs]\"[...]/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.0-java/lib/logstash/inputs/beats.rb:204:in `run'"}, {"thread_id"=\>22, "name"=\>"[oasislogs]\>worker0", "current_call"=\>"[...]/logstash-core/lib/logstash/java_pipeline.rb:239:in `block in start\_workers'"}, {"thread\_id"=\>23, "name"=\>"[oasislogs]\>worker1", "current\_call"=\>"[...]/logstash-core/lib/logstash/java\_pipeline.rb:239:in `block in start_workers'"}, {"thread_id"=\>24, "name"=\>"[oasislogs]\>worker2", "current_call"=\>"[...]/logstash-core/lib/logstash/java_pipeline.rb:239:in `block in start\_workers'"}, {"thread\_id"=\>25, "name"=\>"[oasislogs]\>worker3", "current\_call"=\>"[...]/logstash-core/lib/logstash/java\_pipeline.rb:239:in `block in start_workers'"}, {"thread_id"=\>26, "name"=\>"[oasislogs]\>worker4", "current_call"=\>"[...]/logstash-core/lib/logstash/java_pipeline.rb:239:in `block in start\_workers'"}, {"thread\_id"=\>27, "name"=\>"[oasislogs]\>worker5", "current\_call"=\>"[...]/logstash-core/lib/logstash/java\_pipeline.rb:239:in `block in start_workers'"}, {"thread_id"=\>28, "name"=\>"[oasislogs]\>worker6", "current_call"=\>"[...]/logstash-core/lib/logstash/java_pipeline.rb:239:in `block in start\_workers'"}, {"thread\_id"=\>29, "name"=\>"[oasislogs]\>worker7", "current\_call"=\>"[...]/logstash-core/lib/logstash/java\_pipeline.rb:239:in `block in start\_workers'"}]}} > [2019-12-11T06:41:58,602][ERROR][org.logstash.execution.ShutdownWatcherExt] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information. > [2019-12-11T06:41:59,876][INFO][logstash.javapipeline] Pipeline terminated {"pipeline.id"=\>"oasislogs"} > [2019-12-11T06:42:00,572][INFO][logstash.runner] Logstash shut down. > [2019-12-11T06:42:10,757][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"7.2.0"} > [2019-12-11T06:42:14,859][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://10.16.5.24:9200/](http://10.16.5.24:9200/)]}} > [2019-12-11T06:42:14,997][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://10.16.5.24:9200/](http://10.16.5.24:9200/)"} > [2019-12-11T06:42:15,033][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>7} > [2019-12-11T06:42:15,035][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7} > [2019-12-11T06:42:15,058][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::Elasticsearch", :hosts=\>["[//10.16.5.24:9200](https://10.16.5.24:9200)"]} > [2019-12-11T06:42:15,111][INFO][logstash.outputs.elasticsearch] Using default mapping template > [2019-12-11T06:42:15,135][WARN][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge] A gauge metric of an unknown type (org.jruby.specialized.RubyArrayOneObject) has been create for key: cluster\_uuids. This may result in invalid serialization. It is recommended to log an issue to the responsible developer/development team. > [2019-12-11T06:42:15,138][INFO][logstash.javapipeline] Starting pipeline {:pipeline\_id=\>"oasislogs", "pipeline.workers"=\>8, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>1000, :thread=\>"#\"} > [2019-12-11T06:42:15,176][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"index\_patterns"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s", "number\_of\_shards"=\>1}, "mappings"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}} > [2019-12-11T06:42:15,376][INFO][logstash.inputs.beats] Beats inputs: Starting input listener {:address=\>"0.0.0.0:5044"} > [2019-12-11T06:42:15,383][INFO][logstash.javapipeline] Pipeline started {"pipeline.id"=\>"oasislogs"} > [2019-12-11T06:42:15,454][INFO][logstash.agent] Pipelines running {:count=\>1, :running\_pipelines=\>[:oasislogs], :non\_running\_pipelines=\>} > [2019-12-11T06:42:15,467][INFO][org.logstash.beats.Server] Starting server on port: 5044 > [2019-12-11T06:42:15,702][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} This is however not producing any outputs. Where am I going wrong? Please help me out. ---
### Author: ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark) #### Post date: [December 11, 2019, 12:34pm UTC](https://discuss.elastic.co/t/filebeat-and-logstash-logdata-not-passing/211459/2 "2019-12-11T12:34:53Z")
Hi! Could you share Filebeat's logs too? Thanks! ---
### Author: ![katara](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/katara/32/60143_2.png) [@katara](https://discuss.elastic.co/u/katara) #### Post date: [December 12, 2019, 4:21am UTC](https://discuss.elastic.co/t/filebeat-and-logstash-logdata-not-passing/211459/3 "2019-12-12T04:21:45Z")
@ChrsMark, Thank you for helping me out, and my Apologies for the delayed response, Here's the filebeat logs, > ``` > |2019-12-11T23:17:34.189-0500|INFO|[monitoring]|log/log.go:145|Non-zero metrics in the last 30s|{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":93},"total":{"ticks":171,"value":171},"user":{"ticks":78}},"handles":{"open":107},"info":{"ephemeral_id":"25360bca-ea57-4f32-9bb0-cacf032b86af","uptime":{"ms":65970049}},"memstats":{"gc_next":4194304,"memory_alloc":1876208,"memory_total":101684848,"rss":-8192},"runtime":{"goroutines":15}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}}}}}| > |---|---|---|---|---|---| > |2019-12-11T23:18:04.189-0500|INFO|[monitoring]|log/log.go:145|Non-zero metrics in the last 30s|{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":93},"total":{"ticks":171,"value":171},"user":{"ticks":78}},"handles":{"open":107},"info":{"ephemeral_id":"25360bca-ea57-4f32-9bb0-cacf032b86af","uptime":{"ms":66000049}},"memstats":{"gc_next":4194304,"memory_alloc":1922992,"memory_total":101731632},"runtime":{"goroutines":15}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}}}}}| > |2019-12-11T23:18:34.192-0500|INFO|[monitoring]|log/log.go:145|Non-zero metrics in the last 30s|{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":93},"total":{"ticks":171,"value":171},"user":{"ticks":78}},"handles":{"open":107},"info":{"ephemeral_id":"25360bca-ea57-4f32-9bb0-cacf032b86af","uptime":{"ms":66030049}},"memstats":{"gc_next":4194304,"memory_alloc":1964752,"memory_total":101773392,"rss":-24576},"runtime":{"goroutines":15}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}}}}}| > |2019-12-11T23:19:04.192-0500|INFO|[monitoring]|log/log.go:145|Non-zero metrics in the last 30s|{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":93},"total":{"ticks":171,"value":171},"user":{"ticks":78}},"handles":{"open":107},"info":{"ephemeral_id":"25360bca-ea57-4f32-9bb0-cacf032b86af","uptime":{"ms":66060049}},"memstats":{"gc_next":4194304,"memory_alloc":1801872,"memory_total":101819272,"rss":49152},"runtime":{"goroutines":15}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}}}}}| > > ``` Thanks! ---
### Author: ![katara](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/katara/32/60143_2.png) [@katara](https://discuss.elastic.co/u/katara) #### Post date: [December 12, 2019, 5:19am UTC](https://discuss.elastic.co/t/filebeat-and-logstash-logdata-not-passing/211459/4 "2019-12-12T05:19:03Z")
I had to enable the module `enabled: true` My bad, I totally missed it. Thank you for the time team 🙂 ---
### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system) #### Post date: [January 9, 2020, 5:19am UTC](https://discuss.elastic.co/t/filebeat-and-logstash-logdata-not-passing/211459/5 "2020-01-09T05:19:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.