# Filebeat stop after sometimes while reading yarn logs **URL:** \ **Category:** Beats\ **Created:** [April 6, 2020, 12:00pm UTC](https://discuss.elastic.co/t/filebeat-stop-after-sometimes-while-reading-yarn-logs/226702 "2020-04-06T12:00:50Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![ucguy4u](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ucguy4u/32/62333_2.png) [@ucguy4u](https://discuss.elastic.co/u/ucguy4u)\ **Post date:** [April 6, 2020, 12:00pm UTC](https://discuss.elastic.co/t/filebeat-stop-after-sometimes-while-reading-yarn-logs/226702/1 "2020-04-06T12:00:51Z")
I am using filebeat to ship data of yarn logs of hadoop. I am using is filebeat-7.5.1 ` filebeat.inputs: - type: log enabled: true This is configuration mention in the filebeat.yml ` 2020-04-01T16:12:54.871+0530 INFO kafka/log.go:53 producer/broker/1 maximum request accumulated, waiting for space 2020-04-01T16:12:54.896+0530 INFO kafka/log.go:53 producer/broker/1 maximum request accumulated, waiting for space 2020-04-01T16:12:55.876+0530 INFO kafka/log.go:53 producer/broker/1 maximum request accumulated, waiting for space 2020-04-01T16:13:05.950+0530 INFO beater/filebeat.go:443 Stopping filebeat 2020-04-01T16:13:05.951+0530 INFO crawler/crawler.go:139 Stopping Crawler 2020-04-01T16:13:05.956+0530 INFO crawler/crawler.go:149 Stopping 1 inputs 2020-04-01T16:13:05.962+0530 INFO input/input.go:149 input ticker stopped 2020-04-01T16:13:05.962+0530 INFO input/input.go:167 Stopping Input: 4626669614810496825 2020-04-01T16:13:05.962+0530 INFO log/harvester.go:272 Reader was closed: /scratch/oba/logs/hadoop/userlogs/application\_1582785823666\_2626/container\_1582785823666\_2626\_01\_000002/stderr. Closing. 2020-04-01T16:13:05.962+0530 INFO log/harvester.go:272 Reader was closed: /root/ingest-logs/ingest.log. Closing. 2020-04-01T16:13:05.962+0530 INFO log/harvester.go:272 Reader was closed: /root/ingest-logs/ingest.log. Closing. 2020-04-01T16:13:05.962+0530 INFO log/harvester.go:272 Reader was closed: /root/hadoop/userlogs/application\_1582785823666\_2626/container\_1582785823666\_2626\_01\_000001/stderr. Closing. 2020-04-01T16:13:05.961+0530 INFO cfgfile/reload.go:229 Dynamic config reloader stopped 2020-04-01T16:13:05.971+0530 INFO crawler/crawler.go:165 Crawler stopped 2020-04-01T16:13:05.971+0530 INFO registrar/registrar.go:367 Stopping Registrar 2020-04-01T16:13:05.971+0530 INFO registrar/registrar.go:293 Ending Registrar ` I have a concern about the rolling logs in the yarn and inactive log file. Does it affect the workingof filbeat. Can you please suggest the changes required in filebeat to keep filebeat alive. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [April 6, 2020, 6:53pm UTC](https://discuss.elastic.co/t/filebeat-stop-after-sometimes-while-reading-yarn-logs/226702/2 "2020-04-06T18:53:52Z")
Hey @ucguy4u, welcome to discuss 🙂 > [@ucguy4u](#): > > 2020-04-01T16:12:54.871+0530 INFO kafka/log.go:53 producer/broker/1 maximum request accumulated, waiting for space These log lines seem to indicate some kind of overload in your kafka server, I guess that you are using the kafka output, is your kafka server having any kind of problem? How are you reading the messages from kafka? > [@ucguy4u](#): > > 2020-04-01T16:13:05.950+0530 INFO beater/filebeat.go:443 Stopping filebeat These log entry seems to indicate that filebeat is being explicitly stopped. How are you starting filebeat? ---
**Author:** ![ucguy4u](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ucguy4u/32/62333_2.png) [@ucguy4u](https://discuss.elastic.co/u/ucguy4u)\ **Post date:** [April 7, 2020, 7:49am UTC](https://discuss.elastic.co/t/filebeat-stop-after-sometimes-while-reading-yarn-logs/226702/3 "2020-04-07T07:49:47Z")
Yes logs are being shipping to the Kafka and Logtsash will read from Kafka. kafka output is configured as: ```auto output.kafka: # initial brokers for reading cluster metadata hosts: [":9092"] topic: "bdp" timeout: 60 codec.json: pretty: false ``` ---
**Author:** ![ucguy4u](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ucguy4u/32/62333_2.png) [@ucguy4u](https://discuss.elastic.co/u/ucguy4u)\ **Post date:** [April 7, 2020, 7:50am UTC](https://discuss.elastic.co/t/filebeat-stop-after-sometimes-while-reading-yarn-logs/226702/4 "2020-04-07T07:50:57Z")
I am starting Filebeat as : ```auto ./filebeat -e ``` ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [April 7, 2020, 8:35am UTC](https://discuss.elastic.co/t/filebeat-stop-after-sometimes-while-reading-yarn-logs/226702/5 "2020-04-07T08:35:56Z")
> [@ucguy4u](#): > > I am starting Filebeat as : > > ```auto > ./filebeat -e > > ``` And filebeat stops by itself without logging any error? ---
**Author:** ![ucguy4u](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ucguy4u/32/62333_2.png) [@ucguy4u](https://discuss.elastic.co/u/ucguy4u)\ **Post date:** [April 7, 2020, 10:47am UTC](https://discuss.elastic.co/t/filebeat-stop-after-sometimes-while-reading-yarn-logs/226702/6 "2020-04-07T10:47:31Z")
It is starting with no errors and stop after some time. I have a concern about inactive log files, our application sometimes stop appending logs to log file(inactive) does it affect Filebeat to stop? and also, yarn logs rotate with different container ids. do we need to handle it explicitly in Filebeat? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 5, 2020, 12:47pm UTC](https://discuss.elastic.co/t/filebeat-stop-after-sometimes-while-reading-yarn-logs/226702/7 "2020-05-05T12:47:35Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.