# FileBeat Propector issue **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [August 22, 2018, 6:31am UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494 "2018-08-22T06:31:11Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Post date:** [August 22, 2018, 6:31am UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/1 "2018-08-22T06:31:11Z")
Hi All, i am sending events of tomee via filebeat to kafka but getting below exceptions , can someone help me understanding it , looks like some configuration needs to be updated . ticker time , crawler , prospector outlet . 2018/08/21 06:24:29.966584 crawler.go:109: INFO Stopping Crawler 2018/08/21 06:24:29.966687 crawler.go:119: INFO Stopping 20 prospectors 2018/08/21 06:24:29.966736 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.966771 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.966780 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.966834 prospector.go:138: INFO Stopping Prospector: 9481811630750621847 2018/08/21 06:24:29.966737 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.966873 prospector.go:138: INFO Stopping Prospector: 16250081816100377070 2018/08/21 06:24:29.966741 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.966899 prospector.go:138: INFO Stopping Prospector: 14995750911456661710 2018/08/21 06:24:29.966846 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.966949 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966961 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966749 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.966998 prospector.go:138: INFO Stopping Prospector: 10884091510794477696 2018/08/21 06:24:29.967022 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966964 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967064 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967111 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967009 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967138 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967155 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967213 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967258 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966748 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967297 prospector.go:138: INFO Stopping Prospector: 9524210098540359837 2018/08/21 06:24:29.966750 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967314 prospector.go:138: INFO Stopping Prospector: 2266296135282362560 2018/08/21 06:24:29.967320 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967298 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966762 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967366 prospector.go:138: INFO Stopping Prospector: 5071786216812631489 2018/08/21 06:24:29.967383 prospector.go:410: INFO Scan aborted because prospector stopped. 2018/08/21 06:24:29.967399 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967404 harvester.go:215: INFO Harvester started for file: /var/log//logs/catalina.out-20180814-20180816 2018/08/21 06:24:29.967445 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966758 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967494 prospector.go:138: INFO Stopping Prospector: 9476847815990620746 2018/08/21 06:24:29.967505 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.967513 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.967524 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.967533 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.967567 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967608 prospector.go:138: INFO Stopping Prospector: 4848512172117451415 2018/08/21 06:24:29.967614 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967625 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967638 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967515 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.967676 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.967676 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966770 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967713 prospector.go:138: INFO Stopping Prospector: 4870614798961431722 2018/08/21 06:24:29.967723 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967712 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966728 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967742 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966916 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.966763 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967788 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180814-20180816. Closing. 2018/08/21 06:24:29.967798 prospector.go:138: INFO Stopping Prospector: 9402531377110145866 2018/08/21 06:24:29.967802 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180820. Closing. 2018/08/21 06:24:29.967758 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967807 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180810-20180812-20180814. Closing. 2018/08/21 06:24:29.967846 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180810-20180812-20180814-20180816. Closing. 2018/08/21 06:24:29.967849 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180810-20180812-20180814-20180816-20180818. Closing. ---
**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Post date:** [August 22, 2018, 6:33am UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/2 "2018-08-22T06:33:24Z")
2018/08/21 06:24:29.967846 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180810-20180812-20180814-20180816. Closing. 2018/08/21 06:24:29.967849 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180810-20180812-20180814-20180816-20180818. Closing. 2018/08/21 06:24:29.966988 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967783 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180811-20180813. Closing. 2018/08/21 06:24:29.967879 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180816. Closing. 2018/08/21 06:24:29.966770 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967867 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180816-20180818. Closing. 2018/08/21 06:24:29.967905 prospector.go:138: INFO Stopping Prospector: 2644950983898890120 2018/08/21 06:24:29.967859 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180817. Closing. 2018/08/21 06:24:29.967911 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180809-20180811-20180813-20180815-20180817-20180819. Closing. 2018/08/21 06:24:29.967727 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.967919 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.967991 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.967788 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968006 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967824 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180815-20180817-20180819. Closing. 2018/08/21 06:24:29.968003 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.967988 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968072 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967830 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967835 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180809-20180811-20180813. Closing. 2018/08/21 06:24:29.967836 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180818. Closing. 2018/08/21 06:24:29.968148 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967847 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out. Closing. 2018/08/21 06:24:29.967880 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180817-20180819. Closing. 2018/08/21 06:24:29.966762 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.968206 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968214 prospector.go:138: INFO Stopping Prospector: 649985330363606594 2018/08/21 06:24:29.966771 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.967886 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967849 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967727 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968257 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968274 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968291 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968295 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.968276 harvester.go:236: INFO Reader was closed: /var/log/logs/Server\_HttpMonitoring\_2018-08-13\_05\_0.log. Closing. 2018/08/21 06:24:29.968329 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.967715 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968339 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.967919 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180815-20180817. Closing. 2018/08/21 06:24:29.967749 prospector.go:138: INFO Stopping Prospector: 16207921420499893675 2018/08/21 06:24:29.967920 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180810-20180812. Closing. 2018/08/21 06:24:29.967942 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967779 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180808-20180810-20180812. Closing. 2018/08/21 06:24:29.967971 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.967991 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.966768 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.968463 prospector.go:138: INFO Stopping Prospector: 12971859759279375250 2018/08/21 06:24:29.968495 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968507 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.968520 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.968528 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.968536 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.967825 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180808-20180810-20180812-20180814. Closing. 2018/08/21 06:24:29.968241 prospector.go:138: INFO Stopping Prospector: 17204249658926138838 2018/08/21 06:24:29.967761 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.967901 harvester.go:236: INFO Reader was closed: /var/log//logs/catalina.out-20180811-20180813-20180815-20180817-20180819. Closing. 2018/08/21 06:24:29.966749 prospector.go:138: INFO Stopping Prospector: 11641839350454898397 2018/08/21 06:24:29.966792 prospector.go:138: INFO Stopping Prospector: 7607415600068520164 ---
**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Post date:** [August 22, 2018, 6:33am UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/3 "2018-08-22T06:33:36Z")
2018/08/21 06:24:29.968644 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.968655 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.968657 harvester.go:236: INFO Reader was closed: /var/log/logs/Server\_SLF4J\_BOS\_2018-08-15\_0.log. Closing. 2018/08/21 06:24:29.968376 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.968699 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968687 harvester.go:236: INFO Reader was closed: /var/log/logs/Server\_SLF4J\_BOS\_2018-08-09\_0.log. Closing. 2018/08/21 06:24:29.968664 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.968702 harvester.go:236: INFO Reader was closed: /var/log/logs/Server\_SLF4J\_BOS\_2018-08-08\_0.log. Closing. 2018/08/21 06:24:29.968703 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.967937 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.968743 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.968548 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.968754 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968760 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.968729 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.968750 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.968795 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.968804 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.968715 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.968883 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.968964 logger.go:29: DBG [publish] client: cancelled 0 events 018/08/21 06:24:29.969396 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.969410 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.969418 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.969541 prospector.go:410: INFO Scan aborted because prospector stopped. 2018/08/21 06:24:29.969597 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.969607 prospector.go:138: INFO Stopping Prospector: 5108894301380658126 2018/08/21 06:24:29.969620 harvester.go:215: INFO Harvester started for file: /var/log//logs/Server\_KernelMonitoring\_2018-08-10\_05\_0.log 2018/08/21 06:24:29.969637 forwarder.go:35: INFO Prospector outlet closed 2018/08/21 06:24:29.969641 harvester.go:236: INFO Reader was closed: /var/log//logs/Server\_KernelMonitoring\_2018-08-07\_12\_0.log. Closing. 2018/08/21 06:24:29.972130 prospector.go:410: INFO Scan aborted because prospector stopped. 2018/08/21 06:24:29.972175 prospector.go:121: INFO Prospector ticker stopped 2018/08/21 06:24:29.972184 prospector.go:138: INFO Stopping Prospector: 11496662325398572520 2018/08/21 06:24:29.972205 harvester.go:236: INFO Reader was closed: /var/log/logs/Server\_KernelMonitoring\_2018-08-17\_19\_0.log. Closing. 2018/08/21 06:24:29.972213 harvester.go:215: INFO Harvester started for file: /var/log/logs/Server\_KernelMonitoring\_2018-08-07\_08\_0.log 2018/08/21 06:24:29.976157 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.976175 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.976188 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:29.997936 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:29.997954 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:29.997964 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:30.015878 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:30.015895 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:30.015904 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:30.017812 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:30.017829 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:30.017838 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:30.048195 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:30.048212 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:30.048221 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:24:30.054319 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:24:30.054335 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:24:30.054344 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:29:34.980459 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:29:34.980476 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:29:34.980488 logger.go:29: DBG [publish] client: cancelled 0 events 2018/08/21 06:29:34.980506 crawler.go:135: INFO Crawler stopped 2018/08/21 06:29:34.980530 registrar.go:210: INFO Stopping Registrar 2018/08/21 06:29:34.980603 logger.go:18: DBG [publish] client: closing acker 2018/08/21 06:29:34.980632 logger.go:18: DBG [publish] client: done closing acker 2018/08/21 06:29:34.980644 logger.go:29: DBG [publish] client: cancelled 0 events Thanks, Shashank ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [August 22, 2018, 1:35pm UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/4 "2018-08-22T13:35:58Z")
@SJN8 I did a quick look, the "prospector outlet closed" is triggered when we update the state of the file, to help me debug this I will need a few more information: - Filebeat version - You filebeat configuration. Thanks. ---
**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Post date:** [August 23, 2018, 6:27am UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/5 "2018-08-23T06:27:07Z")
Hi pier , It is resolved by deleting registry file as there were duplicate records for prospector . Thanks for your help also i have one more query , as it is not a good practice to delete registry file always . i change my filebeat configuration everyday to update send daily logs to new kafka topic . So do we have any option in filebeat like logstash to config reload automatic so i dont need to restart Filebeat manually if configuration changes , it should pick new configuration automatically . let me know if i need to provide more information . Thanks, Shashank ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [August 23, 2018, 3:52pm UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/6 "2018-08-23T15:52:48Z")
@SJN8 Not everything can be reloaded in the beats configuration, but the [prospectors are reloadable](https://www.elastic.co/guide/en/beats/filebeat/5.3/filebeat-configuration-reloading.html). But I believe you can use the following to dynamically create the topic name based on the current date. ```auto topic: "logs-%{+yyyy.MM.dd}" ``` ---
**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Post date:** [August 27, 2018, 10:25am UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/7 "2018-08-27T10:25:46Z")
@pierhugues :- Thanks for sharing this for filebeat configuration , it is working fine for me . Now one query on Logstash which is taking input from kafka , the same topic where filebeat would be sending events , as above i tried to make below changes in my logstash config input section , input { kafka { bootstrap\_servers =\> "serverA , serverB , serverC" security\_protocol =\> "SASL\_PLAINTEXT" sasl\_mechanism =\> "PLAIN" jaas\_path =\> "/etc/logstash/jaas.conf" topics =\> ["logs-%{+YYYY.MM.dd}"] } } but above changes are not working , below are exception , /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/file.conf --config.reload.automatic WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console [ERROR] 2018-08-27 12:02:31.512 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, { at line 10, column 22 (byte 379) after # Kafka input doc\n# [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html\n\ninput](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html%5Cn%5Cninput) {\n\tkafka {\n\t\tbootstrap\_servers =\> "serA , serB , serC"\n\t\tsecurity\_protocol =\> "SASL\_PLAINTEXT"\n\t\tsasl\_mechanism =\> "PLAIN"\n\t\tjaas\_path =\> "/etc/logstash/jaas.conf"\n\t\ttopics =\> [PEOINT7-", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in`compile\_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in`map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in`initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:171:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in`execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:335:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in`with\_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:332:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:319:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in`block in converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in`converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in`block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]} can you help what i am missing here ? Thanks, Shashank ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [August 27, 2018, 3:22pm UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/8 "2018-08-27T15:22:42Z")
> [@SJN8](#): > > "Expected one of #, { at line 10, column 22 (byte 379) This look like a malformed logstash config around line 10? ---
**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Post date:** [August 28, 2018, 7:18am UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/9 "2018-08-28T07:18:55Z")
@pierhugues , configuration are good , it is not resolving metadata , like it did in Filebeat , i am getting below error , [WARN] 2018-08-28 09:12:47.568 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 4 : {logs-%{+YYYY-MM-dd}=UNKNOWN\_TOPIC\_OR\_PARTITION} [INFO] 2018-08-28 09:12:47.570 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] AbstractCoordinator - Discovered coordinator vkafka01dsy:9092 (id: 2147483647 rack: null) for group logstash. [INFO] 2018-08-28 09:12:47.572 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] ConsumerCoordinator - Revoking previously assigned partitions [] for group logstash [INFO] 2018-08-28 09:12:47.573 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] AbstractCoordinator - (Re-)joining group logstash [WARN] 2018-08-28 09:12:47.673 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 9 : {logs-%{+YYYY-MM-dd}=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:12:47.774 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 11 : {logs-%{+YYYY-MM-dd}=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:12:47.876 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 12 : {logs-%{+YYYY-MM-dd}=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:12:47.976 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 13 : {logs-%{+YYYY-MM-dd}=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:12:48.076 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 14 : {logs-%{+YYYY-MM-dd}=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:12:48.177 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 15 : {logs-%{+YYYY-MM-dd}=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:12:48.278 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 16 : {logs-%{+YYYY-MM-dd}=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:12:48.379 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 17 : {logs-%{+YYYY-MM-dd}=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:12:48.480 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/ [WARN] 2018-08-28 09:14:00.234 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 4 : {[logs-%{+YYYY-MM-dd}]=UNKNOWN\_TOPIC\_OR\_PARTITION} [INFO] 2018-08-28 09:14:00.236 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] AbstractCoordinator - Discovered coordinator vkafka01dsy:9092 (id: 2147483647 rack: null) for group logstash. [INFO] 2018-08-28 09:14:00.239 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] ConsumerCoordinator - Revoking previously assigned partitions [] for group logstash [INFO] 2018-08-28 09:14:00.239 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] AbstractCoordinator - (Re-)joining group logstash [WARN] 2018-08-28 09:14:00.340 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 10 : {[logs-%{+YYYY-MM-dd}]=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:14:00.441 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 11 : {[logs-%{+YYYY-MM-dd}]=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:14:00.542 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 12 : {[logs-%{+YYYY-MM-dd}]=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:14:00.643 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 13 : {[logs-%{+YYYY-MM-dd}]=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:14:00.744 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 14 : {[logs-%{+YYYY-MM-dd}]=UNKNOWN\_TOPIC\_OR\_PARTITION} [WARN] 2018-08-28 09:14:00.845 [Ruby-0-Thread-15: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.2/lib/logstash/inputs/kafka.rb:241] NetworkClient - Error while fetching metadata with correlation id 15 : {[logs-%{+YYYY-MM-dd}]=UNKNOWN\_TOPIC\_OR\_PARTITION} it is not resolving %{+YYYY.MM.dd} in current date . ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [August 28, 2018, 12:26pm UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/10 "2018-08-28T12:26:21Z")
@SJN8 I am afraid that logstash doesn't support dynamic topic name, I didn't know that ☹ Can I ask why do you want to have multiple topics for kafka? ---
**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Post date:** [August 28, 2018, 1:02pm UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/11 "2018-08-28T13:02:31Z")
@pierhugues :- ohh okay , multiple topic is for managing the space like i can delete old data so i am implementing day wise topics in kafka and day wise index in elastic search . i guess if metadata is supported in Filebeat and elastic search then should be in logstash , looks like i am missing some configuration . please help in this or let know who might be knowing in community will add that person in this thread 🙂 Thanks, Shashank ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 25, 2018, 1:02pm UTC](https://discuss.elastic.co/t/filebeat-propector-issue/145494/12 "2018-09-25T13:02:32Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.