Filebeat not harvesting logs and output not sent to ElasticSearch


(Rajesh) #1

Hi There,

I am new to ELK. I have configure Elastic Search, Kibana on Linux box and Configure Filebeat on Windows. I am able to view Windows Filebeat details from Kibana however, FileBeat is not sending output to Elastic Search and I could not search details from Discover even after filtering the FileBeat filter from Windows.

Please help.

FileBeat Log:

2018-12-03T04:21:31.907-0500 INFO instance/beat.go:273 Setup Beat: filebeat; Version: 6.4.2
2018-12-03T04:21:31.907-0500 INFO elasticsearch/client.go:163 Elasticsearch url: http://cdczlv200.na.convergys.com:9200
2018-12-03T04:21:31.907-0500 INFO pipeline/module.go:98 Beat name: ORLAWV004
2018-12-03T04:21:31.909-0500 INFO elasticsearch/client.go:163 Elasticsearch url: http://cdczlv200.na.convergys.com:9200
2018-12-03T04:21:31.909-0500 INFO elasticsearch/client.go:163 Elasticsearch url: http://cdczlv200.na.convergys.com:9200
2018-12-03T04:21:31.911-0500 INFO [monitoring] log/log.go:114 Starting metrics logging every 30s
2018-12-03T04:21:31.988-0500 INFO elasticsearch/elasticsearch.go:207 Successfully connected to X-Pack Monitoring endpoint.
2018-12-03T04:21:31.988-0500 INFO elasticsearch/elasticsearch.go:219 Start monitoring stats metrics snapshot loop with period 10s.
2018-12-03T04:21:31.988-0500 INFO elasticsearch/elasticsearch.go:219 Start monitoring state metrics snapshot loop with period 1m0s.
2018-12-03T04:21:31.989-0500 INFO elasticsearch/client.go:712 Connected to Elasticsearch version 6.4.2
2018-12-03T04:21:31.989-0500 INFO kibana/client.go:113 Kibana url: http://cdczlv200.na.convergys.com:5601
2018-12-03T04:21:42.063-0500 INFO pipeline/output.go:95 Connecting to backoff(publish(elasticsearch(http://cdczlv200.na.convergys.com:9200)))
2018-12-03T04:21:42.114-0500 INFO pipeline/output.go:105 Connection to backoff(publish(elasticsearch(http://cdczlv200.na.convergys.com:9200))) established
2018-12-03T04:22:00.437-0500 INFO instance/beat.go:659 Kibana dashboards successfully loaded.
2018-12-03T04:22:00.437-0500 INFO instance/beat.go:367 filebeat start running.
2018-12-03T04:22:00.437-0500 INFO registrar/registrar.go:97 No registry file found under: C:\ProgramData\filebeat\registry. Creating a new registry file.
2018-12-03T04:22:00.440-0500 INFO registrar/registrar.go:134 Loading registrar data from C:\ProgramData\filebeat\registry
2018-12-03T04:22:00.440-0500 INFO registrar/registrar.go:141 States Loaded from registrar: 0
2018-12-03T04:22:00.440-0500 INFO crawler/crawler.go:72 Loading Inputs: 1
2018-12-03T04:22:00.441-0500 INFO log/input.go:138 Configured paths: [C:\Windows\system32-D:\app\psoft\FSSYT\pt854\appserv\prcs\FS92TMI\LOGS*.LOG]
2018-12-03T04:22:00.441-0500 INFO input/input.go:114 Starting input of type: log; ID: 8981569312067516219
2018-12-03T04:22:00.441-0500 INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2018-12-03T04:22:00.441-0500 INFO cfgfile/reload.go:141 Config reloader started
2018-12-03T04:22:01.912-0500 INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":46,"time":{"ms":46}},"total":{"ticks":92,"time":{"ms":92},"value":92},"user":{"ticks":46,"time":{"ms":46}}},"info":{"ephemeral_id":"dd26af19-4627-4f13-a45a-8aa1095cb97f","uptime":{"ms":30105}},"memstats":{"gc_next":4194304,"memory_alloc":2996880,"memory_total":11091624,"rss":21381120}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"elasticsearch"},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0},"writes":{"success":1,"total":1}},"system":{"cpu":{"cores":2}}}}}


(Steffen Siering) #2

Have you configured the right path? The log line says you try to collect the path C:\Windows\system32-D:\app\psoft\FSSYT\pt854\appserv\prcs\FS92TMI\LOGS*.LOG. This doesn't look correct to me.