I have setup filebeat at my client machine Windows 8.1 to send Windows Event logs to Elasticsearch for analysis in Kibana
The configuration of filebeat.yml is seemed to be OK but I an not receiving any logs at elasticsearch index. I have setup tcpdump as well on the machine where elasticsearch is installed but not receiving eventlogs packets.
I have viewed the debug logs of filebeat but not able to find the reason.
Here is the configuration of filebeat.yml file.
###################################################################
filebeat:
prospectors:
-
paths:
- C:\\Windows\\System32\\winevt\\Logs\\*
encoding: utf-8
input_type: log
registry_file: "C:/ProgramData/filebeat/registry"
output:
elasticsearch:
hosts: ["http://15.0.0.18:9200"]
index: "filebeat"
template:
name: "filebeat"
path: "filebeat.template.json"
shipper:
#geoip:
#paths:
# - "/usr/share/GeoIP/GeoLiteCity.dat"
# - "/usr/local/var/GeoIP/GeoLiteCity.dat"
############################# Logging #########################################
logging:
To enable logging to files, to_files option has to be set to true
files:
path: C:\ProgramData\filebeat\Logs
name: mybeat
rotateeverybytes: 10485760 # = 10MB
level: debug
Here is the debug logs
################################
2016-06-02T15:48:22+05:00 DBG Disable stderr logging
2016-06-02T15:48:22+05:00 DBG Initializing output plugins
2016-06-02T15:48:22+05:00 INFO GeoIP disabled: No paths were set under output.geoip.paths
2016-06-02T15:48:22+05:00 DBG ES Ping(url=http://15.0.0.18:9200, timeout=1m30s)
2016-06-02T15:48:22+05:00 DBG Ping status code: 200
2016-06-02T15:48:22+05:00 INFO Loading template enabled. Trying to load template: filebeat.template.json
2016-06-02T15:48:22+05:00 DBG HEAD http://15.0.0.18:9200/_template/filebeat
2016-06-02T15:48:22+05:00 INFO Template already exists and will not be overwritten.
2016-06-02T15:48:22+05:00 INFO Activated elasticsearch as output plugin.
2016-06-02T15:48:22+05:00 DBG Create output worker
2016-06-02T15:48:22+05:00 DBG No output is defined to store the topology. The server fields might not be filled.
2016-06-02T15:48:22+05:00 INFO Publisher name: pl-int-hasan
2016-06-02T15:48:22+05:00 INFO Flush Interval set to: 1s
2016-06-02T15:48:22+05:00 INFO Max Bulk Size set to: 50
2016-06-02T15:48:22+05:00 DBG create bulk processing worker (interval=1s, bulk size=50)
2016-06-02T15:48:22+05:00 INFO Init Beat: filebeat; Version: 1.2.3
2016-06-02T15:48:22+05:00 INFO filebeat sucessfully setup. Start running.
2016-06-02T15:48:22+05:00 INFO Registry file set to: C:\ProgramData\filebeat\registry
2016-06-02T15:48:22+05:00 DBG Windows is interactive: false
2016-06-02T15:48:22+05:00 INFO Loading registrar data from C:\ProgramData\filebeat\registry
2016-06-02T15:48:22+05:00 DBG Set idleTimeoutDuration to 5s
2016-06-02T15:48:22+05:00 DBG File Configs: [C:\Windows\System32\winevt\Logs\]
2016-06-02T15:48:22+05:00 INFO Set ignore_older duration to 0
2016-06-02T15:48:22+05:00 INFO Set close_older duration to 1h0m0s
2016-06-02T15:48:22+05:00 INFO Set scan_frequency duration to 10s
2016-06-02T15:48:22+05:00 INFO Input type set to: log
2016-06-02T15:48:22+05:00 INFO Set backoff duration to 1s
2016-06-02T15:48:22+05:00 INFO Set max_backoff duration to 10s
2016-06-02T15:48:22+05:00 INFO force_close_file is disabled
2016-06-02T15:48:22+05:00 DBG Waiting for 1 prospectors to initialise
2016-06-02T15:48:22+05:00 INFO Starting prospector of type: log
2016-06-02T15:48:22+05:00 DBG exclude_files: []
2016-06-02T15:48:22+05:00 DBG scan path C:\Windows\System32\winevt\Logs\
2016-06-02T15:48:22+05:00 DBG scan path C:\Windows\System32\winevt\Logs\*
2016-06-02T15:48:22+05:00 DBG No pending prospectors. Finishing setup
2016-06-02T15:48:22+05:00 INFO All prospectors initialised with 0 states to persist
2016-06-02T15:48:22+05:00 INFO Starting Registrar
2016-06-02T15:48:22+05:00 INFO Start sending events to output
2016-06-02T15:48:22+05:00 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2016-06-02T15:48:24+05:00 DBG Flushing spooler because of timeout. Events flushed: 0
2016-06-02T15:48:29+05:00 DBG Flushing spooler because of timeout. Events flushed: 0
##################################################################
I have deleted the registry file as well to make sure that is not creating this problem but no success.
The index "filebeat" that I have specified in the configuration file is available at elasticsearch as I had executed the load.sh scrtip to load dashboard in kibana.
Thanks in advance for the help.