Windows Eventlogs are not recieved at Elasticsearch via Filebeat


(Hasan Jamal) #1

I have setup filebeat at my client machine Windows 8.1 to send Windows Event logs to Elasticsearch for analysis in Kibana

The configuration of filebeat.yml is seemed to be OK but I an not receiving any logs at elasticsearch index. I have setup tcpdump as well on the machine where elasticsearch is installed but not receiving eventlogs packets.

I have viewed the debug logs of filebeat but not able to find the reason.

Here is the configuration of filebeat.yml file.

###################################################################

filebeat:
prospectors:

-
  paths:
    - C:\\Windows\\System32\\winevt\\Logs\\*

  encoding: utf-8

  input_type: log

registry_file: "C:/ProgramData/filebeat/registry"

output:

elasticsearch:
hosts: ["http://15.0.0.18:9200"]

index: "filebeat"

template:
  name: "filebeat"
  path: "filebeat.template.json"

shipper:

#geoip:
#paths:
# - "/usr/share/GeoIP/GeoLiteCity.dat"
# - "/usr/local/var/GeoIP/GeoLiteCity.dat"

############################# Logging #########################################

logging:

To enable logging to files, to_files option has to be set to true

files:
path: C:\ProgramData\filebeat\Logs
name: mybeat

rotateeverybytes: 10485760 # = 10MB

level: debug

Here is the debug logs

################################

2016-06-02T15:48:22+05:00 DBG Disable stderr logging
2016-06-02T15:48:22+05:00 DBG Initializing output plugins
2016-06-02T15:48:22+05:00 INFO GeoIP disabled: No paths were set under output.geoip.paths
2016-06-02T15:48:22+05:00 DBG ES Ping(url=http://15.0.0.18:9200, timeout=1m30s)
2016-06-02T15:48:22+05:00 DBG Ping status code: 200
2016-06-02T15:48:22+05:00 INFO Loading template enabled. Trying to load template: filebeat.template.json
2016-06-02T15:48:22+05:00 DBG HEAD http://15.0.0.18:9200/_template/filebeat
2016-06-02T15:48:22+05:00 INFO Template already exists and will not be overwritten.
2016-06-02T15:48:22+05:00 INFO Activated elasticsearch as output plugin.
2016-06-02T15:48:22+05:00 DBG Create output worker
2016-06-02T15:48:22+05:00 DBG No output is defined to store the topology. The server fields might not be filled.
2016-06-02T15:48:22+05:00 INFO Publisher name: pl-int-hasan
2016-06-02T15:48:22+05:00 INFO Flush Interval set to: 1s
2016-06-02T15:48:22+05:00 INFO Max Bulk Size set to: 50
2016-06-02T15:48:22+05:00 DBG create bulk processing worker (interval=1s, bulk size=50)
2016-06-02T15:48:22+05:00 INFO Init Beat: filebeat; Version: 1.2.3
2016-06-02T15:48:22+05:00 INFO filebeat sucessfully setup. Start running.
2016-06-02T15:48:22+05:00 INFO Registry file set to: C:\ProgramData\filebeat\registry
2016-06-02T15:48:22+05:00 DBG Windows is interactive: false
2016-06-02T15:48:22+05:00 INFO Loading registrar data from C:\ProgramData\filebeat\registry
2016-06-02T15:48:22+05:00 DBG Set idleTimeoutDuration to 5s
2016-06-02T15:48:22+05:00 DBG File Configs: [C:\Windows\System32\winevt\Logs\]
2016-06-02T15:48:22+05:00 INFO Set ignore_older duration to 0
2016-06-02T15:48:22+05:00 INFO Set close_older duration to 1h0m0s
2016-06-02T15:48:22+05:00 INFO Set scan_frequency duration to 10s
2016-06-02T15:48:22+05:00 INFO Input type set to: log
2016-06-02T15:48:22+05:00 INFO Set backoff duration to 1s
2016-06-02T15:48:22+05:00 INFO Set max_backoff duration to 10s
2016-06-02T15:48:22+05:00 INFO force_close_file is disabled
2016-06-02T15:48:22+05:00 DBG Waiting for 1 prospectors to initialise
2016-06-02T15:48:22+05:00 INFO Starting prospector of type: log
2016-06-02T15:48:22+05:00 DBG exclude_files: []
2016-06-02T15:48:22+05:00 DBG scan path C:\Windows\System32\winevt\Logs\

2016-06-02T15:48:22+05:00 DBG scan path C:\Windows\System32\winevt\Logs\*
2016-06-02T15:48:22+05:00 DBG No pending prospectors. Finishing setup
2016-06-02T15:48:22+05:00 INFO All prospectors initialised with 0 states to persist
2016-06-02T15:48:22+05:00 INFO Starting Registrar
2016-06-02T15:48:22+05:00 INFO Start sending events to output
2016-06-02T15:48:22+05:00 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2016-06-02T15:48:24+05:00 DBG Flushing spooler because of timeout. Events flushed: 0
2016-06-02T15:48:29+05:00 DBG Flushing spooler because of timeout. Events flushed: 0

##################################################################

I have deleted the registry file as well to make sure that is not creating this problem but no success.

The index "filebeat" that I have specified in the configuration file is available at elasticsearch as I had executed the load.sh scrtip to load dashboard in kibana.

Thanks in advance for the help.


(Christian Dahlqvist) #2

Any particular reason you are not using the Winlogbeat, which is specifically designed for Windows Event log collection?


(Hasan Jamal) #3

I tried wnilogbeat as well but after configuration of the config file, Service of winlogbeat was not starting.


(Hasan Jamal) #4

I have re-tried winlogbeat today and have seen that there was seemed to be some indentation issue which I have fixed now.
Anyways thanks for the advice.


(system) #5

This topic was automatically closed after 21 days. New replies are no longer allowed.