Hi, I have a Win10 for client and Ubuntu20 for ELK server.
Previously, I have built Winlogbeat on my client, and it run successfully that the event can be shown on Kibana. Then, I build Filebeat on the same client, and set the output config same as Winlogbeat.
Below is my Filebeat config setting:
#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log
enabled: true
paths:
- c:\testcase\*
#============================= Filebeat modules ===============================
filebeat.config.modules:
path: c:\Program Files\Filebeat\modules.d\*.yml
reload.enabled: false
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 1
#============================== Kibana =====================================
setup.kibana:
host: "ELK_IP_host:5601"
#================================ Outputs =====================================
#------------------------------- File output ----------------------------------
# output.file:
# enabled: true
# path: c:\demp
# filename: filebeat
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["ELK_ip_host:9200"]
#================================ Processors =====================================
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
Then, I put some sample logs in the Filebeat input path and run the testing command:
PS> .\filebeat.exe -c .\filebeat.yml -e -d "*"
It seems that Filebeat is run successfully on local client. Then, I start the service by command:
PS> Start-Service Filebeat
Filebeat doesn't send event to the elasticsearch. I tried to make the output setting be local files, but it still don't generate any files on local.
Below is my Filebeat service log:
2020-06-02T16:24:17.155+0800 INFO instance/beat.go:622 Home path: [C:\Program Files\Filebeat] Config path: [C:\Program Files\Filebeat] Data path: [C:\ProgramData\filebeat] Logs path: [C:\ProgramData\filebeat\logs]
2020-06-02T16:24:17.179+0800 INFO instance/beat.go:630 Beat ID: ae2d8b74-998f-4903-b9aa-12c3d6e32045
2020-06-02T16:24:17.191+0800 INFO [beat] instance/beat.go:958 Beat info {"system_info": {"beat": {"path": {"config": "C:\\Program Files\\Filebeat", "data": "C:\\ProgramData\\filebeat", "home": "C:\\Program Files\\Filebeat", "logs": "C:\\ProgramData\\filebeat\\logs"}, "type": "filebeat", "uuid": "ae2d8b74-998f-4903-b9aa-12c3d6e32045"}}}
2020-06-02T16:24:17.191+0800 INFO [beat] instance/beat.go:967 Build info {"system_info": {"build": {"commit": "d57bcf8684602e15000d65b75afcd110e2b12b59", "libbeat": "7.6.2", "time": "2020-03-26T05:23:36.000Z", "version": "7.6.2"}}}
2020-06-02T16:24:17.192+0800 INFO [beat] instance/beat.go:970 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":4,"version":"go1.13.8"}}}
2020-06-02T16:24:17.196+0800 INFO [beat] instance/beat.go:974 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-05-28T12:40:16.81+08:00","name":"Stan-VM","ip":["fe80::d90a:ca:694d:a307/64","Client_IP_host/16","::1/128","127.0.0.1/8"],"kernel_version":"10.0.18362.836 (WinBuild.160101.0800)","mac":["00:0c:29:5b:ab:7c"],"os":{"family":"windows","platform":"windows","name":"Windows 10 Enterprise","version":"10.0","major":10,"minor":0,"patch":0,"build":"18363.836"},"timezone":"CST","timezone_offset_sec":28800,"id":"31552a7a-6e38-4d05-9b26-009a907b3320"}}}
2020-06-02T16:24:17.201+0800 INFO [beat] instance/beat.go:1003 Process info {"system_info": {"process": {"cwd": "C:\\Windows\\system32", "exe": "C:\\Program Files\\Filebeat\\filebeat.exe", "name": "filebeat.exe", "pid": 4288, "ppid": 680, "start_time": "2020-06-02T16:24:17.081+0800"}}}
2020-06-02T16:24:17.202+0800 INFO instance/beat.go:298 Setup Beat: filebeat; Version: 7.6.2
2020-06-02T16:24:17.202+0800 INFO [index-management] idxmgmt/std.go:182 Set output.elasticsearch.index to 'filebeat-7.6.2' as ILM is enabled.
2020-06-02T16:24:17.202+0800 INFO elasticsearch/client.go:174 Elasticsearch url: http://ELK_IP_host:9200
2020-06-02T16:24:17.202+0800 INFO [publisher] pipeline/module.go:110 Beat name: Stan-VM
2020-06-02T16:24:17.204+0800 INFO instance/beat.go:439 filebeat start running.
2020-06-02T16:24:17.204+0800 INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2020-06-02T16:24:17.204+0800 INFO registrar/registrar.go:145 Loading registrar data from C:\ProgramData\filebeat\registry\filebeat\data.json
2020-06-02T16:24:17.205+0800 INFO registrar/registrar.go:152 States Loaded from registrar: 2
2020-06-02T16:24:17.205+0800 INFO crawler/crawler.go:72 Loading Inputs: 1
2020-06-02T16:24:17.205+0800 INFO log/input.go:152 Configured paths: [c:\testcase\*]
2020-06-02T16:24:17.205+0800 INFO input/input.go:114 Starting input of type: log; ID: 16912143009809420970
2020-06-02T16:24:17.205+0800 INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2020-06-02T16:24:17.205+0800 INFO cfgfile/reload.go:175 Config reloader started
2020-06-02T16:24:17.206+0800 INFO cfgfile/reload.go:235 Loading of config files completed.
2020-06-02T16:24:20.187+0800 INFO add_cloud_metadata/add_cloud_metadata.go:89 add_cloud_metadata: hosting provider type not detected.
2020-06-02T16:24:37.206+0800 INFO log/harvester.go:297 Harvester started for file: c:\testcase\meow.log
2020-06-02T16:24:47.206+0800 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":78,"time":{"ms":78}},"total":{"ticks":109,"time":{"ms":109},"value":109},"user":{"ticks":31,"time":{"ms":31}}},"handles":{"open":254},"info":{"ephemeral_id":"9b9f2180-9154-4a3e-9fa9-189fc98a9fc3","uptime":{"ms":30096}},"memstats":{"gc_next":9989440,"memory_alloc":5055272,"memory_total":13701304,"rss":36425728},"runtime":{"goroutines":30}},"filebeat":{"events":{"added":4,"done":4},"harvester":{"files":{"c14a2b0e-6b5a-43f9-85e0-1a3a7fca1e21":{"last_event_published_time":"","last_event_timestamp":"","name":"c:\\testcase\\meow.log","start_time":"2020-06-02T16:24:37.206Z"}},"open_files":1,"running":1,"started":1},"input":{"log":{"files":{"renamed":1}}}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"type":"elasticsearch"},"pipeline":{"clients":1,"events":{"active":0,"filtered":4,"total":4}}},"registrar":{"states":{"cleanup":2,"current":2,"update":4},"writes":{"success":4,"total":4}},"system":{"cpu":{"cores":4}}}}}
2020-06-02T16:25:17.205+0800 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":78},"total":{"ticks":109,"value":109},"user":{"ticks":31}},"handles":{"open":254},"info":{"ephemeral_id":"9b9f2180-9154-4a3e-9fa9-189fc98a9fc3","uptime":{"ms":60096}},"memstats":{"gc_next":9989440,"memory_alloc":5138608,"memory_total":13784640,"rss":20480},"runtime":{"goroutines":30}},"filebeat":{"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":2}}}}}
I also try the same Filebeat setting on ubuntu with tiny modify path config. And it works correctly.
So, my question is, "How to let Filebeat on Win10 output events?"
How do I solve this issue?