Filebeat output failed

Hi, I have a Win10 for client and Ubuntu20 for ELK server.
Previously, I have built Winlogbeat on my client, and it run successfully that the event can be shown on Kibana. Then, I build Filebeat on the same client, and set the output config same as Winlogbeat.

Below is my Filebeat config setting:

#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - c:\testcase\*
#============================= Filebeat modules ===============================
filebeat.config.modules:
  path: c:\Program Files\Filebeat\modules.d\*.yml
  reload.enabled: false
#==================== Elasticsearch template setting ==========================
setup.template.settings:
  index.number_of_shards: 1
#============================== Kibana =====================================
setup.kibana:
  host: "ELK_IP_host:5601"
#================================ Outputs =====================================
#------------------------------- File output ----------------------------------
# output.file:
#   enabled: true
#   path: c:\demp
#   filename: filebeat
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["ELK_ip_host:9200"]
#================================ Processors =====================================
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

Then, I put some sample logs in the Filebeat input path and run the testing command:

PS> .\filebeat.exe -c .\filebeat.yml -e -d "*"

It seems that Filebeat is run successfully on local client. Then, I start the service by command:

PS> Start-Service Filebeat

Filebeat doesn't send event to the elasticsearch. I tried to make the output setting be local files, but it still don't generate any files on local.

Below is my Filebeat service log:

2020-06-02T16:24:17.155+0800	INFO	instance/beat.go:622	Home path: [C:\Program Files\Filebeat] Config path: [C:\Program Files\Filebeat] Data path: [C:\ProgramData\filebeat] Logs path: [C:\ProgramData\filebeat\logs]
2020-06-02T16:24:17.179+0800	INFO	instance/beat.go:630	Beat ID: ae2d8b74-998f-4903-b9aa-12c3d6e32045
2020-06-02T16:24:17.191+0800	INFO	[beat]	instance/beat.go:958	Beat info	{"system_info": {"beat": {"path": {"config": "C:\\Program Files\\Filebeat", "data": "C:\\ProgramData\\filebeat", "home": "C:\\Program Files\\Filebeat", "logs": "C:\\ProgramData\\filebeat\\logs"}, "type": "filebeat", "uuid": "ae2d8b74-998f-4903-b9aa-12c3d6e32045"}}}
2020-06-02T16:24:17.191+0800	INFO	[beat]	instance/beat.go:967	Build info	{"system_info": {"build": {"commit": "d57bcf8684602e15000d65b75afcd110e2b12b59", "libbeat": "7.6.2", "time": "2020-03-26T05:23:36.000Z", "version": "7.6.2"}}}
2020-06-02T16:24:17.192+0800	INFO	[beat]	instance/beat.go:970	Go runtime info	{"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":4,"version":"go1.13.8"}}}
2020-06-02T16:24:17.196+0800	INFO	[beat]	instance/beat.go:974	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-05-28T12:40:16.81+08:00","name":"Stan-VM","ip":["fe80::d90a:ca:694d:a307/64","Client_IP_host/16","::1/128","127.0.0.1/8"],"kernel_version":"10.0.18362.836 (WinBuild.160101.0800)","mac":["00:0c:29:5b:ab:7c"],"os":{"family":"windows","platform":"windows","name":"Windows 10 Enterprise","version":"10.0","major":10,"minor":0,"patch":0,"build":"18363.836"},"timezone":"CST","timezone_offset_sec":28800,"id":"31552a7a-6e38-4d05-9b26-009a907b3320"}}}
2020-06-02T16:24:17.201+0800	INFO	[beat]	instance/beat.go:1003	Process info	{"system_info": {"process": {"cwd": "C:\\Windows\\system32", "exe": "C:\\Program Files\\Filebeat\\filebeat.exe", "name": "filebeat.exe", "pid": 4288, "ppid": 680, "start_time": "2020-06-02T16:24:17.081+0800"}}}
2020-06-02T16:24:17.202+0800	INFO	instance/beat.go:298	Setup Beat: filebeat; Version: 7.6.2
2020-06-02T16:24:17.202+0800	INFO	[index-management]	idxmgmt/std.go:182	Set output.elasticsearch.index to 'filebeat-7.6.2' as ILM is enabled.
2020-06-02T16:24:17.202+0800	INFO	elasticsearch/client.go:174	Elasticsearch url: http://ELK_IP_host:9200
2020-06-02T16:24:17.202+0800	INFO	[publisher]	pipeline/module.go:110	Beat name: Stan-VM
2020-06-02T16:24:17.204+0800	INFO	instance/beat.go:439	filebeat start running.
2020-06-02T16:24:17.204+0800	INFO	[monitoring]	log/log.go:118	Starting metrics logging every 30s
2020-06-02T16:24:17.204+0800	INFO	registrar/registrar.go:145	Loading registrar data from C:\ProgramData\filebeat\registry\filebeat\data.json
2020-06-02T16:24:17.205+0800	INFO	registrar/registrar.go:152	States Loaded from registrar: 2
2020-06-02T16:24:17.205+0800	INFO	crawler/crawler.go:72	Loading Inputs: 1
2020-06-02T16:24:17.205+0800	INFO	log/input.go:152	Configured paths: [c:\testcase\*]
2020-06-02T16:24:17.205+0800	INFO	input/input.go:114	Starting input of type: log; ID: 16912143009809420970 
2020-06-02T16:24:17.205+0800	INFO	crawler/crawler.go:106	Loading and starting Inputs completed. Enabled inputs: 1
2020-06-02T16:24:17.205+0800	INFO	cfgfile/reload.go:175	Config reloader started
2020-06-02T16:24:17.206+0800	INFO	cfgfile/reload.go:235	Loading of config files completed.
2020-06-02T16:24:20.187+0800	INFO	add_cloud_metadata/add_cloud_metadata.go:89	add_cloud_metadata: hosting provider type not detected.
2020-06-02T16:24:37.206+0800	INFO	log/harvester.go:297	Harvester started for file: c:\testcase\meow.log
2020-06-02T16:24:47.206+0800	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":78,"time":{"ms":78}},"total":{"ticks":109,"time":{"ms":109},"value":109},"user":{"ticks":31,"time":{"ms":31}}},"handles":{"open":254},"info":{"ephemeral_id":"9b9f2180-9154-4a3e-9fa9-189fc98a9fc3","uptime":{"ms":30096}},"memstats":{"gc_next":9989440,"memory_alloc":5055272,"memory_total":13701304,"rss":36425728},"runtime":{"goroutines":30}},"filebeat":{"events":{"added":4,"done":4},"harvester":{"files":{"c14a2b0e-6b5a-43f9-85e0-1a3a7fca1e21":{"last_event_published_time":"","last_event_timestamp":"","name":"c:\\testcase\\meow.log","start_time":"2020-06-02T16:24:37.206Z"}},"open_files":1,"running":1,"started":1},"input":{"log":{"files":{"renamed":1}}}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"type":"elasticsearch"},"pipeline":{"clients":1,"events":{"active":0,"filtered":4,"total":4}}},"registrar":{"states":{"cleanup":2,"current":2,"update":4},"writes":{"success":4,"total":4}},"system":{"cpu":{"cores":4}}}}}
2020-06-02T16:25:17.205+0800	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":78},"total":{"ticks":109,"value":109},"user":{"ticks":31}},"handles":{"open":254},"info":{"ephemeral_id":"9b9f2180-9154-4a3e-9fa9-189fc98a9fc3","uptime":{"ms":60096}},"memstats":{"gc_next":9989440,"memory_alloc":5138608,"memory_total":13784640,"rss":20480},"runtime":{"goroutines":30}},"filebeat":{"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":2}}}}}

I also try the same Filebeat setting on ubuntu with tiny modify path config. And it works correctly.

So, my question is, "How to let Filebeat on Win10 output events?"
How do I solve this issue?

Could you please share the debug logs of the service? Also, the example logs you are testing with?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.