Filebeat Windows Service won't start

Elastic Stack Beats
1

Hi there,

I've recently installed Filebeat 6.7.2 on a new windows server. I'm able to install the service using the included PowerShell scripts but I can't start it. I've tried running it under Local System USer and my user to no avail. If I run from the command line .\filebeat.exe -c C:\elk\filebeat\filebeat.yml -e -d "*" it runs fine. The log below is from a failed service start:

2019-05-07T21:51:01.611-0400 INFO instance/beat.go:611 Home path: [C:\elk\filebeat] Config path: [C:\elk\filebeat] Data path: [C:\ProgramData\filebeat] Logs path: [C:\ProgramData\filebeat\logs]
2019-05-07T21:51:01.623-0400 DEBUG [beat] instance/beat.go:648 Beat metadata path: C:\ProgramData\filebeat\meta.json
2019-05-07T21:51:01.624-0400 INFO instance/beat.go:618 Beat UUID: ef88501c-db97-49c3-bac8-ba2507324b82
2019-05-07T21:51:01.624-0400 DEBUG [seccomp] seccomp/seccomp.go:88 Syscall filtering is only supported on Linux
2019-05-07T21:51:01.624-0400 INFO [beat] instance/beat.go:931 Beat info {"system_info": {"beat": {"path": {"config": "C:\elk\filebeat", "data": "C:\ProgramData\filebeat", "home": "C:\elk\filebeat", "logs": "C:\ProgramData\filebeat\logs"}, "type": "filebeat", "uuid": "ef88501c-db97-49c3-bac8-ba2507324b82"}}}
2019-05-07T21:51:01.624-0400 INFO [beat] instance/beat.go:940 Build info {"system_info": {"build": {"commit": "a8ab26dd1f818d27c17c3049f643652c6a789d88", "libbeat": "6.7.2", "time": "2019-04-29T08:06:34.000Z", "version": "6.7.2"}}}
2019-05-07T21:51:01.624-0400 INFO [beat] instance/beat.go:943 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":4,"version":"go1.10.8"}}}
2019-05-07T21:51:01.634-0400 INFO [beat] instance/beat.go:947 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-04-30T14:39:43.71-04:00","name":"TASKTEST","ip":["fe80::1532:3fa4:13:8879/64","172.30.1.191/22","::1/128","127.0.0.1/8","fe80::5efe:ac1e:1bf/128"],"kernel_version":"6.3.9600.19304 (winblue_ltsb_escrow.190305-1818)","mac":["00:0c:29:34:3c:cb","00:00:00:00:00:00:00:e0"],"os":{"family":"windows","platform":"windows","name":"Windows Server 2012 R2 Standard","version":"6.3","major":3,"minor":0,"patch":0,"build":"9600.19323"},"timezone":"EDT","timezone_offset_sec":-14400,"id":"b25cc28c-1f1d-484c-aa1c-386e7b25bf59"}}}
2019-05-07T21:51:01.637-0400 INFO [beat] instance/beat.go:976 Process info {"system_info": {"process": {"cwd": "C:\Windows\system32", "exe": "C:\elk\filebeat\filebeat.exe", "name": "filebeat.exe", "pid": 7764, "ppid": 532, "start_time": "2019-05-07T21:51:01.500-0400"}}}
2019-05-07T21:51:01.637-0400 INFO instance/beat.go:280 Setup Beat: filebeat; Version: 6.7.2
2019-05-07T21:51:01.637-0400 DEBUG [beat] instance/beat.go:301 Initializing output plugins
2019-05-07T21:51:01.643-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:161 add_cloud_metadata: starting to fetch metadata, timeout=3s
2019-05-07T21:51:01.651-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:193 add_cloud_metadata: received disposition for qcloud after 8.0099ms. result=[provider:qcloud, error=failed requesting qcloud metadata: Get http://metadata.tencentyun.com/meta-data/instance-id: dial tcp: lookup metadata.tencentyun.com: no such host, metadata={}]
2019-05-07T21:51:04.651-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:193 add_cloud_metadata: received disposition for digitalocean after 3.007749s. result=[provider:digitalocean, error=failed requesting digitalocean metadata: Get http://169.254.169.254/metadata/v1.json: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers), metadata={}]
2019-05-07T21:51:04.651-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:193 add_cloud_metadata: received disposition for az after 3.007749s. result=[provider:az, error=failed requesting az metadata: Get http://169.254.169.254/metadata/instance/compute?api-version=2017-04-02: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers), metadata={}]
2019-05-07T21:51:04.651-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:193 add_cloud_metadata: received disposition for openstack after 3.007749s. result=[provider:openstack, error=failed requesting openstack metadata: Get http://169.254.169.254/2009-04-04/meta-data/placement/availability-zone: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers), metadata={}]
2019-05-07T21:51:04.651-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:200 add_cloud_metadata: timed-out waiting for all responses
2019-05-07T21:51:04.651-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:164 add_cloud_metadata: fetchMetadata ran for 3.007749s
2019-05-07T21:51:04.651-0400 INFO add_cloud_metadata/add_cloud_metadata.go:340 add_cloud_metadata: hosting provider type not detected.
2019-05-07T21:51:04.651-0400 DEBUG [processors] processors/processor.go:66 Processors: add_host_metadata=[netinfo.enabled=[false], cache.ttl=[5m0s]], add_cloud_metadata=null
2019-05-07T21:51:04.652-0400 DEBUG [publish] pipeline/consumer.go:137 start pipeline event consumer
2019-05-07T21:51:04.652-0400 INFO [publisher] pipeline/module.go:110 Beat name: TASKTEST
2019-05-07T21:51:04.654-0400 INFO [monitoring] log/log.go:117 Starting metrics logging every 30s
2019-05-07T21:51:04.654-0400 INFO kibana/client.go:118 Kibana url: http://AWSPERFMON01.blkmtnhosting.com:5601
2019-05-07T21:51:04.655-0400 DEBUG [service] service/service_windows.go:68 Windows is interactive: false

Any help would be greatly appreciated.

Thanks!

2

That looks relevant. Where is that in your config?

3

I'm not positive, but it looks like that was coming from the fact that I had the add_cloud_metadata processor enabled. I've turned it off and still no luck getting the service to start. Here's what the log looks like after I turned that off:

2019-05-08T11:17:00.977-0400 INFO instance/beat.go:611 Home path: [C:\elk\filebeat] Config path: [C:\elk\filebeat] Data path: [C:\ProgramData\filebeat] Logs path: [C:\ProgramData\filebeat\logs]
2019-05-08T11:17:00.987-0400 DEBUG [beat] instance/beat.go:648 Beat metadata path: C:\ProgramData\filebeat\meta.json
2019-05-08T11:17:00.987-0400 INFO instance/beat.go:618 Beat UUID: ef88501c-db97-49c3-bac8-ba2507324b82
2019-05-08T11:17:00.987-0400 DEBUG [seccomp] seccomp/seccomp.go:88 Syscall filtering is only supported on Linux
2019-05-08T11:17:00.988-0400 INFO [beat] instance/beat.go:931 Beat info {"system_info": {"beat": {"path": {"config": "C:\elk\filebeat", "data": "C:\ProgramData\filebeat", "home": "C:\elk\filebeat", "logs": "C:\ProgramData\filebeat\logs"}, "type": "filebeat", "uuid": "ef88501c-db97-49c3-bac8-ba2507324b82"}}}
2019-05-08T11:17:00.988-0400 INFO [beat] instance/beat.go:940 Build info {"system_info": {"build": {"commit": "a8ab26dd1f818d27c17c3049f643652c6a789d88", "libbeat": "6.7.2", "time": "2019-04-29T08:06:34.000Z", "version": "6.7.2"}}}
2019-05-08T11:17:00.988-0400 INFO [beat] instance/beat.go:943 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":4,"version":"go1.10.8"}}}
2019-05-08T11:17:00.997-0400 INFO [beat] instance/beat.go:947 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-04-30T14:39:43.33-04:00","name":"TASKTEST","ip":["fe80::1532:3fa4:13:8879/64","172.30.1.191/22","::1/128","127.0.0.1/8","fe80::5efe:ac1e:1bf/128"],"kernel_version":"6.3.9600.19304 (winblue_ltsb_escrow.190305-1818)","mac":["00:0c:29:34:3c:cb","00:00:00:00:00:00:00:e0"],"os":{"family":"windows","platform":"windows","name":"Windows Server 2012 R2 Standard","version":"6.3","major":3,"minor":0,"patch":0,"build":"9600.19323"},"timezone":"EDT","timezone_offset_sec":-14400,"id":"b25cc28c-1f1d-484c-aa1c-386e7b25bf59"}}}
2019-05-08T11:17:01.000-0400 INFO [beat] instance/beat.go:976 Process info {"system_info": {"process": {"cwd": "C:\Windows\system32", "exe": "C:\elk\filebeat\filebeat.exe", "name": "filebeat.exe", "pid": 6492, "ppid": 532, "start_time": "2019-05-08T11:17:00.875-0400"}}}
2019-05-08T11:17:01.000-0400 INFO instance/beat.go:280 Setup Beat: filebeat; Version: 6.7.2
2019-05-08T11:17:01.000-0400 DEBUG [beat] instance/beat.go:301 Initializing output plugins
2019-05-08T11:17:01.005-0400 DEBUG [processors] processors/processor.go:66 Processors: add_host_metadata=[netinfo.enabled=[false], cache.ttl=[5m0s]]
2019-05-08T11:17:01.006-0400 DEBUG [publish] pipeline/consumer.go:137 start pipeline event consumer
2019-05-08T11:17:01.006-0400 INFO [publisher] pipeline/module.go:110 Beat name: TASKTEST
2019-05-08T11:17:01.008-0400 INFO [monitoring] log/log.go:117 Starting metrics logging every 30s
2019-05-08T11:17:01.008-0400 DEBUG [service] service/service_windows.go:68 Windows is interactive: false
2019-05-08T11:17:01.009-0400 INFO kibana/client.go:118 Kibana url: http://AWSPERFMON01.blkmtnhosting.com:5601

Thank you!

4

does

Start-Service -Name "filebeat"

execute successfully or returns an error

5

That also fails with the generic error below. I've also found that I have the same issue after setting up metricbeat. Both of them will run from the command line and I now have them running from Windows Scheduled Tasks, but neither will run as a service. I've tried setting the services to run with 3 different users, but so far, no luck even though the app runs fine from the command line as those users.

Start-Service Error:
image

6

We've just sorted it for filebeat and metricbeat. The issue we were having was somehow related to the extra quotes in the binaryPathName for the service. Changing that line in install-service-filebeat.ps1 to look like this solved it for us:
-binaryPathName "$workdir\filebeat.exe -c $workdir\filebeat.yml --path.home $workdir --path.data C:\ProgramData\filebeat --path.logs C:\ProgramData\filebeat\logs"

2 Likes
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.