Filebeat windows service not starting

Hi,

I follow up to install Filebeat 7.10.1 on Win server 2016 as below link but could not starting the Filebeat service by powershell or services console.

Appreciate to your kindly help.

PS C:\Program Files\Filebeat> Start-Service filebeat
Start-Service : Failed to start service 'filebeat (filebeat)'.
At line:1 char:1
+ Start-Service filebeat
+ ~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController
   ServiceCommandException
    + FullyQualifiedErrorId : StartServiceFailed,Microsoft.PowerShell.Commands.StartServiceCommand

My filebeat.yml config content:

#---------------------------------- Input --------------------------------------
# Read in Internet Information Services (IIS) logs, ignoring directives.
filebeat.inputs:
  - type: log
    enabled: true
    exclude_lines:
      - '^#'
    paths:
      - "C:/inetpub/logs/LogFiles/*/*.log"

processors:
  #--------------------------------- Dissect -------------------------------------
  # Parse IIS log fields to variables. All fields must match in the tokenizer for log messages
  # to be processed. If you wish to add, or remove fields written to logs by IIS, the tokenizer
  # must be updated with the new values.
  #
  # https://www.elastic.co/guide/en/beats/filebeat/current/dissect.html
  # https://docs.microsoft.com/en-us/windows/win32/http/w3c-logging
  - dissect:
      tokenizer: "%{DATE} %{TIME} %{ssitename} %{scomputername} %{sip} %{METHOD} %{URI_PATH} %{URI_QUERY} %{sport} %{csusername} %{CLIENT_IP} %{csversion} %{USER_AGENT} %{REFERRER} %{HOST} %{STATUS_CODE} %{scsubstatus} %{scwin32status} %{SENT_BYTES} %{RECEIVED_BYTES} %{timetaken} %{CONTENT_TYPE} %{X_FORWARDED_FOR}"
      field: "message"
      target_prefix: "iis"

  #--------------------------------- Script --------------------------------------
  # Script conditionally formats URI_QUERY to valid values, and unescapes X_FORWARDED_FOR, 
  # USER_AGENT, and CONTENT_TYPE.
  - script:
      lang: javascript
      id: query
      source: >
        function process(event) {
            if (event.Get("iis.URI_QUERY") === "-") {
                event.Put("iis.URI_QUERY", "");
            } else {
                event.Put("iis.URI_QUERY", "?" + event.Get("iis.URI_QUERY"));
            }

            // Escape backslashes
            event.Put("iis.HOST", event.Get("iis.HOST").replace(/\\/g, "\\\\"));
            event.Put("iis.X_FORWARDED_FOR", event.Get("iis.X_FORWARDED_FOR").replace(/\\/g, "\\\\"));
            event.Put("iis.CONTENT_TYPE", event.Get("iis.CONTENT_TYPE").replace(/\\/g, "\\\\"));
            event.Put("iis.REFERRER", event.Get("iis.REFERRER").replace(/\\/g, "\\\\"));
            event.Put("iis.USER_AGENT", event.Get("iis.USER_AGENT").replace(/\\/g, "\\\\"));
            event.Put("iis.URI_PATH", event.Get("iis.URI_PATH").replace(/\\/g, "\\\\"));
            event.Put("iis.URI_QUERY", event.Get("iis.URI_QUERY").replace(/\\/g, "\\\\"));

            // Escape double quotes
            event.Put("iis.HOST", event.Get("iis.HOST").replace(/"/g, "\\\""));
            event.Put("iis.X_FORWARDED_FOR", event.Get("iis.X_FORWARDED_FOR").replace(/"/g, "\\\""));
            event.Put("iis.CONTENT_TYPE", event.Get("iis.CONTENT_TYPE").replace(/"/g, "\\\""));
            event.Put("iis.REFERRER", event.Get("iis.REFERRER").replace(/"/g, "\\\""));
            event.Put("iis.USER_AGENT", event.Get("iis.USER_AGENT").replace(/"/g, "\\\""));
            event.Put("iis.URI_PATH", event.Get("iis.URI_PATH").replace(/"/g, "\\\""));
            event.Put("iis.URI_QUERY", event.Get("iis.URI_QUERY").replace(/"/g, "\\\""));

            // Un-encode plus encoded white space
            event.Put("iis.X_FORWARDED_FOR", event.Get("iis.X_FORWARDED_FOR").replace(/\+/g, " "));
            event.Put("iis.USER_AGENT", event.Get("iis.USER_AGENT").replace(/\+/g, " "));
            event.Put("iis.CONTENT_TYPE", event.Get("iis.CONTENT_TYPE").replace(/\+/g, " "));
        }

#------------------------------- File output -----------------------------------
# Write out logs in WLALOG format to C:/inetpub/logs/wla. Defaults are to rotate at 10mb,
# keeping a max of 7 files, and appending an incrementing integer.
#
# https://www.elastic.co/guide/en/beats/filebeat/current/file-output.html
output.file:
  enabled: true
  path: "C:/inetpub/logs/wla"
  filename: iis.log
  codec.format:
    string: '@WLALOG "%{[iis.DATE]} %{[iis.TIME]}", "%{[iis.HOST]}", "%{[iis.CLIENT_IP]}", "%{[iis.X_FORWARDED_FOR]}", "%{[iis.METHOD]}", "%{[iis.URI_PATH]}%{[iis.URI_QUERY]}", "%{[iis.STATUS_CODE]}", "%{[iis.SENT_BYTES]}", "%{[iis.RECEIVED_BYTES]}", "%{[iis.REFERRER]}", "%{[iis.USER_AGENT]}", "%{[iis.CONTENT_TYPE]}"'

Hi,

Anyone have the same issue ?

Thanks,

Welcome to our community! :smiley:

Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you :slight_smile:

Hi,

Sorry for this inconvenience, i'm not sure how to correct/edit the content as your suggestion.

Can you try running the Filebeat binary directly and with debug and seeing what it says?

Hi warkolm,

First of all, I would to say thanks for your help.

Here is result as check config:

PS C:\Program Files\Filebeat> .\filebeat.exe -e test config
2021-01-12T00:18:43.875-0800    INFO    instance/beat.go:645    Home path: [C:\Program Files\Filebeat] Config path: [C:\
Program Files\Filebeat] Data path: [C:\Program Files\Filebeat\data] Logs path: [C:\Program Files\Filebeat\logs]
2021-01-12T00:18:43.876-0800    INFO    instance/beat.go:653    Beat ID: 72535466-1265-4111-8a5c-95535f797457
2021-01-12T00:18:43.878-0800    INFO    [beat]  instance/beat.go:981    Beat info       {"system_info": {"beat": {"path"
: {"config": "C:\\Program Files\\Filebeat", "data": "C:\\Program Files\\Filebeat\\data", "home": "C:\\Program Files\\Fil
ebeat", "logs": "C:\\Program Files\\Filebeat\\logs"}, "type": "filebeat", "uuid": "72535466-1265-4111-8a5c-95535f797457"
}}}
2021-01-12T00:18:43.878-0800    INFO    [beat]  instance/beat.go:990    Build info      {"system_info": {"build": {"comm
it": "1da173a9e716715a7a54bb3ff4db05b5c24fc8ce", "libbeat": "7.10.1", "time": "2020-12-04T23:27:16.000Z", "version": "7.
10.1"}}}
2021-01-12T00:18:43.878-0800    INFO    [beat]  instance/beat.go:993    Go runtime info {"system_info": {"go": {"os":"wi
ndows","arch":"amd64","max_procs":24,"version":"go1.14.12"}}}
2021-01-12T00:18:43.882-0800    INFO    [beat]  instance/beat.go:997    Host info       {"system_info": {"host": {"archi
tecture":"x86_64","boot_time":"2021-01-05T00:14:02.81-08:00","name":"se-web6-it-mk6p","ip":["10.100.0.7/22","::1/128","1
27.0.0.1/8"],"kernel_version":"10.0.14393.4104 (rs1_release.201202-1742)","mac":["42:01:0a:64:00:07"],"os":{"family":"wi
ndows","platform":"windows","name":"Windows Server 2016 Datacenter","version":"10.0","major":10,"minor":0,"patch":0,"bui
ld":"14393.4104"},"timezone":"PST","timezone_offset_sec":-28800,"id":"0a31c4db-6ef3-450f-a0ed-15c8fb7bb5f1"}}}
2021-01-12T00:18:43.882-0800    INFO    [beat]  instance/beat.go:1026   Process info    {"system_info": {"process": {"cw
d": "C:\\Program Files\\Filebeat", "exe": "C:\\Program Files\\Filebeat\\filebeat.exe", "name": "filebeat.exe", "pid": 20
164, "ppid": 26088, "start_time": "2021-01-12T00:18:42.865-0800"}}}
2021-01-12T00:18:43.882-0800    INFO    instance/beat.go:299    Setup Beat: filebeat; Version: 7.10.1
2021-01-12T00:18:43.883-0800    INFO    [file]  fileout/file.go:101     Initialized file output. path=C:\inetpub\logs\wl
a\iis.log max_size_bytes=10485760 max_backups=7 permissions=-rw-------
2021-01-12T00:18:43.883-0800    INFO    [publisher]     pipeline/module.go:113  Beat name: se-web6-it-mk6p
2021-01-12T00:18:43.889-0800    WARN    beater/filebeat.go:178  Filebeat is unable to load the Ingest Node pipelines for
 the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Inges
t Node pipelines or are using Logstash pipelines, you can ignore this warning.
Config OK

This is filebeat log:

2021-01-12T00:21:24.309-0800	INFO	instance/beat.go:645	Home path: [C:\Program Files\Filebeat] Config path: [C:\Program Files\Filebeat] Data path: [C:\ProgramData\filebeat] Logs path: [C:\ProgramData\filebeat\logs]
2021-01-12T00:21:24.310-0800	INFO	instance/beat.go:653	Beat ID: 85d8abd9-6027-4acd-bc97-1b0664f33daa
2021-01-12T00:21:24.312-0800	INFO	[beat]	instance/beat.go:981	Beat info	{"system_info": {"beat": {"path": {"config": "C:\\Program Files\\Filebeat", "data": "C:\\ProgramData\\filebeat", "home": "C:\\Program Files\\Filebeat", "logs": "C:\\ProgramData\\filebeat\\logs"}, "type": "filebeat", "uuid": "85d8abd9-6027-4acd-bc97-1b0664f33daa"}}}
2021-01-12T00:21:24.312-0800	INFO	[beat]	instance/beat.go:990	Build info	{"system_info": {"build": {"commit": "1da173a9e716715a7a54bb3ff4db05b5c24fc8ce", "libbeat": "7.10.1", "time": "2020-12-04T23:27:16.000Z", "version": "7.10.1"}}}
2021-01-12T00:21:24.313-0800	INFO	[beat]	instance/beat.go:993	Go runtime info	{"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":24,"version":"go1.14.12"}}}
2021-01-12T00:21:24.318-0800	INFO	[beat]	instance/beat.go:997	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-01-05T00:14:02.81-08:00","name":"se-web6-it-mk6p","ip":["10.100.0.7/22","::1/128","127.0.0.1/8"],"kernel_version":"10.0.14393.4104 (rs1_release.201202-1742)","mac":["42:01:0a:64:00:07"],"os":{"family":"windows","platform":"windows","name":"Windows Server 2016 Datacenter","version":"10.0","major":10,"minor":0,"patch":0,"build":"14393.4104"},"timezone":"PST","timezone_offset_sec":-28800,"id":"0a31c4db-6ef3-450f-a0ed-15c8fb7bb5f1"}}}
2021-01-12T00:21:24.318-0800	INFO	[beat]	instance/beat.go:1026	Process info	{"system_info": {"process": {"cwd": "C:\\Windows\\system32", "exe": "C:\\Program Files\\Filebeat\\filebeat.exe", "name": "filebeat.exe", "pid": 30044, "ppid": 872, "start_time": "2021-01-12T00:21:23.126-0800"}}}
2021-01-12T00:21:24.318-0800	INFO	instance/beat.go:299	Setup Beat: filebeat; Version: 7.10.1
2021-01-12T00:21:24.318-0800	INFO	[file]	fileout/file.go:101	Initialized file output. path=C:\inetpub\logs\wla\iis.log max_size_bytes=10485760 max_backups=7 permissions=-rw-------
2021-01-12T00:21:24.319-0800	INFO	[publisher]	pipeline/module.go:113	Beat name: se-web6-it-mk6p
2021-01-12T00:21:24.322-0800	WARN	beater/filebeat.go:178	Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2021-01-12T00:21:24.322-0800	INFO	instance/beat.go:455	filebeat start running.
2021-01-12T00:21:24.322-0800	INFO	[monitoring]	log/log.go:118	Starting metrics logging every 30s
2021-01-12T00:21:24.333-0800	INFO	memlog/store.go:119	Loading data file of 'C:\ProgramData\filebeat\registry\filebeat' succeeded. Active transaction id=2044767
2021-01-12T00:21:24.722-0800	INFO	memlog/store.go:124	Finished loading transaction log file for 'C:\ProgramData\filebeat\registry\filebeat'. Active transaction id=2065127
2021-01-12T00:21:24.724-0800	WARN	beater/filebeat.go:381	Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2021-01-12T00:21:24.728-0800	INFO	[registrar]	registrar/registrar.go:109	States Loaded from registrar: 536
2021-01-12T00:21:24.728-0800	INFO	[crawler]	beater/crawler.go:71	Loading Inputs: 1

Here is the result for running command:
PS C:\Program Files\Filebeat> .\filebeat.exe -c filebeat.yml -e -d "*"

{Timestamp:0001-01-01 00:00:00 +0000 UT
C Meta:null Fields:null Private:{Id:native::524288-81651-3962916764 PrevId: Finished:true Fileinfo: Source:C:\inetpub\logs\LogFiles\W3SVC6\u_ex180530.log Offset:4138444 Timest
amp:2021-01-09 11:48:27.0786609 -0800 PST TTL:-1ns Type:log Meta:map FileStateOS:524288-81651-3962916764 IdentifierName:native} TimeSeries:false}
2021-01-12T00:35:18.847-0800 DEBUG [registrar] registrar/registrar.go:231 Registrar state updates processed. Count: 1
2021-01-12T00:35:18.849-0800 DEBUG [input] file/states.go:68 New state added for C:\inetpub\logs\LogFiles\W3SVC7\u_ex190228.log
2021-01-12T00:35:18.850-0800 DEBUG [acker] beater/acker.go:59 stateful ack {"count": 1}
2021-01-12T00:35:18.850-0800 DEBUG [registrar] registrar/registrar.go:254 Registrar states cleaned up. Before: 8677, After: 8677, Pending: 0
2021-01-12T00:35:18.851-0800 DEBUG [registrar] registrar/registrar.go:201 Registry file updated. 8677 active states.
2021-01-12T00:35:19.130-0800 DEBUG [registrar] registrar/registrar.go:264 Processing 1 events
2021-01-12T00:35:19.130-0800 DEBUG [publisher] pipeline/client.go:231 Pipeline client receives callback 'onFilteredOut' for event: {Timestamp:0001-01-01 00:00:00 +0000 UT
C Meta:null Fields:null Private:{Id:native::10158080-367138-3962916764 PrevId: Finished:true Fileinfo: Source:C:\inetpub\logs\LogFiles\W3SVC7\u_ex190228.log Offset:2196283 Tim
estamp:2021-01-12 00:09:19.2842529 -0800 PST TTL:-1ns Type:log Meta:map FileStateOS:10158080-367138-3962916764 IdentifierName:native} TimeSeries:false}
2021-01-12T00:35:19.130-0800 DEBUG [registrar] registrar/registrar.go:231 Registrar state updates processed. Count: 1
2021-01-12T00:35:19.133-0800 DEBUG [input] file/states.go:68 New state added for C:\inetpub\logs\LogFiles\W3SVC13\u_ex180923.log
2021-01-12T00:35:19.133-0800 DEBUG [registrar] registrar/registrar.go:254 Registrar states cleaned up. Before: 8677, After: 8677, Pending: 0
2021-01-12T00:35:19.134-0800 DEBUG [acker] beater/acker.go:59 stateful ack {"count": 1}
2021-01-12T00:35:19.134-0800 DEBUG [registrar] registrar/registrar.go:201 Registry file updated. 8677 active states.

Pls review and give your advise if any information need to provide.

Thanks

Please don't post pictures of text, they are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them :slight_smile:

I've just done correct the mistake.

Thanks for your reminder.

1 Like

I can't figure out why this is happening sorry, hopefully one of the devs will stop by and comment :slight_smile:

Thanks for adding all that detail!

:worried:

hi @vuong43807, is the iis.log file being created when running from cmd, does it contain any logs?
Also, when installing the service , were you running with elevated permission?

Hi @MarianaD,

We got many log look like below format

@WLALOG "2021-01-12 00:08:20", "www.abc.com", "130.211.0.157", "73.34.198.43, 152.199.98.136, 152.195.125.113, x.x.x.x, "GET", "/p/sporti-sofia-flowy-racerback-tankini-top-8181499/?&color=9325&size=9467&utm_source=google&utm_medium=cpc&utm_campaign=google-shopping", "200", "85060", "2405", "https://www.google.com/", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36", "-"

@WLALOG "2021-01-05 00:51:11", “m.abc.com", "35.191.8.74", "35.184.179.37, 72.21.87.116, 152.195.124.185, x.x.x.x”, "GET", "/productbrand.asp?Caching=Y&Brand=1409", "200", "142520", "579", "-", "-", "-"

@WLALOG "2021-01-01 01:11:56", “[m.abc.com](http://m.abc.com), "35.191.8.77", "35.184.179.37, 72.21.87.116, 152.195.124.185, x.x.x.x, "GET", "/productbrand.asp?Caching=Y&Brand=1565", "200", "141590", "577", "-", "-", "-"

I alway installing the service with administrator permission . Any missing with this step ?

Could you confirm if the iis.log file being created with amount of content that mean the Filebeat working despite of running on foreground mode ?

Thanks,

:thinking:

I'm going to install the new version of Filebeat to see if it can fix the issue.
Keep you updated.

Thanks,

Hi,

After installed the latest version 7.11.1 , we still get stuff with services starting.
"Windows could not start the filebeat service on Local Computer.
Error 1067: The process terminated unexpectedly"

Appreciate for your kindly help if possible.

:worried:

Hi,

It seems this is encountering a bug with Filebeat's fileout/file.go module, where it seems to be trying to access an object's key which does not exist but I don't know how to fix it out.

We are using Filebeat with Alertlogic on Windows server 2016 datacenter.

Anyone get the same issue ?

Thanks,

ERROR [file] fileout/file.go:127 Failed to serialize the event: key not found
github.com/elastic/beats/v7/libbeat/common.init
/go/src/github.com/elastic/beats/libbeat/common/mapstr.go:41
runtime.doInit
/usr/local/go/src/runtime/proc.go:5474
runtime.doInit
/usr/local/go/src/runtime/proc.go:5469
runtime.doInit
/usr/local/go/src/runtime/proc.go:5469
runtime.doInit
/usr/local/go/src/runtime/proc.go:5469
runtime.doInit
/usr/local/go/src/runtime/proc.go:5469
runtime.main
/usr/local/go/src/runtime/proc.go:190
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1373

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.