Winlogbeat - unable to start service on windows

I'm using the ELK stack as a syslog server on an debian server in order to gather logs from Cisco devices and Windows server.

Everything was doing fine until I wanted to use HTTPS instead of HTTP, now kibana and Elasticsearch are only available on https.

So I changed my winlogbeat config on my Windows server as below :

setup.kibana:
  hosts: ["https://<server_ip>:5601"]
  protocol: "https"
  ssl.certificate_authorities: ["certs/ca.crt"]
  ssl.enabled: true
  username: "elastic"
  password: "<my_password>"

output.elasticsearch:
  hosts: ["https://<server_IP>:9200"]
  username: "elastic"
  password: "<my_password>"
  ssl.certificate_authorities: ["certs/ca.crt"]
  ssl.enabled: true
  pipeline: "winlogbeat-%{[agent.version]}-routing"

these lines are the only ones I changed when I went from HTTP to HTTPS, and when using the winlogbeat tool, I get this :

PS C:\Program Files\Winlogbeat> .\winlogbeat.exe test config
Config OK
PS C:\Program Files\Winlogbeat> .\winlogbeat.exe test output
elasticsearch: https://<server_IP>:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: <server_IP>
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 8.11.3

Everything looks fine without any issue. But when I try starting the service :

PS C:\Program Files\Winlogbeat> Start-Service winlogbeat
Start-Service : Le service «winlogbeat (winlogbeat)» ne peut pas démarrer en raison de l'erreur suivante: Impossible
de démarrer le service winlogbeat sur l'ordinateur '.'.
Au caractère Ligne:1 : 1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service],
   ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

and same with services.msc.

Even more surprising, when I use the .exe (as below) file to run it, it works !

.\winlogbeat.exe -c .\winlogbeat.yml run -e -d "*"

And with this command, I get the logs in the ELK. For me it's nonsense, but I may have forgotten something

I tried to figure out what was wrong (some rights in the folder, files missing, wrong arguments), but I found nothing.
I tried uninstalling and reinstalling the service, but didn't worked either.

And when I go back to an HTTP config, everything works fine, but I just followed the documentation here

here's my config in the ELK server :

elasticsearch.yml :

node.name: node-1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: "server_ip"
http.port: 9200
xpack.security.enabled: true

xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  key: certs/LRFSYSLOG01S.key
  certificate: certs/LRFSYSLOG01S.crt
  certificate_authorities: certs/ca.crt

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  key: certs/LRFSYSLOG01S.key
  certificate: certs/LRFSYSLOG01S.crt
  certificate_authorities: certs/ca.crt
  verification_mode: certificate

kibana.yml:

server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/certs/LRFSYSLOG01S.crt
server.ssl.key: /etc/kibana/certs/LRFSYSLOG01S.key
elasticsearch.hosts: ["https://<server_ip>:9200"]

elasticsearch.username: "kibana"
elasticsearch.password: "<password>"

elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/ca.crt" ]

filebeat.yml:

setup.kibana:
  host: "https://<server_ip>:5601"
  ssl.certificate_authorities: ["certs/ca.crt"]
  username: "elastic"
  password: "<password>"


output.elasticsearch:
  hosts: ["https://<server_ip>:9200"]

  username: "elastic"
  password: "<password>"
  ssl.certificate_authorities: ["/etc/filebeat/certs/ca.crt"]

I don't think the issue come from my server config, but from my Windows server. I hope someone have an explanation to this issue

I also tried to modify the time windows allow for a service to start (with help of this website : How to extend the timeout for services if they fail to start or stop (codetwo.com))

but after a reboot of the server, still the same :
image
it's in French, but it says, error 1053: the service did not respond quickly enough to the control launch request

and still the same with starting it with PowerShell :

PS C:\Windows\system32> Start-Service winlogbeat
Start-Service : The "winlogbeat (winlogbeat)" service cannot be started due to the following error: Impossible
to start winlogbeat service on computer '.'.
At character Ligne:1 : 1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service],
 ServiceCommandException
 + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

but as before, launching wingbeat in console mode (with the .exe file) is working perfectly fine :

PS C:\Program Files\Winlogbeat> .\winlogbeat.exe -c .\Config_test.yml run -e -d "*"

and I do recieve the logs in my ELK server :

I also looked at the issues talking about the windows service that doesn't want to start in the forum, but none of them got a solution.

I found a temporary solution :

  • Creating a .bat file to execute the cmd command I used above
  • Creating a scheduled task that launches this script in a cmd instance as an admin
  • trying it and seeing those logs in the ELK.

This is working for now, but it's not the best solution.

I forgot to give the version of each element :

  • filebeat v 8.11.3
  • debian 12
  • kibana v 8.11.3
  • winlogbeat v 8.11.4
  • elasticsearch v 8.11.3

Hi @fracorbas could you see if your issue might be similar to this one from 8.13? Winlogbeat 8.13.0 dont start service - #6 by tapiojaa

If its the same problem as that topic, 8.13 has trailing / on some of the service paths if it was installed via the MSI. The linked comment has a suggestion on how to fix it, or it should be fixed in the next release.

Hi @NickFritts,

thanks for the answer. I did not install winlogbeat with the msi installer, but directly using the .zip file and installing the service with the powershell script.

I tried doing what was in the topic you mentionned, but I just got an error, because I guess, I don't have the same installation, and so I can't use the sc command on winlogbeat.

It does not seem to be the same issue, because I can start winlogbeat in an administrator shell, but not as a service (with PowerShell or services.msc).

It seems like my problem is because of HTTPS, but I found no topic about this, did I forget something ?

@fracorbas Since you're using powershell, I think you would have to run sc.exe instead of sc

Could you try that? And if it does work can you show what the output of sc.exe qc winlogbeat (but change winlogbeat to the name of your service if it is different.

Hi @NickFritts, I didn't think about using PowerShell, but it works, here's the result :

PS C:\Windows\system32> .\sc.exe qc winlogbeat
[SC] QueryServiceConfig réussite(s)

SERVICE_NAME: winlogbeat
        TYPE               : 10  WIN32_OWN_PROCESS
        START_TYPE         : 2   AUTO_START  (DELAYED)
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : "C:\Program Files\Winlogbeat\winlogbeat.exe" --environment=windows_service -c "C:\Program Files\Winlogbeat\winlogbeat.yml" --path.home "C:\Program Files\Winlogbeat" --path.data "C:\ProgramData\winlogbeat" --path.logs "C:\ProgramData\winlogbeat\logs" -E logging.files.redirect_stderr=true
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : winlogbeat
        DEPENDENCIES       :
        SERVICE_START_NAME : LocalSystem
PS C:\Windows\system32>

So I tried using the command from the topic :

PS C:\Windows\system32> .\sc.exe config winlogbeat binPath="\"C:\Program Files\Winlogbeat\winlogbeat.exe" --path.home \"C:\Program Files\Winlogbeat\" --path.config \"C:\Program Files\Winlogbeat\" --path.data \"C:\Program Files\Winlogbeat\" --path.logs \"C:\Program Files\Winlogbeat\" -E logging.files.redirect_stderr=true"
DESCRIPTION:
        Modifie l'entrée d'un service dans le Registre et la base de données des services.
UTILISATION:
        sc <serveur> config [nom service] <option1> <option2>...

OPTIONS:
REMARQUE: le nom d'option contient le signe Ă©gal.
           Un espace est nécessaire entre le signe égal et la valeur.
      Pour supprimer la dépendance, utilisez un seul / comme valeur de dépendance.
 type= <own|share|interact|kernel|filesys|rec|adapt|userown|usershare>
 start= <boot|system|auto|demand|disabled|delayed-auto>
 error= <normal|severe|critical|ignore>
 binPath= <NomCheminAccèsBinaire au fichier .exe>
 group= <LoadOrderGroup>
 tag= <yes|no>
 depend= <Dépendances (séparées par / (barre oblique))>
 obj= <NomCompte|NomObjet>
 DisplayName= <nom complet>
 password= <mot de passe>

and I got an error that I don't understand. It tells me to put a space between the "=" and the value.
When I do so, it tells me the same.

because I'm kinda dumb and don't understand how this command works, I just used the brute way :
going to regedit (here's tht PATH : HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\winlogbeat), look for the specific key and modify it :
image

the value before was exactly the one displayed by the sc command

and now I have :

PS C:\Windows\system32> .\sc.exe qc winlogbeat
[SC] QueryServiceConfig réussite(s)

SERVICE_NAME: winlogbeat
        TYPE               : 10  WIN32_OWN_PROCESS
        START_TYPE         : 2   AUTO_START  (DELAYED)
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : "\"C:\Program Files\Winlogbeat\winlogbeat.exe\" --environment=windows_service -c \"C:\Program Files\Winlogbeat\winlogbeat.yml\" --path.home \"C:\Program Files\Winlogbeat\" --path.config \"C:\Program Files\Winlogbeat\" --path.data \"C:\Program Files\Winlogbeat\" --path.logs \"C:\Program Files\Winlogbeat\" -E logging.files.redirect_stderr=true"
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : winlogbeat
        DEPENDENCIES       :
        SERVICE_START_NAME : LocalSystem

so the modifications have been taken, but :

PS C:\Windows\system32> Start-Service winlogbeat
Start-Service : Le service « winlogbeat (winlogbeat) » ne peut pas démarrer en raison de l'erreur suivante : Impossible de démarrer le service winlogbeat sur l'ordinateur '.'.
Au caractère Ligne:1 : 1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service], ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

I still got the same error.

Are the changes I've made the ones expected by the sc command ?

I think both binpaths should have worked.

If you use the binpath you currently have or the one from before and just attempt to run it from the command line do either of them work?

If they don't, can you try removing one extra option at a time until they do and let me know which one makes it start working?

from a command line, it's working (with the latest binPath) :

but the service is not :

PS C:\Program Files\Winlogbeat> Start-Service winlogbeat
Start-Service : Le service « winlogbeat (winlogbeat) » ne peut pas démarrer en raison de l'erreur suivante : Impossible de démarrer le service winlogbeat sur l'ordinateur '.'.
Au caractère Ligne:1 : 1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service], ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

with "\"C:\Program Files\Winlogbeat\winlogbeat.exe\" -c \"C:\Program Files\Winlogbeat\winlogbeat.yml\" --path.home \"C:\Program Files\Winlogbeat\" --path.config \"C:\Program Files\Winlogbeat\" --path.data \"C:\Program Files\Winlogbeat\" --path.logs \"C:\Program Files\Winlogbeat\" -E logging.files.redirect_stderr=true" as a binPath, I get :

PS C:\Program Files\Winlogbeat> Start-Service winlogbeat
Start-Service : Le service « winlogbeat (winlogbeat) » ne peut pas démarrer en raison de l'erreur suivante : Impossible de démarrer le service winlogbeat sur l'ordinateur '.'.
Au caractère Ligne:1 : 1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service], ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

PS C:\Program Files\Winlogbeat> .\winlogbeat.exe -c .\Config_test.yml run -e -d "*"
{"log.level":"info","@timestamp":"2024-04-30T12:23:24.265+0200","log.origin":{"file.name":"instance/beat.go","file.line":806},"message":"Home path: [C:\\Program Files\\Winlogbeat] Config path: [C:\\Program Files\\Winlogbeat] Data path: [C:\\Program Files\\Winlogbeat\\data] Logs path: [C:\\Program Files\\Winlogbeat\\logs]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:23:24.364+0200","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":897},"message":"Beat metadata path: C:\\Program Files\\Winlogbeat\\data\\meta.json","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-30T12:23:24.369+0200","log.origin":{"file.name":"instance/beat.go","file.line":814},"message":"Beat ID: 8dbbbb47-2410-4c0c-b688-6c72cb8b6d2a","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:23:24.414+0200","log.logger":"conditions","log.origin":{"file.name":"conditions/conditions.go","file.line":98},"message":"New condition contains: map[]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:23:24.414+0200","log.logger":"conditions","log.origin":{"file.name":"conditions/conditions.go","file.line":98},"message":"New condition !contains: map[]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:23:24.418+0200","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":130},"message":"add_cloud_metadata: starting to fetch metadata, timeout=3s","service.name":"winlogbeat","ecs.version":"1.6.0"}

with "\"C:\Program Files\Winlogbeat\winlogbeat.exe\" --path.home \"C:\Program Files\Winlogbeat\" --path.config \"C:\Program Files\Winlogbeat\" --path.data \"C:\Program Files\Winlogbeat\" --path.logs \"C:\Program Files\Winlogbeat\" -E logging.files.redirect_stderr=true", I get :

PS C:\Program Files\Winlogbeat> Start-Service winlogbeat
Start-Service : Le service « winlogbeat (winlogbeat) » ne peut pas démarrer en raison de l'erreur suivante : Impossible de démarrer le service winlogbeat sur l'ordinateur '.'.
Au caractère Ligne:1 : 1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service], ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

with "\"C:\Program Files\Winlogbeat\winlogbeat.exe\" --path.home \"C:\Program Files\Winlogbeat\" --path.data \"C:\Program Files\Winlogbeat\" --path.logs \"C:\Program Files\Winlogbeat\" -E logging.files.redirect_stderr=true" I have the same

with "\"C:\Program Files\Winlogbeat\winlogbeat.exe\" --path.home \"C:\Program Files\Winlogbeat\" --path.logs \"C:\Program Files\Winlogbeat\" -E logging.files.redirect_stderr=true" same

with "\"C:\Program Files\Winlogbeat\winlogbeat.exe\" -E logging.files.redirect_stderr=true" same

with "\"C:\Program Files\Winlogbeat\winlogbeat.exe\"" same

but with every dataPath, running the winlogbeat with the .exe works :

PS C:\Program Files\Winlogbeat> .\winlogbeat.exe -c .\Config_test.yml run -e -d "*"
{"log.level":"info","@timestamp":"2024-04-30T12:29:26.328+0200","log.origin":{"file.name":"instance/beat.go","file.line":806},"message":"Home path: [C:\\Program Files\\Winlogbeat] Config path: [C:\\Program Files\\Winlogbeat] Data path: [C:\\Program Files\\Winlogbeat\\data] Logs path: [C:\\Program Files\\Winlogbeat\\logs]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:29:26.406+0200","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":897},"message":"Beat metadata path: C:\\Program Files\\Winlogbeat\\data\\meta.json","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-30T12:29:26.411+0200","log.origin":{"file.name":"instance/beat.go","file.line":814},"message":"Beat ID: 8dbbbb47-2410-4c0c-b688-6c72cb8b6d2a","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:29:26.469+0200","log.logger":"conditions","log.origin":{"file.name":"conditions/conditions.go","file.line":98},"message":"New condition contains: map[]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:29:26.469+0200","log.logger":"conditions","log.origin":{"file.name":"conditions/conditions.go","file.line":98},"message":"New condition !contains: map[]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:29:26.476+0200","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":130},"message":"add_cloud_metadata: starting to fetch metadata, timeout=3s","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:29:29.480+0200","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":167},"message":"add_cloud_metadata: received disposition for gcp after 3.0020806s. result=[provider:gcp, error=failed requesting gcp metadata: Get \"http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json\": dial tcp 169.254.169.254:80: i/o timeout, metadata={}]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2024-04-30T12:29:29.481+0200","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":173},"message":"add_cloud_metadata: received error failed requesting gcp metadata: Get \"http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json\": dial tcp 169.254.169.254:80: i/o timeout","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:29:29.485+0200","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":176},"message":"add_cloud_metadata: timed-out waiting for all responses","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:29:29.486+0200","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/providers.go","file.line":133},"message":"add_cloud_metadata: fetchMetadata ran for 3.0080136s","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-30T12:29:29.487+0200","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/add_cloud_metadata.go","file.line":100},"message":"add_cloud_metadata: hosting provider type not detected.","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:29:29.487+0200","log.logger":"processors","log.origin":{"file.name":"processors/processor.go","file.line":114},"message":"Generated new processors: add_host_metadata=[netinfo.enabled=[true], cache.ttl=[5m0s]], condition=!contains: map[], add_cloud_metadata={}","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-04-30T12:29:29.489+0200","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":97},"message":"Syscall filtering is only supported on Linux","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-30T12:29:29.490+0200","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1332},"message":"Beat info","service.name":"winlogbeat","system_info":{"beat":{"path":{"config":"C:\\Program Files\\Winlogbeat","data":"C:\\Program Files\\Winlogbeat\\data","home":"C:\\Program Files\\Winlogbeat","logs":"C:\\Program Files\\Winlogbeat\\logs"},"type":"winlogbeat","uuid":"8dbbbb47-2410-4c0c-b688-6c72cb8b6d2a"},"ecs.version":"1.6.0"}}

and the config still works :

PS C:\Program Files\Winlogbeat> .\winlogbeat.exe test output
elasticsearch: https://<server_ip>:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: <server_ip>
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 8.13.2

I just upgraded my ELK to 8.13.2

BUT
by setting the config in HTTP with the latest binPath => not working
with HTTP with the before binPath => working :upside_down_face:
with HTTPS with the before binPath => not working
with HTTPS with the latest binPath => not working

so I stayed with the first binPath, but it's weird that I can use winlogbeat with the .exe and is working perfectly fine, but when I start the service, it's not working.

I though, maybe the name of my file was wrong, so I renamed the file Config_test.yml to winlogbeat.yml because it's the one used by default. and it changed nothing.

So :

  • the .exe works with everything (he doesn't care at all about the binPath I think)
  • the service works only in http with the first binPath

is there an argument that we can add to tell the service to work in HTTPS or something ? Or it should be working with the config I have ?

I just found in the logs that I have this :

Process Create: RuleName: - UtcTime: 2024-04-30 10:50:09.054 ProcessGuid: {fb27ab9b-cce1-6630-1242-000000001300} ProcessId: 12468 Image: C:\Program Files\Winlogbeat\winlogbeat.exe FileVersion: 8.11.4 Description: Winlogbeat ships Windows event logs to Elasticsearch or Logstash. Product: Winlogbeat Company: Elastic OriginalFileName: winlogbeat.exe CommandLine: "C:\Program Files\Winlogbeat\winlogbeat.exe" --environment=windows_service -c "C:\Program Files\Winlogbeat\winlogbeat.yml" --path.home "C:\Program Files\Winlogbeat\" --path.data "C:\ProgramData\winlogbeat\" --path.logs "C:\ProgramData\winlogbeat\logs\" -E logging.files.redirect_stderr=true CurrentDirectory: C:\Windows\system32\ User: AUTORITE NT\Système

so I changed the binPath a bit to see if something changed :
"C:\Program Files\Winlogbeat\winlogbeat.exe" --environment=windows_service -c "C:\Program Files\Winlogbeat\winlogbeat.yml" --path.home "C:\Program Files\Winlogbeat\" --path.data "C:\ProgramData\winlogbeat\" --path.logs "C:\ProgramData\winlogbeat\logs\" -E logging.files.redirect_stderr=true
I added a \ at the end of each folder name

but it's not working either.

And I got this error too :
Le dépassement de délai (25000 millisecondes) a été atteint lors de l’attente de la connexion du service winlogbeat
The timeout (25000 milliseconds) was reached while waiting for the winlogbeat service to connect.

@fracorbas unfortunately I think we've reached the limit of what I can troubleshoot. I'm going to have another engineer on my team jump in to assist further.

I do want to dig in further though to make sure if there's a bug we get it fixed in the next release.

1 Like

Hi,

Today I tried using the latest version : the 8.13, I downloaded the zip, copied every needed files I created, and put the new version in the folder the 8.11 was in.

I first tried to do the tests :

PS C:\Program Files\Winlogbeat> .\winlogbeat.exe -c .\winlogbeat.yml test config
Config OK
PS C:\Program Files\Winlogbeat> .\winlogbeat.exe -c .\winlogbeat.yml test output
elasticsearch: https://<some_IP>:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: <some_IP>
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 8.13.2
PS C:\Program Files\Winlogbeat> .\install-service-winlogbeat.ps1

Status   Name               DisplayName
------   ----               -----------
Stopped  winlogbeat         winlogbeat

and the test are working, the service is installed too :

PS C:\Program Files\Winlogbeat> C:\Windows\System32\sc.exe qc winlogbeat
[SC] QueryServiceConfig réussite(s)

SERVICE_NAME: winlogbeat
        TYPE               : 10  WIN32_OWN_PROCESS
        START_TYPE         : 2   AUTO_START  (DELAYED)
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : "C:\Program Files\Winlogbeat\winlogbeat.exe" --environment=windows_service -c "C:\Program Files\Winlogbeat\winlogbeat.yml" --path.home "C:\Program Files\Winlogbeat" --path.data "C:\ProgramData\winlogbeat" --path.logs "C:\ProgramData\winlogbeat\logs" -E keystore.path="C:\Program Files\Winlogbeat\data\winlogbeat.keystore" -E logging.files.redirect_stderr=true
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : winlogbeat
        DEPENDENCIES       :
        SERVICE_START_NAME : LocalSystem

but as before :

PS C:\Program Files\Winlogbeat> Start-Service winlogbeat
Start-Service : Le service «winlogbeat (winlogbeat)» ne peut pas démarrer en raison de l'erreur suivante: Impossible
de démarrer le service winlogbeat sur l'ordinateur '.'.
Au caractère Ligne:1 : 1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service],
   ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

And same with services.msc, it's impossible to start the service.

But as before, using the .exe to start it is working fine .\winlogbeat.exe -c .\winlogbeat.yml run -e -d "*" and I get the logs in kibana.

I changed nothing to the config :

setup.kibana:

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  hosts: ["https://<some_IP>:5601"]
  protocol: "https"
  ssl.certificate_authorities: ["certs/ca.crt"]
  ssl.enabled: true
  username: "elastic"
  password: "<some_password>"

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["https://<some_IP>:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "<some_password>"
  ssl.certificate_authorities: ["certs/ca.crt"]
  ssl.enabled: true


  # Pipeline to route events to security, sysmon, or powershell pipelines.
  pipeline: "winlogbeat-%{[agent.version]}-routing"

The certificates are working (by looking at the test output command) but the service won't start for a mysterious reason.

I also tried to see if it was working with HTTP :

setup.kibana:
  hosts: ["http://<some_IP>:5601"]
  protocol: "http"

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://some_IP>:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "http"

And this is what I got when starting the service

PS C:\Program Files\Winlogbeat> Start-Service winlogbeat
PS C:\Program Files\Winlogbeat>

the service is working :
image

the issue is not only on 8.11 but also on 8.13, or it may be me that do something wrong, but I found nothing in the doc that is different to what I did.

Today I got an idea : the binPath may is the way the .exe is used, so if I use this in the same way I'm using it in my scheduled task, it should work

I changed the binpath to

"C:\Program Files\Winlogbeat\winlogbeat.exe" --environment=windows_service -c "C:\Program Files\Winlogbeat\winlogbeat.yml" --path.home "C:\Program Files\Winlogbeat" --path.data "C:\ProgramData\winlogbeat" --path.logs "C:\ProgramData\winlogbeat\logs" -E keystore.path="C:\Program Files\Winlogbeat\data\winlogbeat.keystore" -E logging.files.redirect_stderr=true run

I just added a run at the end
And it's not working.

I also tried to set the binpath to

"C:\Program Files\Winlogbeat\winlogbeat.exe" -c "C:\Program Files\Winlogbeat\winlogbeat.yml" run

But, it's not working either.

I really don't understand, I just hope I did not do something really stupid somewhere that causes this issue.

And some version of what I used changed :

  • elasticsearch : 8.13.2
  • kibana : 8.13.2
  • winlogbeat 8.13.0

I thought, maybe the version was doing something weird, but it's not.

Hello!, since the issue seems to only appear when https is enabled, do you know if your certificate is trusted system wide so it can be used by services?

If not, you could try the steps described here and see if trusting it either Service or Computer account helps.

Hi @marc.guasch,

I did import the certificate before, but the thing is that my server is in a restricted network, so it cannot join my PKI server.
I imported the root-ca of my PKI in the "personal" and "trusted authorities" of my local server, but the error is still the same.

If the error is from my certificate, why the .exe does not show any type of error when I do the test config and test output command ?

There are several certificate stores (Local User , System , Services) . Usually windows services run as a special account called System which needs certificates to be added to its store.

If your certificate is added to the local user account alone the service can't verify the certificate.

Is there a chance you can test this in another environment to confirm it is the issue?

Sorry that it took me some time, I missclicked and deleted all my certificates... So I done it from the beginning :

  • generate a CA with the certutil tool ./elasticsearch-certutil ca -pem I want to have the key
  • generate a certificate for elastic with that CA and with an instances.yml file :
./elasticsearch-certutil cert --in /root/instances.yml --out LRFSYSLOG01S.zip --ca-cert /usr/share/elasticsearch/certs/CA/ca.crt --ca-key /usr/share/elasticsearch/certs/CA/ca.key -pem
  • putting the new certificate, certificate key and CA into my certs folder of elastic, filebeat and kibana.

Restarting the services and I got no error, now my certificate is loaded when I go in the kibana interface.

  • the last thing is importing the CA I generated into the Windows server I have, so I went there in admin :
    image
    it's in the root certificate authorities of my machine. After importing my new CA :

I went to my profile to see if this new CA was loaded, and it was (I also looked at different users, and he was there), so I conclude this manipulation loaded the certificate in every users profile.

  • finally, I loaded the certificate I generated with the CA in the personal certificate section of the users :
    image
    so windows recognize the root CA and the certificate I generated.

But now, I'm just trying to figure out where to add those certificates in the System user. I know he exists, but I find him nowhere. The only thing I found was to add the certificates in the "local computer" in the certificates app in mmc which I did.

and for the result :

PS C:\Program Files\Winlogbeat> Start-Service winlogbeat
Start-Service : Impossible de démarrer le service « winlogbeat (winlogbeat) ».
Au caractère Ligne:1 : 1
+ Start-Service winlogbeat
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service],
   ServiceCommandException
    + FullyQualifiedErrorId : StartServiceFailed,Microsoft.PowerShell.Commands.StartServiceCommand

it's still the same.

is there a command to add directly the root-ca to the user or what I did should work ?

and again, with the .exe :

PS C:\Program Files\Winlogbeat> .\winlogbeat.exe test output -c .\winlogbeat.yml
elasticsearch: https://<ip>:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: <ip>
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 8.13.2
PS C:\Program Files\Winlogbeat> .\winlogbeat.exe test config -c .\winlogbeat.yml
Config OK
PS C:\Program Files\Winlogbeat>

I changed the CA that should be used by winlogbeat (so the CA of elastic), and it works with the .exe :

PS C:\Program Files\Winlogbeat> .\winlogbeat.exe -c .\winlogbeat.yml run -e
{"log.level":"info","@timestamp":"2024-05-06T17:14:48.149+0200","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":811},"message":"Home path: [C:\\Program Files\\Winlogbeat] Config path: [C:\\Program Files\\Winlogbeat] Data path: [C:\\Program Files\\Winlogbeat\\data] Logs path: [C:\\Program Files\\Winlogbeat\\logs]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-05-06T17:14:48.238+0200","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":819},"message":"Beat ID: d7aa1cee-76f2-4cf9-852e-4ff30fc1457b","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-05-06T17:14:48.282+0200","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1365},"message":"Beat info","service.name":"winlogbeat","system_info":{"beat":{"path":{"config":"C:\\Program Files\\Winlogbeat","data":"C:\\Program Files\\Winlogbeat\\data","home":"C:\\Program Files\\Winlogbeat","logs":"C:\\Program Files\\Winlogbeat\\logs"},"type":"winlogbeat","uuid":"d7aa1cee-76f2-4cf9-852e-4ff30fc1457b"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-05-06T17:14:48.284+0200","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1374},"message":"Build info","service.name":"winlogbeat","system_info":{"build":{"commit":"26aad5d437d592cea2d8d3e0b950f885ff47fe41","libbeat":"8.13.0","time":"2024-03-22T02:41:42.000Z","version":"8.13.0"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-05-06T17:14:48.288+0200","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1377},"message":"Go runtime info","service.name":"winlogbeat","system_info":{"go":{"os":"windows","arch":"amd64","max_procs":4,"version":"go1.21.8"},"ecs.version":"1.6.0"}}

but the service won't work, have I done something wrong ?

When you mention this, to which account you mean? There are Service and Computer accounts, have you tried both?

On top of that, could be interesting to try to grab some logs about why it is failing, maybe there are some more detailed ones in the Event viewer (System log)? Maybe we can get a bit more specific error message there.