Trying to setup winlogbeat to ship windows events to working ELK stack

iqbal_nazir · June 13, 2016, 2:38pm

Hello,
I have a working ELK server on Ubuntu 16.04. I have another ubuntu 14.04 which is sending logs to my ELK successfully. Now, I want to add a windows client to my ELK which will send its events to ELK. To do so, I'm trying to install winlogbeat on my windows 10. I'm following the official documentation. From there, when I try to run Invoke-WebRequest -Method Put -InFile winlogbeat.template.json -Uri http://My_ELK_server_IP:9200/_template/winlogbeat?pretty command in the powershell it shows me the following error:

`Invoke-WebRequest : Unable to connect to the remote server
At line:1 char:1

Invoke-WebRequest -Method Put -InFile winlogbeat.template.json -Uri 1 ...

  + CategoryInfo          : NotSpecified: (:) [Invoke-WebRequest], WebException
  + FullyQualifiedErrorId : System.Net.WebException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand`

I would really appreciate if someone comes forward to help me.

--best regards,
Iqbal

andrewkroh · June 13, 2016, 2:52pm

Based on the error, I would say you have a connectivity problem between the Windows host and Elasticsearch. Can you ping the ES host from Windows? Is there a firewall somewhere in between?

iqbal_nazir · June 13, 2016, 3:24pm

Hi,

Yes, I can ping my ES host from windows. And they are in the same network, so no firewall in between.

andrewkroh · June 13, 2016, 10:16pm

Are you able to access http://My_ELK_server_IP:9200/ in the browser from that Windows host?

iqbal_nazir · June 14, 2016, 6:56am

No, but I can browse http://My_ELK_server_IP/ which redirects to Kibana. I can't telnet My_ELK_server_IP with 9200 from Windows host.

andrewkroh · June 14, 2016, 11:12am

Can you run netstat -an | grep 9200 on the host running Elasticsearch and provide the output. Maybe ES is not listening on the interface which you are trying to connect to.

iqbal_nazir · June 14, 2016, 11:17am

root@ELK:~# netstat -an | grep 9200
tcp        0      0 127.0.0.1:51494         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51458         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51560         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51432         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51586         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51446         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51558         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51564         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51452         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51436         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51514         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51512         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51418         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51546         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51392         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51450         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51466         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51496         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51400         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51524         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51580         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51490         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51456         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51442         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51448         127.0.0.1:9200          TIME_WAIT

iqbal_nazir · June 14, 2016, 11:18am

    tcp        0      0 127.0.0.1:51556         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51588         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51388         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51542         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51462         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51414         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51474         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51390         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51422         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51424         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51464         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51508         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51434         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51590         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51502         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51396         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51540         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51426         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51444         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51568         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51398         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51520         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51402         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51522         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51538         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51576         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51554         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51498         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51412         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51440         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51416         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51548         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51510         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51488         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51584         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51550         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51582         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51504         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51470         127.0.0.1:9200          TIME_WAIT
    tcp        0      0 127.0.0.1:51536         127.0.0.1:9200          TIME_WAIT

iqbal_nazir · June 14, 2016, 11:18am

tcp        0      0 127.0.0.1:51578         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51478         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51410         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51516         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51528         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51534         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51482         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51484         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51394         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51486         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51526         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51544         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51500         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51518         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51460         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51428         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51530         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51562         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51408         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51430         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51454         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51552         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51570         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51572         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51492         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51468         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51480         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51406         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51574         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51476         127.0.0.1:9200          TIME_WAIT
tcp        0      0 127.0.0.1:51420         127.0.0.1:9200          TIME_WAIT
tcp6       0      0 ::1:9200                :::*                    LISTEN
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN
tcp6       0      0 127.0.0.1:9200          127.0.0.1:51566         ESTABLISHED
tcp6       0      0 127.0.0.1:38368         127.0.0.1:9200          ESTABLISHED
tcp6       0      0 127.0.0.1:51472         127.0.0.1:9200          TIME_WAIT
tcp6       0      0 127.0.0.1:37920         127.0.0.1:9200          ESTABLISHED
tcp6       0      0 127.0.0.1:51438         127.0.0.1:9200          TIME_WAIT
tcp6       0      0 127.0.0.1:51506         127.0.0.1:9200          TIME_WAIT
tcp6       0      0 127.0.0.1:51566         127.0.0.1:9200          ESTABLISHED
tcp6       0      0 127.0.0.1:51404         127.0.0.1:9200          TIME_WAIT
tcp6       0      0 127.0.0.1:9200          127.0.0.1:38368         ESTABLISHED
tcp6       0      0 127.0.0.1:9200          127.0.0.1:37920         ESTABLISHED
tcp6       0      0 127.0.0.1:51370         127.0.0.1:9200          TIME_WAIT
tcp6       0      0 127.0.0.1:51532         127.0.0.1:9200          TIME_WAIT

andrewkroh · June 14, 2016, 11:21am

It looks like Elasticsearch is only listening on the loopback interface (which is the default for security purposes). You need to configure it to accept outside connections. See https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#common-network-settings

iqbal_nazir · June 14, 2016, 12:21pm

Hi,

Thanks a lot. I have tried changing network.host value from localhost to my_ELK_server_IP with http.port 9200. It makes my ELK "RED". I am a newbie. Could you please tell what I should change exactly?

andrewkroh · June 14, 2016, 12:27pm

The simplest (and most insecure) change would be to set network.host: 0.0.0.0.

iqbal_nazir · June 14, 2016, 3:01pm

ok.. thanks again.. now I can see a log file is created in (C:)\ProgramData\winlogbeat\Logs\winlogbeat and it shows error as the following:
2016-06-14T16:57:45+02:00 INFO Error publishing events (retrying): EOF 2016-06-14T16:57:45+02:00 INFO send fail 2016-06-14T16:57:45+02:00 INFO backoff retry: 1m0s 2016-06-14T16:58:45+02:00 INFO Error publishing events (retrying): EOF 2016-06-14T16:58:45+02:00 INFO send fail 2016-06-14T16:58:45+02:00 INFO backoff retry: 1m0s 2016-06-14T16:59:45+02:00 INFO Error publishing events (retrying): EOF 2016-06-14T16:59:45+02:00 INFO send fail 2016-06-14T16:59:45+02:00 INFO backoff retry: 1m0s

andrewkroh · June 14, 2016, 3:21pm

Can you set the logging level to debug in your configuration file. This may provide bit more detail around what the error is.

Are you now able to access http://My_ELK_server_IP:9200/ from the browser on the Windows host?

iqbal_nazir · June 14, 2016, 3:31pm

Yes, now I can access my_elk_server_ip:9200 which shows:
{ "name" : "ELK_1st_node", "cluster_name" : "ELK", "version" : { "number" : "2.3.3", "build_hash" : "218bdf10790eef486ff2c41a3df5cfa32dadcfde", "build_timestamp" : "2016-05-17T15:40:04Z", "build_snapshot" : false, "lucene_version" : "5.5.0" }, "tagline" : "You Know, for Search" }

For your 1st question: sorry I don't know how to set the logging level to 'debug' in config file.

andrewkroh · June 14, 2016, 3:43pm

You must modify the configuration file and restart the service. See docs for level: https://www.elastic.co/guide/en/beats/winlogbeat/current/configuration-logging.html#_level

iqbal_nazir · June 15, 2016, 11:28am

Hi again,

I have set the logging level to debug as you said. Still getting the same error:

2016-06-15T13:26:34+02:00 INFO Error publishing events (retrying): EOF
2016-06-15T13:26:34+02:00 INFO send fail
2016-06-15T13:26:34+02:00 INFO backoff retry: 1m0s

Do you want to review my config file?
regards.

andrewkroh · June 15, 2016, 9:17pm

Yeah, can you please share you config.

Also please check you Elasticsearch logs to see if there is anything interesting going on there.

iqbal_nazir · June 16, 2016, 8:59am

Hi,

I have been able to ship Windows logs to my ELK stack. What I did was that I kept elasticsearch as the output and commented out logstash. Till yesterday, I was trying to keep logstash as output commenting out the elasticsearch part , since I have logstash installed on my ELK server and the document was telling me to do so.

Just to know, is Winlogbeat the only beat which ships Windows events to ELK stack or I can use the others (filebeat, topbeat etc.) too?

Kevin_Csuka · June 16, 2016, 10:13am

Yeah you can use them too.
See below: Support Matrix | Elastic

Topic		Replies	Views
Winlogbeat unable to connect ELK Ubuntu Elasticsearch elastic-stack-security	4	511	May 20, 2021
Winlogbeat can't connect to the ELK Stack server Beats winlogbeat	8	2351	April 9, 2020
New winlogbeat seutp Beats winlogbeat	11	3641	August 4, 2017
Winlogbeat not working Beats winlogbeat	5	659	September 23, 2019
Winlogbeat not sending to remote computers elasticsearch Beats winlogbeat	1	702	June 6, 2019

Trying to setup winlogbeat to ship windows events to working ELK stack

Related topics