Hi,
I am very new to the ELK stack and Beats. I have a ELK stack setup on a Windows 2012 R2 server and am collecting winlogbeat, packetbeat and topbeat from the host of the ELK stack. I'm now trying to add winlogbeat to one of my domain controllers. The ELK host and DC are on the same subnet, I've installed the winlogbeat service and here is the .yml I'm using:
winlogbeat:
registry_file: C:/ProgramData/winlogbeat/.winlogbeat.yml
event_logs:
- name: Application
- name: Security
- name: System
output:
logstash:
hosts: ["10.20.1.27:5045"]
logging:
to_files: true
files:
path: C:/ProgramData/winlogbeat/Logs
level: info
In the winlogbeat / logs file I see the following log error:
2016-07-29T09:45:53-07:00 INFO Connecting error publishing events (retrying): dial tcp 10.20.1.27:5045: connectex: No connection could be made because the target machine actively refused it. 2016-07-29T09:45:53-07:00 INFO send fail 2016-07-29T09:45:53-07:00 INFO backoff retry: 1m0s
By target, I assume it means the ELK host. I've tried turning off the windows Firewall, no AV is installed currently and there are no Policies on the network that would prevent this traffic from hitting the host.
Any help would be appreciated.
Thanks,
Andrew
Are you able to telnet 10.20.1.27 5045 from the host running Winlogbeat to the host running Logstash? What output does that give?
It could be a firewall or AV on the Winlogbeat host too. Check those as well.
I am NOT able to telnet from the Winlogbeat host to the Logstash server on that port, I can telnet to it on the default telnet port and even on 5601 which is the web host port. I've turned off windows firewall on both source and destination, there is no AV on either server currently.
Not sure why with windows firewall off on both clients and no AV why I can not telnet to it on any port I want. Well... Can I use 5601 to send my logs? Or is that only for the web host?
Are you sure Logstash is both running and listening on 5045? Check that the process is still running. If not check the logs and post the config. Also check the output of netstat to verify that there is a tcp port listening on 0.0.0.0:5045.
No, that's the Kibana web server. It doesn't accept data from Beats.
I answered my own question as to if I can use 5601. I get the following error: An existing connection was forcibly closed by the remote host.
I will work to figure out why I can't connect on port 5045. This appears to be the root of the issue.
Logstash is running as a service, I restarted it and then checked netstat but don't see 5045 as listening.
Please post your Logstash config. Use the </> button for code formatting.
input { stdin { } }
output {
elasticsearch { hosts => ["localhost:5045"] }
stdout { codec => rubydebug }
}
Please check out the documentation on how to use Winlogbeat with Logstash.
Step 3: Configuring Winlogbeat to Use Logstash
Setting Up Logstash
You are missing a beats input and you need a few more parameters in your elasticsearch output. Then you should be good to go. 
Solved this bundles issue. Now back to the beats.
Okay, So I've run through both documents and made the necessary changes, however I still don't see a port open for logstash on 5044 using netstat.
So to update: Here is my winlogbeat.yml from the DC i am shipping the logs from.
winlogbeat:
registry_file: C:/ProgramData/winlogbeat/.winlogbeat.yml
event_logs:
- name: Application
- name: Security
- name: System
output:
logstash:
hosts: ["10.20.1.27:5044"]
logging:
to_files: true
files:
path: C:/ProgramData/winlogbeat/Logs
level: info`
Then here is the logstash.conf from the ELK server
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
Something interesting is that with the other beats files such as winlogbeat, topbeat, packetbeat, I see those in the Windows C:\ProgramData folder with Logs in each, however logstash is not listed in C:\ProgramData. Not sure if that is normal, but doesn't seem consistent.
Andrew
Logstash writes to stdout by default. So you want to get more verbose information logged to stdout you can add flags like --verbose or --debug, and if you want that data logged to a file you can use --log FILE. See Logstash Command-line flags.
Here is the log output
PS C:\elk-stack\logstash> bin\logstash.bat -f logstash.conf -v --debug --verbose
io/console not supported; tty will not be manipulated
←[32mstarting agent {:level=>:info}←[0m
←[32mstarting pipeline {:id=>"main", :level=>:info}←[0m
Settings: Default pipeline workers: 8
←[32mBeats inputs: Starting input listener {:address=>"0.0.0.0:5044", :level=>:info}←[0m
←[32mNew Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["localhost:9200"], :level=>:info}←[0
m
←[32mStarting pipeline {:id=>"main", :pipeline_workers=>8, :batch_size=>125, :batch_delay=>5, :max_inflight=>1000, :leve
l=>:info}←[0m
Pipeline main started
It hangs at pipeline main started.