Hi there,
I am trying to test out setting up 1 windows host and sending winlogbeat data to logstash. So far I have opened up a pipeline in logstash for beats, then configured the winlogbeat.yml file to point to logstash. The pipeline sucesssfully opens a port, however data does not ever get sent to elastic. I have included my pipline, winlogbeat.yml, logstash.yml and log output for winlogbeat when started. Is there something I am missing here to get winlogbeat to work with logstash?
Configurations:
Pipeline:
input {
beats {
port => 5959
}
}
output {
elasticsearch {
hosts => ["https://x.251.10.68:9200", "https://x.251.10.84:9200", "https://x.251.10.59:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
ssl => true
user => 'elastic'
password => 'MYPASSWORD'
document_type => "%{[@metadata][type]}"
}
}
winlogbeat.yml
winlogbeat.event_logs:
- name: Application
ignore_older: 72h - name: Security
- name: System
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 3
output.logstash:
hosts: ["x.251.10.59:5959", "x.251.10.68:5959"]
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
logging.level: info
logstash.yml
node.name: elastichost
path.data: /var/lib/logstash
log.level: info
path.logs: /var/log/logstash
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password: MYPASSWORD
xpack.monitoring.elasticsearch.url: ["https://x.251.10.68:9200", "https://x.251.10.84:9200", "https://x.251.10.59:9200"]
xpack.monitoring.elasticsearch.ssl.ca: /etc/logstash/certs/ca/ca.crt
xpack.management.enabled: true
xpack.management.pipeline.id: ["syslog", "pa", "winlogbeats"]
xpack.management.elasticsearch.username: elastic
xpack.management.elasticsearch.password: MYPASSWORD
xpack.management.elasticsearch.url: ["https://x.251.10.68:9200", "https://x.251.10.84:9200", "https://x.251.10.59:9200"]
xpack.management.elasticsearch.ssl.ca: /etc/logstash/certs/ca/ca.crt
winlogbeat log output:
S C:\winlogbeats> .\winlogbeat.exe -c .\winlogbeat.yml -e
2019-01-14T10:05:54.335-0800 INFO instance/beat.go:592 Home path: [C:\winlogbeats] Config path: [C:\winlogbeats] Data path: [C:\winlogbeats\data] Logs path: [C:\winlogbeats\logs]
2019-01-14T10:05:54.340-0800 INFO instance/beat.go:599 Beat UUID: 85d37c1a-9405-435f-bfce-b05c2c685047
2019-01-14T10:05:54.340-0800 INFO [beat] instance/beat.go:825 Beat info {"system_info": {"beat": {"path": {"config": "C:\winlogbeats", "data": "C:\winlogbeats\data", "home": "C:\winlogbeats", "logs": "C:\winlogbeats\logs"}, "type": "winlogbeat", "uuid": "85d37c1a-9405-435f-bfce-b05c2c685047"}}}
2019-01-14T10:05:54.340-0800 INFO [beat] instance/beat.go:834 Build info {"system_info": {"build": {"commit": "bd8922f1c7e93d12b07e0b3f7d349e17107f7826", "libbeat": "6.5.4", "time": "2018-12-17T20:37:05.000Z", "version": "6.5.4"}}}
2019-01-14T10:05:54.340-0800 INFO [beat] instance/beat.go:837 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":1,"version":"go1.10.6"}}}
2019-01-14T10:05:54.358-0800 INFO [beat] instance/beat.go:841 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-01-14T03:20:28.46-08:00","name":"DESKTOP-40A6JBC","ip":["fe80::64a9:ba51:3cf6:fd6d/64","x.251.12.174/24","::1/x","127.0.0.1/8"],"kernel_version":"10.0.17134.407 (WinBuild.160101.0800)","mac":["08:00:27:f8:68:3a"],"os":{"family":"windows","platform":"windows","name":"Windows 10 Pro","version":"10.0","major":10,"minor":0,"patch":0,"build":"17134.407"},"timezone":"PST","timezone_offset_sec":-28800,"id":"2b5de777-4c3a-4efb-86e7-1c2ee4db388d"}}}
2019-01-14T10:05:54.362-0800 INFO [beat] instance/beat.go:870 Process info {"system_info": {"process": {"cwd": "C:\winlogbeats", "exe": "C:\winlogbeats\winlogbeat.exe", "name": "winlogbeat.exe", "pid": 7708, "ppid": 1640, "start_time": "2019-01-14T10:05:54.126-0800"}}}
2019-01-14T10:05:54.362-0800 INFO instance/beat.go:278 Setup Beat: winlogbeat; Version: 6.5.4
2019-01-14T10:05:58.952-0800 INFO add_cloud_metadata/add_cloud_metadata.go:319 add_cloud_metadata: hosting provider type not detected.
2019-01-14T10:05:59.109-0800 INFO [publisher] pipeline/module.go:110 Beat name: DESKTOP-40A6JBC
2019-01-14T10:05:59.159-0800 INFO beater/winlogbeat.go:68 State will be read from and persisted to C:\winlogbeats\data.winlogbeat.yml
2019-01-14T10:05:59.213-0800 INFO instance/beat.go:400 winlogbeat start running.
2019-01-14T10:05:59.370-0800 INFO [monitoring] log/log.go:117 Starting metrics logging every 30s
2019-01-14T10:06:03.492-0800 INFO pipeline/output.go:95 Connecting to failover(backoff(async(tcp://x.251.10.59:5959)),backoff(async(tcp://x.251.10.68:5959)))
2019-01-14T10:06:03.493-0800 WARN transport/tcp.go:53 DNS lookup failure "x.251.10.59": lookup x.251.10.59: no such host
2019-01-14T10:06:05.564-0800 ERROR pipeline/output.go:100 Failed to connect to failover(backoff(async(tcp://x.251.10.59:5959)),backoff(async(tcp://x.251.10.68:5959))): lookup x.251.10.59: no such host
2019-01-14T10:06:05.564-0800 INFO pipeline/output.go:93 Attempting to reconnect to failover(backoff(async(tcp://x.251.10.59:5959)),backoff(async(tcp://x.251.10.68:5959))) with 1 reconnect attempt(s)
2019-01-14T10:06:05.566-0800 WARN transport/tcp.go:53 DNS lookup failure "x.251.10.68": lookup x.251.10.68: no such host
2019-01-14T10:06:07.072-0800 ERROR pipeline/output.go:100 Failed to connect to failover(backoff(async(tcp://x.251.10.59:5959)),backoff(async(tcp://x.251.10.68:5959))): lookup x.251.10.68: no such host
2019-01-14T10:06:07.072-0800 INFO pipeline/output.go:93 Attempting to reconnect to failover(backoff(async(tcp://x.251.10.59:5959)),backoff(async(tcp://x.251.10.68:5959))) with 2 reconnect attempt(s)
2019-01-14T10:06:07.074-0800 WARN transport/tcp.go:53 DNS lookup failure "x.251.10.59": lookup x.251.10.59: no such host
2019-01-14T10:06:10.097-0800 ERROR pipeline/output.go:100 Failed to connect to failover(backoff(async(tcp://x.251.10.59:5959)),backoff(async(tcp://x.251.10.68:5959))): lookup x.251.10.59: no such host
2019-01-14T10:06:10.097-0800 INFO pipeline/output.go:93 Attempting to reconnect to failover(backoff(async(tcp://x.251.10.59:5959)),backoff(async(tcp://x.251.10.68:5959))) with 3 reconnect attempt(s)
2019-01-14T10:06:10.098-0800 WARN transport/tcp.go:53 DNS lookup failure "x.251.10.68": lookup x.251.10.68: no such host
processing.