emstats":{"gc_next":57446288,"memory_alloc":28758904,"memory_total":636265600},"runtime":{"goroutines":28}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"errors":1},"write":{"bytes":124}},"pipeline":{"clients":3,"events":{"active":4119,"retry":50}}}}}}
2020-05-03T03:31:45.550+0530 ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://52.66.120.80:5044)): Get http://52.66.120.80:5044: read tcp 192.168.0.115:20072->52.66.120.80:5044: wsarecv: An existing connection was forcibly closed by the remote host.
2020-05-03T03:31:45.550+0530 INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://52.66.120.80:5044)) with 16 reconnect attempt(s)
2020-05-03T03:31:45.550+0530 INFO [publisher] pipeline/retry.go:196 retryer: send unwait-signal to consumer
2020-05-03T03:31:45.553+0530 DEBUG [elasticsearch] elasticsearch/client.go:733 ES Ping(url=http://52.66.120.80:5044)
2020-05-03T03:31:45.553+0530 INFO [publisher] pipeline/retry.go:198 done
2020-05-03T03:31:45.554+0530 INFO [publisher] pipeline/retry.go:173 retryer: send wait signal to consumer
2020-05-03T03:31:45.554+0530 INFO [publisher] pipeline/retry.go:175 done
2020-05-03T03:31:45.629+0530 DEBUG [elasticsearch] elasticsearch/client.go:737 Ping request failed with: Get http://52.66.120.80:5044: read tcp 192.168.0.115:20077->52.66.120.80:5044: wsarecv: An existing connection was forcibly closed by the remote host.
I installed ELK on AWS and Elasticsearch only accessible on localhost and Kibana on Public IP. I wish to send the logs from my local system to aws. Please tell how can I solve this. I dont want to send direct log to elasticsearch. I want Winlogbeat------> Logstash----> Elasticsearch
Winlogbeat requires a connection to Logstash or to Elasticsearch (depending which output you've configured in Winlogbeat).
If Elasticsearch is only available on localhost (on AWS), Winlogbeat or Logstash will never be able to reach it.
I would suggest enabling the security (TLS & Authentication, Authorization) using a Basic license and make Elasticsearch accessible via a public IP and/or a VPN.
Once you've done so, always ensure you install the index templates if you are going through Logstash. See:
I am able to send logs Winlogbeat (LOCAL SYS) ---> Elasticsearch (AWS )----> Kibana without any TLS and Authentication. I just added network.host : 0.0.0.0
& able to access Elasticsearch publicly but this method is not secured.
My Planning is to send the logs to the Local Sys to Logstash.
Syslog-ng to Logstash.
2020-05-04T00:53:15.001+0530 DEBUG [winlogbeat] beater/eventlogger.go:152 EventLog[Application] Read() returned 0 records
2020-05-04T00:53:15.619+0530 DEBUG [elasticsearch] elasticsearch/client.go:737 Ping request failed with: Get http://logs.tweetfblikes.com:5504: dial tcp 52.57.57.0:5504: connectex: No connection could be made because the target machine actively refused it.
2020-05-04T00:53:16.004+0530 DEBUG [eventlog_detail] eventlog/wineventlog.go:296 WinEventLog[Application] No more events
2020-05-04T00:53:16.004+0530 DEBUG [winlogbeat] beater/eventlogger.go:152 EventLog[Application] Read() returned 0 records
2020-05-04T00:53:17.006+0530 DEBUG [eventlog_detail] eventlog/wineventlog.go:296 WinEventLog[Application] No more events
2020-05-04T00:53:17.006+0530 DEBUG [winlogbeat] beater/eventlogger.go:152 EventLog[Application] Read() returned 0 records
2020-05-04T00:53:18.007+0530 DEBUG [eventlog_detail] eventlog/wineventlog.go:296 WinEventLog[Application] No more events
2020-05-04T00:53:18.007+0530 DEBUG [winlogbeat] beater/eventlogger.go:152 EventLog[Application] Read() returned 0 records
2020-05-04T00:53:18.261+0530 ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://logs.tweetfblikes.com:5504)): Get http://logs.tweetfblikes.com:5504: dial tcp 52.57.57.0:5504: connectex: No connection could be made because the target machine actively refused it.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.