ENVIRONMENT:
Amazon EC2 Instance
ELK has been installed
XPACK has been enabled
Passwords have been set for built-in users
metricbeat has been untar'd
ISSUE:
When I start metricbeat it gives the following output indicating it can't connect...
2020-05-19T20:39:59.505Z ERROR [publisher_pipeline_output] pipeline/output.go:106 Failed to connect to backoff(elasticsearch(http://localhost:9200)): Get http://localhost:9200: dial tcp 127.0.0.1:9200: connect: connection refused
2020-05-19T20:39:59.505Z INFO [publisher_pipeline_output] pipeline/output.go:99 Attempting to reconnect to backoff(elasticsearch(http://localhost:9200)) with 8 reconnect attempt(s)
2020-05-19T20:39:59.505Z INFO [publisher] pipeline/retry.go:196 retryer: send unwait-signal to consumer
2020-05-19T20:39:59.505Z INFO [publisher] pipeline/retry.go:198 done
2020-05-19T20:39:59.506Z INFO [publisher] pipeline/retry.go:173 retryer: send wait signal to consumer
2020-05-19T20:39:59.506Z INFO [publisher] pipeline/retry.go:175 done
2020-05-19T20:40:12.086Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":1310,"time":{"ms":223}},"total":{"ticks":2370,"time":{"ms":392},"value":2370},"user":{"ticks":1060,"time":{"ms":169}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":10},"info":{"ephemeral_id":"b248a7f0-212a-4b41-90d2-5697735d4150","uptime":{"ms":180094}},"memstats":{"gc_next":30397152,"memory_alloc":23091744,"memory_total":325630568,"rss":3928064},"runtime":{"goroutines":65}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":10,"events":{"active":313,"published":51,"retry":17,"total":51}}},"metricbeat":{"system":{"cpu":{"events":3,"success":3},"load":{"events":3,"success":3},"memory":{"events":3,"success":3},"network":{"events":15,"success":15},"process":{"events":21,"success":21},"process_summary":{"events":3,"success":3},"socket_summary":{"events":3,"success":3}}},"system":{"load":{"1":0,"15":0.05,"5":0.01,"norm":{"1":0,"15":0.025,"5":0.005}}}}}}
It's saying that it failed to connect to localhost:9200 but in my metricbeat.yml file I specifically told it that elasticsearch is 172.32.56.218:9200 as well as where Kibana is as indicated below...
metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
setup.dashboards.enabled: true
setup.kibana:
host: "172.32.56.218.5601"
output.elasticsearch:
hosts: ["172.32.56.218.9200"]
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
NOTE:
I have setup built-in passwords for the built-in users only. That's all the security i've setup at this time.
Therefore, I'm confused why it's failing to connect and why it says localhost when I've already told it in the metricbeat yaml that it's not local host.