I have an infrastructure of 20 servers , standalones, no lan, only public ip behind firewall.
I would like monitoring them with elk using metricbeats & winlogsbeats
My cluster is ok, i tested metricbeats from a vm on the same lan , all is ok
But for "output.elasticsearch:"
hosts: ["publicadresskibanannode1:9200"]
username: "elastic"
password: "xxxxxxxxxxxxxxx"
with publicaddress:9200 i cant contacting elasticsearch from my server
Network:
elasticnode1 have a a private IP
elasticnode2 have a private IP
elasticnode 3 have a private IP
kibananode1 have a public + a private IP , with elastic installed with all roles (data, master, ingest) : false
Seems you have networking issues - if I read this right, you have TWO separate systems, the ELK stack and some other 20 servers. And the 20 servers can't reach your ELK stack to send data.
I would:
You say Kibana has a public IP, but it seems that's via NAT and firewall, not on the VM/server; if it's on the server, make sure it's listening on the public IP.
If it's via NAT, make sure your firewall is correctly configured, and that you can telnet to publicaddress:9200 - this is not really an elasticsearch thing but your network, firewall, NAT, etc. You must get to a point where the 20 servers can do this telnet (or cURL or whatever).
Not sure I understand your question but a LOT of people put their ES servers on the public Internet and get hacked instantly and all their data stolen & deleted; so don't do that
This all depends on how your 'other' servers being monitored will reach your ES system; a VPN or similar would be ideal, but if over the Internet, of course you need to limit the IP range in Security Groups and Firewalls, probably at couple levels to be safe, plus enable security on the ES system ASAP so the clients need a user/PW, key, etc.
Is there others ports listened by kibana, elastic ? (i see 9300 sometimes but i dont know what is it)
9200 is for ES REST API for data in/out, and would need to be reachable by all clients. Protect it as much as you can.
9300 is called Transport and is for the cluster to talk internally - all nodes need to be able to reach all other nodes on this port, but it's very private and is NEVER put on the Internet or should even be reachable outside the cluster (not sure if it's used for cross-cluster replication).
Kibana is listening I believe, as a server, but you only need to expose the port 443 for your browser, and also protect it as best you can via security groups, etc. plus user/password.
There are generally no other ports, other than if you use Logstash or other tools, or maybe older JVM JMX, etc.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.