Bossman wants to deploy a cluster of logstash, elasticsearch and kibana on one machine. He says high availability doesn't matter much since we have a different platform to monitor servers and this would be used to store logs.
He wants to know specs it should have and I just wanted to make sure it aligns with the purpose it will serve.
The cluster will be used to collect and display logs from about 100 servers. These logs are not extremely valuable and are there to view just in case. It's being set up due to compliance.
Well, it all depends on how many logs you really get and what you need to look at, but we have a small dev/test cluster right now that I downsized to a single node with 8GB ES Heap on 16GB VM taking logs from a few hundred machines, though LS & Kibana run on their own VMs.
I'd think you could combine them on a 16GB VM without much trouble, such as 1-2GB for LS and Kibana is okay at low loads.
BUT, if you are doing daily logs by type like we are, your index and shard counts will grow quickly and cause RAM, etc. issues so use ILM or something to close or purge them.
Also, one node of course can lose data - suggest maybe using snapshots to at least save data somewhere but if you not really care then never mind and if it dies, just rebuild it and pick up for new logs (I've seen that done).
These are great suggestions. I think that's the way I'll go. So set ILM to delete old logs, take snapshots in case of disaster. By the way, may I ask about the heap thing? It's default is 1 right? Should it be left at that? I'm not really sure what it does.
No, you must set the heap larger - in this like like 7-8GB - the default will likely be too small, though you never know, but if you have a 16GB VM around, use that.
For example, I tried downsizing to 4GB heap but it failed due to 1800 indexes and 3000 shards, which was too many - I had to remove 50% of those for it to work, so of course, it all depends.
I just don't really understand its role. Does the heap size say how much ram it would get to use? Also, may I ask how you have so many indexes? I think I might be using it wrong then. I output all my logs in logstash to one index.
In Java-based systems, the code runs in a Java Virtual Machine which is like a VM inside your VM - and the Heap is the memory in that JVM, all the code, the data, etc. (with some exceptions) must fit in that - it's the main thing you must set in Java systems - for Elasticsearch, generally you set to 50% of your servers RAM size, so you have 16GB, set the heap (both min/max) to 8GB.
Then once the cluster is up, you can enable self-monitoring in Kibana and see the heap use, to see if mostly/hardly used, etc. over time.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.