I am a brand new user to elastic community, I'm planning to setup ELK environment in our environment. Before to this I would like to know the different components which we have in ELK observability setup along with the benefits of each component followed by number of servers required, hardware details of each server to build ELK observability setup.
--> Hope ELK setup is of complete Opensource right?
I would review the specs We use in our cloud and you can find that in the docs here.
But there are many factors in cluster design... If you're just learning, I wouldn't overfocus on hardware design.... Unless you're going to go straight from being a newbie to ingesting hundreds of gigabytes a day.
Until you know what you're actually trying to accomplish, Search use case, logs use case, security use case recommending any hardware Architecture is premature.
What are the minimum hardware specs needed to setup complete ELK stack to monitor end to end application stack including logs, database, applications, infrastructure, security, network, services, RUM & synthetic metrics.
All I need If we want to setup ELK stack to monitor the above modules how many servers we need along with the hardware spec details.
No one can answer that... You basically just said ... design my whole system... for everything but without any details
What people normally do is set up a small cluster (1 or 3 small to medium data nodes) and then POC a set of the data and benchmark and work from there.
If you want to learn would focus on setting up a small 3 node cluster... and work from there.
Small Data Nodes... lets say 2cpu 8GB RAM 128GB SSD Disk
Medium Data Nodes lets say 4cpu 32GB RAM 500GB SSD Disk
In the end your Cluster Architecture will Depend on
Overall Volume / Rate of Data you ingest
The complexity of the Data
How much Real Time Analytics and Queries
And how long you retain the data