Guidance for Production ELK stack box size and other memory components discuss

[elk.beginners]

Hello All,

Can anyone share the information about production configuration Box for ELK stack.

Actually my aim to design centralised ELK Server with (Elastic+Logstash+Kibana installation) with sufficient memory configuration and resources. Moreover, I will have to connect MySQL database servers. and I want send below data from MySQL servers to ELK stack.

Plan to configure like below :

1. There will be couple of MySQL server with 4 its instances each mean total 8 MySQL send their error logs and slow logs ----> Logstash ---> ES---> Kibana

2. Have 50+ JDBC input(with respective of 10 instances of MySQL with simple select queries) from Logstash ------> ES and display stats with Kibana
   eg:

input {
jdbc {
jdbc_driver_library
query-3 from mysql
}
jdbc {
jdbc_driver_library
query-1 from mysql
}
jdbc {
jdbc_driver_library
query-2 from mysql
}
.......50+
}
Need your help to decide

Question-1 : What size of sever i will be procured ?
Question-2 : How much need memory+ disk+ cores etc technically ?
Question-3 : Security suggestions

Thanks in advance !!!!

[Christian_Dahlqvist]

Without knowing how much data this will generate or how long you are going to keep the data it is hard to give any recommendations.

[elk.beginners]

let me share you one data stats of my pilot project with testing server.

[elk.beginners]

and

@ [Christian_Dahlqvist]

here above snap which I started as pilot/test to actual work load and its captured continuous data from last 3 weeks. yesterday, I stopped because need to decide other things.

Now what you say about this ?

[elk.beginners]

[Christian_Dahlqvist]

It does not tell me anything apart from that you have far too many small indices and shards.

[elk.beginners]

So you can tell me which information I will provide that can be more confidently said about my requirement ?