# Requirements planning
**URL:** https://discuss.elastic.co/t/requirements-planning/240856
**Category:** Elasticsearch
**Created:** [July 12, 2020, 7:12pm UTC](https://discuss.elastic.co/t/requirements-planning/240856 "2020-07-12T19:12:02Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)
#### Post date: [July 12, 2020, 7:12pm UTC](https://discuss.elastic.co/t/requirements-planning/240856/1 "2020-07-12T19:12:02Z")
I am running Ubuntu server 20.04 LTS in a VM hosted on a NAS (Synology.) Configuration is:
1. Compute: Intel Celeron J3455 (4 cores allocated to the VM.) 1.5 (base) / 2.3 (burst) GHz
2. Memory: 4 GB DDR3.
3. Storage: 7200 RPM drive.
**Following is the usage:**
**I am planning to move this from the NAS to a workstation running:**
1. VMWare ESXi (free)
2. Compute: Intel Xeon W 1290 with 10 cores (20 threads) with 3.20 GHz base and 5.20 GHz Turbo frequency.
3. Memory: 64 GB DDR4 RAM @ 2933 MHz
4. Storage: 7200 RPM drive + PCIe M.2 SSD (Dell Class 40)
I am a self-funded student, and I need to use this system as part of my final year perusing masters in software and systems security. I wanted assistance calculating systems requirements if the load on the system remains consistent. I am using Elasticsearch stack on this system. I am facing the following issues:
1. System load is always over 4.0 is this due to IOPs issue? If so, will adding RAM during migration to another system help? Or does this require faster storage such as SSD?
2. Will providing the following configuration help situation load and improve performance without over provisioning?
```auto
Compute: 3 processors x 2 cores per processor = 6
16 GB RAM
Storage will be local (SATA, 7200 RPM NAS grade drive.)
```
Underlying operating system continue to be Ubuntu 20.04 LTS. I want to comprehensively learn the Elastic Stack and hence I am hoping to put multiple nodes to test roles, query scheduling, etc. for using Elastic stack as an enterprise SIEM.
Current primary purpose of the system is to ingest logs from 50 honeypots deployed around the world in AWS and Azure with 500 being average EPS (events per second.) These events go into a logstash pipeline running on a raspberry pi before being sent the elastic stack.
What steps should I take to ensure I am not over provisioning and able to use the system for other projects too?
Thank you.
---
### Author:  [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)
#### Post date: [July 13, 2020, 7:00am UTC](https://discuss.elastic.co/t/requirements-planning/240856/2 "2020-07-13T07:00:56Z")
Any reason not to run this on Docker, as then way more flexible to control versions, allocations, etc. 4GB not a lot for a real ELK stack, though your 64GB workstation is nice.
IOWait that high on 50 IOPS means small IO and maybe fsyncs() going on; certainly SSD will fix that.
---
### Author:  [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)
#### Post date: [July 13, 2020, 8:27am UTC](https://discuss.elastic.co/t/requirements-planning/240856/3 "2020-07-13T08:27:45Z")
Let me look at the docker option. Since I don't specialise in it; I had not considered it.
Besdies:[https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html)
Is there documentation to look at fsync() that I should refer to?
I am hoping to deploy two nodes on single hardware (workstation) mentioned above with following configuration for each:
3 cores - 6 threads, 16 GB RAM and data on two different HDDs (running at 10000 RPM.)
Do you reckon any bottle necks to continue here? I will upload proposed low level diagram in sometime.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 13, 2020, 8:31am UTC](https://discuss.elastic.co/t/requirements-planning/240856/4 "2020-07-13T08:31:21Z")
Indexing is I/O intensive so can easily saturate disks. Have a look at [this video for a good comparison](https://youtu.be/nKUpfJCBiS4).
---
### Author:  [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)
#### Post date: [July 13, 2020, 12:43pm UTC](https://discuss.elastic.co/t/requirements-planning/240856/5 "2020-07-13T12:43:02Z")
Yeah a little confused on your goals as if you are starting out you have a ways to go before having Enterprise SIEM, and 500 EPS is not high, but could be for spinning disks; hard to know as they are so rare these days on primary data stores 😉 and all you can do it test & benchmark.
---
### Author:  [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)
#### Post date: [July 13, 2020, 1:12pm UTC](https://discuss.elastic.co/t/requirements-planning/240856/6 "2020-07-13T13:12:24Z")
So my goal is to carry out my final year research on security monitoring. I've selected ELK as one of the platforms to use for benchmark and comparison to traditional SIEMs such as IBMs QRadar.
As part of my research I have series of honeypot systems deployed on AWS/Azure/Google cloud. These systems will be sending telemetry data. Currently I have 25+ such systems including few laptops of my family which also log netflow data. I hope this helps clear my usecase. I apologise for the ambiguity due to my initial post.
---
### Author:  [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)
#### Post date: [July 13, 2020, 1:13pm UTC](https://discuss.elastic.co/t/requirements-planning/240856/7 "2020-07-13T13:13:04Z")
This was extremely insightful video. I've instantly shared it with all my friends 🙂 Thank you very much. Is there a way to calculate EPS in ELK?
---
### Author:  [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)
#### Post date: [July 14, 2020, 4:12am UTC](https://discuss.elastic.co/t/requirements-planning/240856/8 "2020-07-14T04:12:19Z")
Calc EPS by running cluster stats some seconds apart and get indices.docs.count and then divide it by the time. Our [ELKman.io](http://ELKman.io) tool (which you can use for free) also shows this to you on our dashboard.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 11, 2020, 4:12am UTC](https://discuss.elastic.co/t/requirements-planning/240856/9 "2020-08-11T04:12:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.