Advice to process events in elasticsearch


(Ganesh) #1

Hi Team,
Can anyone clear my doubt, we have stand-alone elasticsearch service is running and we are trying to parse 1 lakhs of record in 1s, is it possible to achieve that with good performance tuning.

Parsing is happening through postman with Go-lang


(Christian Dahlqvist) #2

That depends a lot on the size and complexity of your documents as well as the specification of the node. How you index it will also play a part.


(Nagarjuna Gadikatla) #3

What's your standalone server capacity.


(Ganesh) #4

500Gb Harddisk
16GB RAM


(Christian Dahlqvist) #5

To ingest that number of documents per second, you will be sending large amounts of data to Elasticsearch in parallel. I therefore suspect you will need more RAM and a good amount of CPU to process all the documents. You will also most likely require fast local SSDs as indexing is very I/O intensive.