Advice to process events in elasticsearch

Ganesh2303 · February 13, 2019, 11:07am

Hi Team,
Can anyone clear my doubt, we have stand-alone elasticsearch service is running and we are trying to parse 1 lakhs of record in 1s, is it possible to achieve that with good performance tuning.

Parsing is happening through postman with Go-lang

Christian_Dahlqvist · February 13, 2019, 11:37am

That depends a lot on the size and complexity of your documents as well as the specification of the node. How you index it will also play a part.

Nagarjuna_Gadikatla · February 13, 2019, 11:41am

What's your standalone server capacity.

Ganesh2303 · February 13, 2019, 11:44am

500Gb Harddisk
16GB RAM

Christian_Dahlqvist · February 13, 2019, 11:53am

To ingest that number of documents per second, you will be sending large amounts of data to Elasticsearch in parallel. I therefore suspect you will need more RAM and a good amount of CPU to process all the documents. You will also most likely require fast local SSDs as indexing is very I/O intensive.

system · March 13, 2019, 11:53am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.