We are doing few performance run to measure the product performance. We were trying ~100 single ingests/sec and ~300 search/sec. the data we are ingesting is of size ~1KB (i mean a record size which we insert). Every thing was good and response times were <200ms.
When we add just 1 single ingest/sec with a payload of size 10MB, all other transaction response times jumped to >1sec.
Basically we are ingesting content of a file.
Any idea how we can reduce this impact? is that really a right design to ingest 10MB file content to elastic search?