What's the best practices when doing both of searching and indexing on same index

[btungut (Burak Tungut)]

Hi all,

I have some elasticsearch and stack experiences so I got knowledge about most of technical concepts related to elastic ecosystem.
Now I should deliver a project which needs advanced searching features, sure elastic is the best option for this.

Here is the some details.
We will have daily index with exactly same type and expected monthly document count is 200m and that will be increased day by day.
I know that so far it seems like what we're doing for application or access logs, e.g. beat -> logstash -> elastic, with daily index.

But my concern is that results of query will be shown to end-user and that means we have to doing both of indexing and searching at same time, but we don't want indexing operations create impact or bottleneck for searching.

What should we do on about this?

Regards.

[Christian_Dahlqvist]

Mke sure you do not end up with a lot of small shards and then benchmark the cluster to find the required size in order to handle your query requirements and expected indexing load.