Elasticsearch indexing performance

When indexing simple data of text type with elasticsearch, the performance difference is large according to each data size.

100byte -> 2000/s(0.2MB/s)
1KB -> 2000/s(2MB/s)
10KB -> 2000/s(20MB/s)

Why?
What is the reason?
Can't i speed up case of 100byte?

How are you sending data to Elasticsearch? How large bulk requests are you using? How many concurrent indexing threads/processes?

What is the specification of your cluster in terms of size and hardware?

How are you sending data to Elasticsearch?
app -> (file read)fluentd -> es

How large bulk requests are you using?
-> bulk 20MB

How many concurrent indexing threads/processes?
-> 3 processes(thread same)

What is the specification of your cluster in terms of size and hardware?
-> 4 GB MEM, 2 CPU

Regardless of the indexing method or hardware, this problem is the same.

How much data do you have built up on the source systems? How fast is data generated there? Does throughput vary service if you increase the number of clients workers?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.