Clear Cache After Bulk Insertion

Elastic Stack Elasticsearch
1

I am creating index which have 1 million data. I have many indexes with similar size. After creatng some indexes, this bulk insertion is too slow. If we are restarting the elasticsearch service in server, insert will be fast.

Is there any cache for this bulk insertion? so that I can clear this cache. I tried the flush and clearcache functions. But no luck.

I am using elasticsearch 5.6.3 versiona nd php sdk 5.

Anyone have the same issue? What is the best solution of this issue?

2

Anyone have the same issue?

When I'm doing that, I have a constant injection rate.

What is the best solution of this issue?

Check your code. Check the logs and see if you have some GC problems...

3

But my doubt is that, when I am restarting the elasticsearch service insertion is becoming fast. After some insertion its getting delayed. Thats why I am doubtful about the cache.

I am inserting 11K records per request.

4

There is no bulk cache.

5

How much heap do you have? What is the size of your documents? Have you tried with smaller bulk requests?

6

I am inserting 11k documents at a time. This index itself having 150K data (around 266MB).

If I am inserting after the elasticsearch servoce restart, index will create with in 1.3 minutes. But after creating some other indexes in I am deleting anad recreating the same index it will take more than 6 minutes.

This is the result of " /_nodes/stats "

{"heap_used_in_bytes":1659868400,"heap_used_percent":79,"heap_committed_in_bytes":2075918336,"heap_max_in_bytes":2075918336,"non_heap_used_in_bytes":135452384,"non_heap_committed_in_bytes":143286272,"
7

Do we need to set separate heap size for ES ? can you share the syntax for that ?

8

You can set the heap size in the jvm.options configuration file.

9

I increased the heap size and now it seems fine.

Thank you for the support.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.