# Write slow on elastic cluster **URL:** \ **Category:** Elasticsearch\ **Created:** [June 22, 2021, 8:47am UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622 "2021-06-22T08:47:00Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Sunil\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunil_chand/32/90629_2.png) [@Sunil\_Chand](https://discuss.elastic.co/u/Sunil_Chand)\ **Post date:** [June 22, 2021, 8:47am UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/1 "2021-06-22T08:47:00Z")
Hi All One of clusters has the following specs 8 nodes (8 data and 5 master eligible) each with 15GB heap space running on servers with 32GB of RAM. Elasticsearch version 7.6.2 on linux each node has 8 TB of space allocated around 30 billion documents are there Each node has shards of 1443 total write i/o is around 350/sec The application team is complaining the writes are slow, how can we improve writes to cluster? Thanks Sunil ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 22, 2021, 9:09am UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/2 "2021-06-22T09:09:06Z")
That looks like a lot of shards per node. Significantly more than recommended. How many indices and shards are you actively indexing into? How are you indexing into Elasticsearch? What type of storage do you have? ---
**Author:** ![Sunil\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunil_chand/32/90629_2.png) [@Sunil\_Chand](https://discuss.elastic.co/u/Sunil_Chand)\ **Post date:** [June 22, 2021, 12:55pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/3 "2021-06-22T12:55:49Z")
Hi Christian storage is XIO Flash storage ingestion happening from a python script, any given time there are 34 processes that are capable of writing. each process will only write to a single index at a time, index refresh is disabled when they start on a particular work item on completion of the work item they trigger a manual refresh the volume of 2.2k items per day will be roughly ingested, which isn't 1:1 with indexes (As several of these items share an index) Thanks Sunil ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 22, 2021, 12:59pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/4 "2021-06-22T12:59:45Z")
What bulk size are you using? ---
**Author:** ![Sunil\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunil_chand/32/90629_2.png) [@Sunil\_Chand](https://discuss.elastic.co/u/Sunil_Chand)\ **Post date:** [June 22, 2021, 2:20pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/5 "2021-06-22T14:20:10Z")
Hi Christian 500 events/chunk, max size of 50MB , 4 queues Thanks Sunil ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 22, 2021, 3:09pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/6 "2021-06-22T15:09:58Z")
What is the average size of your events? Are you using dynamic mappings? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 22, 2021, 10:55pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/7 "2021-06-22T22:55:30Z")
> [@Sunil\_Chand](#): > > Each node has shards of 1443 That's too many, you need to at least halve that. ---
**Author:** ![Sunil\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunil_chand/32/90629_2.png) [@Sunil\_Chand](https://discuss.elastic.co/u/Sunil_Chand)\ **Post date:** [June 23, 2021, 1:25pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/8 "2021-06-23T13:25:47Z")
Hi Christian average size of event is around 3kb and dynamic sampling is being used Thanks Sunil ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 23, 2021, 1:28pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/9 "2021-06-23T13:28:45Z")
> [@Sunil\_Chand](#): > > total write i/o is around 350/sec What unit is this? What ingest rate are you seeing? ---
**Author:** ![Sunil\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunil_chand/32/90629_2.png) [@Sunil\_Chand](https://discuss.elastic.co/u/Sunil_Chand)\ **Post date:** [June 23, 2021, 1:29pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/10 "2021-06-23T13:29:15Z")
Hi warkolm do we need to add additional nodes? or increase in memory? our infrastructure team has said even half of the given memory is not getting allocated. Thanks Sunil ---
**Author:** ![Sunil\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunil_chand/32/90629_2.png) [@Sunil\_Chand](https://discuss.elastic.co/u/Sunil_Chand)\ **Post date:** [June 23, 2021, 2:11pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/11 "2021-06-23T14:11:02Z")
Hi Christian i/o statistics captured from kibana console and ingestion rate is 1.2 k events per second Thanks Sunil ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 23, 2021, 4:18pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/12 "2021-06-23T16:18:36Z")
How have you determined that it is the Elasticsearch cluster that is the bottleneck and not the Python scripts you are using to ingest the data? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 21, 2021, 4:19pm UTC](https://discuss.elastic.co/t/write-slow-on-elastic-cluster/276622/13 "2021-07-21T16:19:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.