# Unable to bulk upload the stats payload to the local cluster Kibana 7.3

**URL:** <https://discuss.elastic.co/t/unable-to-bulk-upload-the-stats-payload-to-the-local-cluster-kibana-7-3/196724>\
**Category:** Kibana\
**Created:** [August 26, 2019, 7:22am UTC](https://discuss.elastic.co/t/unable-to-bulk-upload-the-stats-payload-to-the-local-cluster-kibana-7-3/196724 "2019-08-26T07:22:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mohamad-Sobhie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohamad-sobhie/32/53623_2.png) [@Mohamad-Sobhie](https://discuss.elastic.co/u/Mohamad-Sobhie)\
**Post date:** [August 26, 2019, 7:22am UTC](https://discuss.elastic.co/t/unable-to-bulk-upload-the-stats-payload-to-the-local-cluster-kibana-7-3/196724/1 "2019-08-26T07:22:26Z")

</div>

Hey there,

Kibana 7.3

I recieve this message when attempting to bulk :

"tags":["warning","monitoring","kibana-monitoring"],"pid":1,"message":"Unable to bulk upload the stats payload to the local cluster"}

Note that I am testing my elastic stack by generating lots of documents every hour.

my JVM Heap

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/9/49eda353fb9aacad4c093bbcedd695ee85facbbe.png)

my indexing rate and searching rate

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/b/cb639c086d06dbf257a630b7399ffed0054bad16.png)

* * *

some logs of my elasticsearch node

"Caused by: org.elasticsearch.common.ValidationException: Validation Failed: 1: this action would add [1] total shards, but this cluster currently has [1000]/[1000] maximum shards open;",  
"at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService.checkShardLimit(MetaDataCreateIndexService.java:657) ~[elasticsearch-7.3.0.jar:7.3.0]",  
"at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$IndexCreationTask.checkShardLimit(MetaDataCreateIndexService.java:613) ~[elasticsearch-7.3.0.jar:7.3.0]",  
"at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$IndexCreationTask.execute(MetaDataCreateIndexService.java:446) ~[elasticsearch-7.3.0.jar:7.3.0]",  
"at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:47) ~[elasticsearch-7.3.0.jar:7.3.0]",  
"at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:687) ~[elasticsearch-7.3.0.jar:7.3.0]",  
"at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:310) ~[elasticsearch-7.3.0.jar:7.3.0]",  
"at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:210) ~[elasticsearch-7.3.0.jar:7.3.0]",  
"... 9 more"] }

* * *

---

<div class="post-metadata">

**Author:** ![Mike\_Place](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_place/32/39555_2.png) [@Mike\_Place](https://discuss.elastic.co/u/Mike_Place)\
**Post date:** [August 26, 2019, 11:17am UTC](https://discuss.elastic.co/t/unable-to-bulk-upload-the-stats-payload-to-the-local-cluster-kibana-7-3/196724/2 "2019-08-26T11:17:08Z")

</div>

Hi @Mohamad-Sobhie

Could you please describe how you are "generating lots of documents"? Just by guessing from the error you're seeing, I am wondering if you are creating many indices and, in turn, many primary shards.

If you go to the "Overview" page in Stack Monitoring for your Elasticsearch cluster, you should see toward the top a bar with metrics, which shows "Total Shards", "Unassigned Shards", and "Total Documents". What are those values?

Best,

-Mike

---

<div class="post-metadata">

**Author:** ![Mohamad-Sobhie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohamad-sobhie/32/53623_2.png) [@Mohamad-Sobhie](https://discuss.elastic.co/u/Mohamad-Sobhie)\
**Post date:** [August 26, 2019, 1:46pm UTC](https://discuss.elastic.co/t/unable-to-bulk-upload-the-stats-payload-to-the-local-cluster-kibana-7-3/196724/3 "2019-08-26T13:46:53Z")

</div>

yes I am creating indices (not sure how many indices is many). Here is my overview

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/b/1bb7a8b094dae615f6e5ac93d1f42a86b43204eb.png)

---

<div class="post-metadata">

**Author:** ![Mike\_Place](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_place/32/39555_2.png) [@Mike\_Place](https://discuss.elastic.co/u/Mike_Place)\
**Post date:** [August 26, 2019, 1:54pm UTC](https://discuss.elastic.co/t/unable-to-bulk-upload-the-stats-payload-to-the-local-cluster-kibana-7-3/196724/4 "2019-08-26T13:54:18Z")

</div>

You should aim to keep the number of shards per node as low as possible for the expected load.

There is a good introduction to designing for scale which can be found here: [https://www.elastic.co/guide/en/elasticsearch/guide/2.x/scale.html](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/scale.html)

I might also recommend this webinar on quantitative sizing which should help you to better understand how to test and size an Elasticsearch cluster: [https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 23, 2019, 2:09pm UTC](https://discuss.elastic.co/t/unable-to-bulk-upload-the-stats-payload-to-the-local-cluster-kibana-7-3/196724/5 "2019-09-23T14:09:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.