# \[logstash.outputs.opensearch\]\[main\]\[9182 Retrying individual bulk actions that failed or were rejected by the previous bulk request {:count=\>125}

**URL:** <https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408>\
**Category:** Elasticsearch\
**Created:** [June 13, 2024, 12:47pm UTC](https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408 "2024-06-13T12:47:52Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rathiga\_Thambu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rathiga_thambu/32/126785_2.png) [@Rathiga\_Thambu](https://discuss.elastic.co/u/Rathiga_Thambu)\
**Post date:** [June 13, 2024, 12:47pm UTC](https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408/1 "2024-06-13T12:47:52Z")

</div>

bin/logstash -f /opt/logstash-8.9.0/config/conf.d/pipe.conf

[2024-06-13T12:42:19,876][INFO][logstash.outputs.opensearch][main][91823b58f6fb0958c7671051e63dcf1ae4101824338810d25e044a5a52cd36e4] Retrying failed action {:status=\>429, :action=\>["index", {:\_id=\>nil, :\_index=\>"rpmappserver-01ribbideo\_logs", :routing=\>nil}, {"event"=\>{"original"=\>"10.30.1.132 - - [10/Sep/2023:01:00:02 -0700] "GET /rx/api/config HTTP/1.1" 200 2\n"}, "message"=\>"10.30.1.132 - - [10/Sep/2023:01:00:02 -0700] "GET /rx/api/config HTTP/1.1" 200 2\n", "@version"=\>"1", "type"=\>"s3", "@timestamp"=\>2024-06-13T12:40:10.852904110Z}], :error=\>{"type"=\>"cluster\_block\_exception", "reason"=\>"index [rpmappserver-01ribbideo\_logs] blocked by: [TOO\_MANY\_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];"}}

[logstash.outputs.opensearch][main][91823b58f6fb0958c7671051e63dcf1ae4101824338810d25e044a5a52cd36e4] Retrying individual bulk actions that failed or were rejected by the previous bulk request {:count=\>125}

cd /opt/logstash-8.9.0/config/conf.d  
vi pipe.conf  
input {  
s3 {  
access\_key\_id =\> "AKIAUNWJFISALFXWOPVQ"  
secret\_access\_key =\> "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"  
bucket =\> "serverlogs-01backup"  
region =\> "us-west-2"  
type =\> "s3"  
prefix =\> "appserver/rpmappserver-01/ribbideo\_logs/"  
interval =\> 60  
}

s3 {  
access\_key\_id =\> "AKIAUNWJFISALFXWOPVQ"  
secret\_access\_key =\> "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"  
bucket =\> "serverlogs-01backup"  
region =\> "us-west-2"  
type =\> "s3"  
prefix =\> "appserver/rpmappserver-01/tomcat8\_logs/"  
interval =\> 60  
}  
}

filter {

# Add any necessary filters here to parse or enrich your data

}

output {  
opensearch {  
hosts =\> ["[https://localhost:9200](https://localhost:9200)"]  
index =\> "rpmappserver-01ribbideo\_logs"  
user =\> "admin"  
password =\> "admin"  
ssl\_certificate\_verification =\> false  
}

opensearch {  
hosts =\> ["[https://localhost:9200](https://localhost:9200)"]  
index =\> "rpmappserver-01tomcat8\_logs"  
user =\> "admin"  
password =\> "admin"  
ssl\_certificate\_verification =\> false  
}  
}

```auto

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 13, 2024, 12:47pm UTC](https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408/2 "2024-06-13T12:47:53Z")

</div>

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance. See [What is OpenSearch and the OpenSearch Dashboard? | Elastic](https://www.elastic.co/elasticsearch/opensearch) for more details.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![Rathiga\_Thambu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rathiga_thambu/32/126785_2.png) [@Rathiga\_Thambu](https://discuss.elastic.co/u/Rathiga_Thambu)\
**Post date:** [June 13, 2024, 12:48pm UTC](https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408/3 "2024-06-13T12:48:48Z")

</div>

pls help me , how to fix this

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 13, 2024, 12:50pm UTC](https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408/4 "2024-06-13T12:50:50Z")

</div>

> [@Rathiga\_Thambu](#):
>
> TOO\_MANY\_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block

Opensearch is not supported here, but this is your issue, your disk does not have enough space, you need to clean up some space on your disk.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 13, 2024, 12:51pm UTC](https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408/5 "2024-06-13T12:51:12Z")

</div>

> [@Rathiga\_Thambu](#):
>
> TOO\_MANY\_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block

Looks like your server has run out of disk space.

---

<div class="post-metadata">

**Author:** ![Rathiga\_Thambu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rathiga_thambu/32/126785_2.png) [@Rathiga\_Thambu](https://discuss.elastic.co/u/Rathiga_Thambu)\
**Post date:** [June 13, 2024, 1:52pm UTC](https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408/6 "2024-06-13T13:52:31Z")

</div>

already i added new volume to my server

/dev/xvdd 69G 53M 66G 1% /opt/new\_volume

---

<div class="post-metadata">

**Author:** ![Rathiga\_Thambu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rathiga_thambu/32/126785_2.png) [@Rathiga\_Thambu](https://discuss.elastic.co/u/Rathiga_Thambu)\
**Post date:** [June 13, 2024, 1:54pm UTC](https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408/7 "2024-06-13T13:54:22Z")

</div>

-bash-4.2# df -h  
Filesystem Size Used Avail Use% Mounted on  
devtmpfs 7.9G 0 7.9G 0% /dev  
tmpfs 7.9G 0 7.9G 0% /dev/shm  
tmpfs 7.9G 528K 7.9G 1% /run  
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup  
/dev/xvda1 71G 67G 4.8G 94% /  
/dev/xvdc 12G 893M 11G 8% /var/lib/mongo  
/dev/xvdb 12G 41M 12G 1% /usr/local/ribb ideo/logs  
/dev/xvdf 12G 2.1G 10G 18% /data  
tmpfs 7.9G 249M 7.6G 4% /tmp  
[fs-56b3fdfc.efs.us-west-2.amazonaws.com](http://fs-56b3fdfc.efs.us-west-2.amazonaws.com)😕 8.0E 1.8G 8.0E 1% /mnt/staging/ef s  
/dev/xvdd 69G 53M 66G 1% /opt/new\_volume  
tmpfs 1.6G 0 1.6G 0% /run/user/1000

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 13, 2024, 2:41pm UTC](https://discuss.elastic.co/t/logstash-outputs-opensearch-main-9182-retrying-individual-bulk-actions-that-failed-or-were-rejected-by-the-previous-bulk-request-count-125/361408/8 "2024-06-13T14:41:16Z")

</div>

> [@Rathiga\_Thambu](#):
>
> already i added new volume to my server
> 
> /dev/xvdd 69G 53M 66G 1% /opt/new\_volume

I don't think this makes any difference, the issue is related to the space where your data is being saved.

As mentioned Opensearch is not supported here, you need to check where the data is being saved and increase that partition or remove any data.

I would assume that it is saving under `/var/lib`.
