My Logstash nodes started logging "retrying failed action with response code: 429" error lines.
I attached part of my logs below.
Q1. There are lines as below. "Retrying indivisual bulk ctions that failed or were rejected by the previous bulk request {:count=>32}"
what does "{:count=>32}" means?
Q2. When error 429 occurs, the bulk request will be anyway completed by retry? If so, how many retry will be performed?
Please help me to understand about 429 error. Thank you!!
[2018-09-19T00:00:27,264][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@3b8a7c1c on EsThreadPoolExecutor[bulk, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@28034f53[Running, pool size = 16, active threads = 16, queued tasks = 200, completed tasks = 1000287291]]"})
[2018-09-19T00:00:27,264][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@56745c15 on EsThreadPoolExecutor[bulk, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@28034f53[Running, pool size = 16, active threads = 16, queued tasks = 200, completed tasks = 1000287291]]"})
[2018-09-19T00:00:27,264][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@67a6a62a on EsThreadPoolExecutor[bulk, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@28034f53[Running, pool size = 16, active threads = 16, queued tasks = 200, completed tasks = 1000287291]]"})
[2018-09-19T00:00:27,264][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@36e39ad on EsThreadPoolExecutor[bulk, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@28034f53[Running, pool size = 16, active threads = 16, queued tasks = 200, completed tasks = 1000287291]]"})
[2018-09-19T00:00:27,264][INFO ][logstash.outputs.elasticsearch] Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=>35}