Encountered a retryable error. Will Retry with exponential backoff {:code=>500,

Elastic Stack Logstash
1

Continuing the discussion from Encountered a retryable error. Will Retry with exponential backoff code=>413:

[2022-03-07T02:18:15,991][ERROR][logstash.outputs.Elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>500, :url=>"http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/_monitoring/bulk?system_id=logstash&system_api_version=7&interval=1s"}

May I know code 500 belongs to which error.... And please suggest the solution for this error.

Advance thanks..

2

If Elasticsearch returned a 500 error then it will also have logged an error in its logs. You need to show what that error is.

3

[2022-03-08T00:45:03,841][ERROR][logstash.outputs.Elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>500, :url=>"http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/_monitoring/bulk?system_id=logstash&system_api_version=7&interval=1s"}
[2022-03-08T01:06:46,107][ERROR][logstash.outputs.Elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>503, :url=>"http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/_bulk"}
[2022-03-08T01:06:46,277][WARN ][logstash.filters.Elasticsearch] Failed to query Elasticsearch for previous event {:index=>"dmb_prod-", :error=>"[503] "}
[2022-03-08T01:06:46,308][WARN ][logstash.filters.Elasticsearch] Failed to query Elasticsearch for previous event {:index=>"dmb_prod-", :error=>"[503] "}
[2022-03-08T01:06:46,317][WARN ][logstash.filters.Elasticsearch] Failed to query Elasticsearch for previous event {:index=>"dmb_prod-", :error=>"[503] "}
[2022-03-08T01:06:46,333][WARN ][logstash.filters.Elasticsearch] Failed to query Elasticsearch for previous event {:index=>"dmb_prod-", :error=>"[503] "}
[2022-03-08T01:06:46,347][WARN ][logstash.filters.Elasticsearch] Failed to query Elasticsearch for previous event {:index=>"dmb_prod-", :error=>"[503] "}
[2022-03-08T01:06:46,353][WARN ][logstash.filters.Elasticsearch] Failed to query Elasticsearch for previous event {:index=>"dmb_prod-", :error=>"[503] "}
[2022-03-08T01:06:55,423][WARN ][logstash.outputs.Elasticsearch] Marking url as dead. Last error: [LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/][Manticore::ClientProtocolException] internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200 failed to respond {:url=>http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/, :error_message=>"Elasticsearch Unreachable: [http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/][Manticore::ClientProtocolException] internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200 failed to respond", :error_class=>"LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError"}
[2022-03-08T01:06:55,547][ERROR][logstash.outputs.Elasticsearch] Attempted to send a bulk request to Elasticsearch' but Elasticsearch appears to be unreachable or down! {:error_message=>"Elasticsearch Unreachable: [http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/][Manticore::ClientProtocolException] internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200 failed to respond", :class=>"LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError", :will_retry_in_seconds=>2}
[2022-03-08T01:06:56,761][WARN ][logstash.outputs.Elasticsearch] Restored connection to ES instance {:url=>"http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/"}
[2022-03-08T01:34:16,643][ERROR][logstash.outputs.Elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>500, :url=>"http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/_monitoring/bulk?system_id=logstash&system_api_version=7&interval=1s"}
[2022-03-08T05:27:30,479][ERROR][logstash.outputs.Elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>500, :url=>"http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/_monitoring/bulk?system_id=logstash&system_api_version=7&interval=1s"}
[2022-03-08T15:08:51,451][ERROR][logstash.outputs.Elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>500, :url=>"http://internal-elastic-search-rbi-elb-1978030293.us-east-1.elb.amazonaws.com:9200/_monitoring/bulk?system_id=logstash&system_api_version=7&interval=1s"}

Yeah this is the complete error Badger

4

That is the logstash log, you need to show us the Elasticsearch log.

5

Sorry I don't have access to that instance ,if I got I will paste here.

thanks.......

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.