Hello,
I am using logstash pipelines with http input/ouput plugin.
e.g. pipeline1 gives http request to piepline2 on http port. lot of elasticsearch calls and business logic causing below error in logstash logs and input plugin stops receiving requests from backend entity.
How can i enhance performance
Logs
[HTTP Output Failure] Could not fetch URL {:url=>"http://10.63.56.202:31314", :method=>:post, :body=>"{"CIRCLE":"405874","IMSI":"405874054647890","@timestamp":"2019-12-23T13:23:24.254Z","AFTERIMEI":"3530090997718722","MSISDN":"917304069221","BEFOREIMEI":"","ACTION":"HSS","tags":["validimsi","validmsisdn","validimei"],"IMEI":"353009099771879"}", :headers=>{"Content-Type"=>"application/json"}, :message=>"Read timed out", :class=>"Manticore::SocketTimeout", :backtrace=>["/opt/rjildm/JIODM-Core-7.2.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:37:in block in initialize'", "/opt/rjildm/JIODM-Core-7.2.0/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:79:in call'", "/opt/rjildm/JIODM-Core-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-http-5.2.4/lib/logstash/outputs/http.rb:239:in send_event'", "/opt/rjildm/JIODM-Core-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-http-5.2.4/lib/logstash/outputs/http.rb:175:in send_events'", "/opt/rjildm/JIODM-Core-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-output-http-5.2.4/lib/logstash/outputs/http.rb:124:in multi_receive'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:118:in multi_receive'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:101:in multi_receive'", "/opt/rjildm/JIODM-Core-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:239:in block in start_workers'"], :will_retry=>true}
[2019-12-23T19:03:38,798][INFO ][logstash.outputs.http ] Retrying http request, will sleep for 0 seconds
Thanks