Hi Team,
I am extracting contents of thousands of URLs using http filter.
But some urls throws below error which is stopping pipeline.
Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"new-english-pdf", :error=>"(NoMethodError) undefined method `strip' for nil:NilClass", :exception=>Java::OrgJrubyExceptions::NoMethodError, :backtrace=>["usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_http_minus_1_dot_2_dot_1.lib.logstash.filters.http.process_response(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-http-1.2.1/lib/logstash/filters/http.rb:150)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_http_minus_1_dot_2_dot_1.lib.logstash.filters.http.filter(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-http-1.2.1/lib/logstash/filters/http.rb:96)", "usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.do_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:159)", "usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:178)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1821)", "usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:175)", "org.logstash.config.ir.compiler.AbstractFilterDelegatorExt.multi_filter(org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:134)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:299)"], :thread=>"#<Thread:0xdd1850b run>"}
And below is my http filter configuration:
http {
url => "%{basedataurl}"
cacert => "/etc/logstash/certs/english.pem"
target_body => "doc_body"
ssl_verification_mode => none
connect_timeout => 300
automatic_retries => 5
follow_redirects => true
}
Please let me know how can I avoid this error to stop pipeline.