Issue with HTTP Poller when using headers - invalid url

Elastic Stack Logstash
1

Having trouble getting http_poller to work querying an API, as soon as I try adding headers it starts failing with Invalid URL without any other details as to why its failing. There is little documentation on using headers and even in debug mode I have trouble trying to work out why its failing.

input {
   http_poller {
         schedule => { every => "10s"}
         urls => { 
           myurl => "https://myurl.com/site/report/report.jsp?reportoption=csv&datefrom=TODAY&dateto=TODAY"
           headers => {
            Auth => "XXXXXXX"
            }
        }
   }
}

I have tried many combinations of the headers section with no luck.

[ERROR] 2019-09-30 09:17:33.721 [[main]-pipeline-manager] javapipeline - Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Invalid URL >, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:103:in validate_request!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:95:innormalize_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:55:in block in setup_requests!'", "org/jruby/RubyHash.java:1419:ineach'", "org/jruby/RubyEnumerable.java:833:in map'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:55:insetup_requests!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-5.0.1/lib/logstash/inputs/http_poller.rb:45:in register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:192:inblock in register_plugins'", "org/jruby/RubyArray.java:1792:in each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:inregister_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:292:in start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:248:instart_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:146:in run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:105:inblock in start'"], :thread=>"#<Thread:0xbef4677 run>"}
[ERROR] 2019-09-30 09:17:33.735 [Converge PipelineAction::Create] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil}

2

I believe that should be

urls => {
    myurl => {
        url => "https://myurl.com/site/report/report.jsp?reportoption=csv&datefrom=TODAY&dateto=TODAY"
        headers => { Auth => "XXXXXXX" }
    }
}
3

Thanks Badger that works, I must say the doco around using headers is ordinary at best but thanks for helping with the syntax.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.