Http_poller is not working for me, please help

Hi,
i'm trying to fetch data from solarwinds via the rest-api, when i use the url in my browser, everything works fine.
when i run this config file, i get this weird error, im not even sure how to read it.
this is my first try with http_poller, please help..

This is my conf file:

input {
  http_poller {
    urls => {
      solarwinds => {
        # Supports all options supported by ruby's Manticore HTTP client
        method => get
        user => "admin"
        password => "Zxasqw12"
        url =>
		"https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT NodeID FROM Orion.Nodes"
		verify_cert => true
        headers => {
          Accept => "application/json"
        }
     }
    }
    request_timeout => 60
    # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
    schedule => { cron => "* * * * * UTC"}
    codec => "json"
    # A hash of request metadata info (timing, response headers, etc.) will be sent here
    metadata_target => "http_poller_metadata"
  }
}

output {
  stdout {
  codec => rubydebug 
  }
  elasticsearch {
   hosts => "WIN-EJ0HVLFU246:9200"
 #  user => "elastic"
 #  password => "Zxasqw12"
   index => "Solarwinds__test"
  }
}

will post the trace in the next massage...

any ideas ?

thanks,
David

This is the runtime trace:

C:\Install\6.2.2\logstash-6.2.2\bin>logstash -f logstashPipeLine_SolarWinds.conf
Sending Logstash's logs to C:/Install/6.2.2/logstash-6.2.2/logs which is now configured via log4j2.properties
[2018-03-15T11:00:17,217][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"C:/Install/6.2.2/logstash-6.2.2/modules/fb_apache/configuration"}
[2018-03-15T11:00:17,248][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"C:/Install/6.2.2/logstash-6.2.2/modules/netflow/configuration"}
[2018-03-15T11:00:17,686][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-03-15T11:00:18,951][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.2.2"}
[2018-03-15T11:00:20,138][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-03-15T11:00:27,623][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-03-15T11:00:28,436][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://WIN-EJ0HVLFU246:9200/]}}
[2018-03-15T11:00:28,451][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://WIN-EJ0HVLFU246:9200/, :path=>"/"}
[2018-03-15T11:00:28,779][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://WIN-EJ0HVLFU246:9200/"}
[2018-03-15T11:00:28,889][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>nil}
[2018-03-15T11:00:28,889][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-03-15T11:00:28,920][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-03-15T11:00:28,967][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "
match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"
}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-03-15T11:00:29,044][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//WIN-EJ0HVLFU246:9200"]}
[2018-03-15T11:00:29,123][INFO ][logstash.inputs.http_poller] Registering http_poller Input {:type=>nil, :schedule=>{"cron"=>"* * * * * UTC"}, :timeout=>nil}
[2018-03-15T11:00:29,232][ERROR][logstash.pipeline        ] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::HTTP_Poller urls=>{\"solarwinds\"=>{\"method\"=>\"get\", \"user\"=>\"admin\", \"password\"=>\"Zxasqw12\", \"url\"=>\"https://solarwinds-orion:17778/SolarWinds/Info
rmationService/v3/Json/Query?query=SELECT NodeID FROM Orion.Nodes\", \"verify_cert\"=>\"true\", \"headers\"=>{\"Accept\"=>\"application/json\"}}}, request_timeout=>60, schedule=>{\"cron\"=>\"* * * * * UTC\"}, codec=><LogStash::Codecs::JSON id=>\"json_36fe591a-c38e-4226-883c-b22b0828e331\", enable_me
tric=>true, charset=>\"UTF-8\">, metadata_target=>\"http_poller_metadata\", id=>\"5d6bfa1d4d737144c6d9fe68b85bfee593ef7b98e8181802ffa44797f07bbcaf\", enable_metric=>true, socket_timeout=>10, connect_timeout=>10, follow_redirects=>true, pool_max=>50, pool_max_per_route=>25, keepalive=>true, automatic
_retries=>1, retry_non_idempotent=>false, validate_after_inactivity=>200, keystore_type=>\"JKS\", truststore_type=>\"JKS\", cookies=>true>", :error=>"Invalid URL https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT NodeID FROM Orion.Nodes", :thread=>"#<Thread:0x46a
e6370 run>"}
[2018-03-15T11:00:29,983][ERROR][logstash.pipeline        ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Invalid URL https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT NodeID FROM Orion.Nodes>, :backtrace=>["C:/
Install/6.2.2/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:105:in `validate_request!'", "C:/Install/6.2.2/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:97:in `norm
alize_request'", "C:/Install/6.2.2/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:57:in `block in setup_requests!'", "org/jruby/RubyHash.java:1343:in `each'", "org/jruby/RubyEnumerable.java:830:in `map'", "C:/Install/6.2.2/logstash-6
.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:57:in `setup_requests!'", "C:/Install/6.2.2/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:47:in `register'", "C:/Install/6.2.2/l
ogstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:341:in `register_plugin'", "C:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:352:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "C:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline
.rb:352:in `register_plugins'", "C:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:502:in `start_inputs'", "C:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:393:in `start_workers'", "C:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:289
:in `run'", "C:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:249:in `block in start'"], :thread=>"#<Thread:0x46ae6370 run>"}
[2018-03-15T11:00:30,029][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: LogStash::PipelineAction::Create/pipeline_id:main, action_result: false", :backtrace=>nil}

and finally.
this is what i get when i run the url from the browser...

Capture

Hi David,

It looks like the URL is an HTTPS url. Is it a self-signed certificate? If yes, then you might need to take care of that portion as mentioned in the documentation.

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html#_example

Also, could you provide the output of the following command:

bin/logstash-plugin list

hi,
it is a self signed certificate,
i will try to read the link you sent.

this is the plugin output:

C:\Install\6.2.2\logstash-6.2.2\bin>logstash-plugin list
logstash-codec-cef
logstash-codec-collectd
logstash-codec-dots
logstash-codec-edn
logstash-codec-edn_lines
logstash-codec-es_bulk
logstash-codec-fluent
logstash-codec-graphite
logstash-codec-json
logstash-codec-json_lines
logstash-codec-line
logstash-codec-msgpack
logstash-codec-multiline
logstash-codec-netflow
logstash-codec-plain
logstash-codec-rubydebug
logstash-filter-aggregate
logstash-filter-anonymize
logstash-filter-cidr
logstash-filter-clone
logstash-filter-csv
logstash-filter-date
logstash-filter-de_dot
logstash-filter-dissect
logstash-filter-dns
logstash-filter-drop
logstash-filter-elasticsearch
logstash-filter-fingerprint
logstash-filter-geoip
logstash-filter-grok
logstash-filter-jdbc_static
logstash-filter-jdbc_streaming
logstash-filter-json
logstash-filter-kv
logstash-filter-metrics
logstash-filter-mutate
logstash-filter-ruby
logstash-filter-sleep
logstash-filter-split
logstash-filter-syslog_pri
logstash-filter-throttle
logstash-filter-translate
logstash-filter-truncate
logstash-filter-urldecode
logstash-filter-useragent
logstash-filter-xml
logstash-input-beats
logstash-input-dead_letter_queue
logstash-input-elasticsearch
logstash-input-exec
logstash-input-file
logstash-input-ganglia
logstash-input-gelf
logstash-input-generator
logstash-input-graphite
logstash-input-heartbeat
logstash-input-http
logstash-input-http_poller
logstash-input-imap
logstash-input-jdbc
logstash-input-kafka
logstash-input-pipe
logstash-input-rabbitmq
logstash-input-redis
logstash-input-s3
logstash-input-snmptrap
logstash-input-sqs
logstash-input-stdin
logstash-input-syslog
logstash-input-tcp
logstash-input-twitter
logstash-input-udp
logstash-input-unix
logstash-output-cloudwatch
logstash-output-csv
logstash-output-elasticsearch
logstash-output-email
logstash-output-file
logstash-output-graphite
logstash-output-http
logstash-output-kafka
logstash-output-lumberjack
logstash-output-nagios
logstash-output-null
logstash-output-pagerduty
logstash-output-pipe
logstash-output-rabbitmq
logstash-output-redis
logstash-output-s3
logstash-output-sns
logstash-output-sqs
logstash-output-stdout
logstash-output-tcp
logstash-output-udp
logstash-output-webhdfs
logstash-patterns-core

i tried generating the certificates, now the error is different:

c:\Install\6.2.2\logstash-6.2.2\bin>logstash -f logstashPipeLine_SolarWinds.conf
Sending Logstash's logs to c:/Install/6.2.2/logstash-6.2.2/logs which is now configured via log4j2.properties
[2018-03-15T17:12:56,070][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"c:/Install/6.2.2/logstash-6.2.2/modules/fb_apache/configuration"}
[2018-03-15T17:12:56,101][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"c:/Install/6.2.2/logstash-6.2.2/modules/netflow/configuration"}
[2018-03-15T17:12:56,461][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-03-15T17:12:58,179][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.2.2"}
[2018-03-15T17:12:58,866][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-03-15T17:13:07,664][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-03-15T17:13:07,773][INFO ][logstash.inputs.http_poller] Registering http_poller Input {:type=>nil, :schedule=>{"cron"=>"* * * * * UTC"}, :timeout=>nil}
[2018-03-15T17:13:07,804][ERROR][logstash.pipeline        ] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::HTTP_Poller urls=>{\"solarwinds\"=>{\"method\"=>\"get\", \"user\"=>\"admin\", \"password\"=>\"Zxasqw12\", \"url\"=>\"https://solarwinds-orion:17778/SolarWinds/Info
rmationService/v3/Json/Query?query=SELECT NodeID FROM Orion.Nodes\", \"cacert\"=>\"C:\\\\Install\\\\6.2.2\\\\logstash-6.2.2\\\\bin\\\\Orion_certs\\\\downloaded_truststore.pem\", \"truststore\"=>\"C:\\\\Install\\\\6.2.2\\\\logstash-6.2.2\\\\bin\\\\Orion_certs\\\\downloaded_truststore.jks\", \"trustst
ore_password\"=>\"Zxasqw1212\", \"headers\"=>{\"Accept\"=>\"application/json\"}}}, request_timeout=>60, schedule=>{\"cron\"=>\"* * * * * UTC\"}, codec=><LogStash::Codecs::JSON id=>\"json_92f7598b-a96c-4408-af45-42b82011dd8e\", enable_metric=>true, charset=>\"UTF-8\">, metadata_target=>\"http_poller_
metadata\", id=>\"b544672aae12e10953bfc1a9baebc02d34b09c5060398187d6e214c597d48cd6\", enable_metric=>true, socket_timeout=>10, connect_timeout=>10, follow_redirects=>true, pool_max=>50, pool_max_per_route=>25, keepalive=>true, automatic_retries=>1, retry_non_idempotent=>false, validate_after_inactiv
ity=>200, keystore_type=>\"JKS\", truststore_type=>\"JKS\", cookies=>true>", :error=>"Invalid URL https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT NodeID FROM Orion.Nodes", :thread=>"#<Thread:0x5ae56971 run>"}
[2018-03-15T17:13:07,992][ERROR][logstash.pipeline        ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Invalid URL https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT NodeID FROM Orion.Nodes>, :backtrace=>["c:/
Install/6.2.2/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:105:in `validate_request!'", "c:/Install/6.2.2/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:97:in `norm
alize_request'", "c:/Install/6.2.2/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:57:in `block in setup_requests!'", "org/jruby/RubyHash.java:1343:in `each'", "org/jruby/RubyEnumerable.java:830:in `map'", "c:/Install/6.2.2/logstash-6
.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:57:in `setup_requests!'", "c:/Install/6.2.2/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-http_poller-4.0.4/lib/logstash/inputs/http_poller.rb:47:in `register'", "c:/Install/6.2.2/l
ogstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:341:in `register_plugin'", "c:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:352:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "c:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline
.rb:352:in `register_plugins'", "c:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:502:in `start_inputs'", "c:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:393:in `start_workers'", "c:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:289
:in `run'", "c:/Install/6.2.2/logstash-6.2.2/logstash-core/lib/logstash/pipeline.rb:249:in `block in start'"], :thread=>"#<Thread:0x5ae56971 run>"}
[2018-03-15T17:13:08,070][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: LogStash::PipelineAction::Create/pipeline_id:main, action_result: false", :backtrace=>nil}

this is the new logstash config file:

input {
  http_poller {
    urls => {
      solarwinds => {
        # Supports all options supported by ruby's Manticore HTTP client
        method => get
        user => "admin"
        password => "Zxasqw12"
        url =>
		"https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT NodeID FROM Orion.Nodes"
		cacert => "C:\Install\6.2.2\logstash-6.2.2\bin\Orion_certs\downloaded_truststore.pem"
		truststore => "C:\Install\6.2.2\logstash-6.2.2\bin\Orion_certs\downloaded_truststore.jks"
		truststore_password => "Zxasqw1212"
        headers => {
          Accept => "application/json"
        }
     }
    }
    request_timeout => 60
    # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
    schedule => { cron => "* * * * * UTC"}
    codec => "json"
    # A hash of request metadata info (timing, response headers, etc.) will be sent here
    metadata_target => "http_poller_metadata"
  }
}

output {
  stdout {
  codec => rubydebug 
  }
}

Spaces aren't valid in urls.

Many browsers will let you type spaces, and will silently encode them for you before making the request (that is, replace them with either a plus character (+) or a code-point escape sequence (%20)).

You likely can simply replace the spaces in the URL with plus signs.

2 Likes

You are trully a life saver :wink:
the spaces was indeed the issue, but now for some reason i get this error:

c:\Install\6.2.2\logstash-6.2.2\bin>logstash -f logstashPipeLine_SolarWinds.conf
Sending Logstash's logs to c:/Install/6.2.2/logstash-6.2.2/logs which is now configured via log4j2.properties
[2018-03-18T10:33:57,336][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"c:/Install/6.2.2/logstash-6.2.2/modules/fb_apache/configuration"}
[2018-03-18T10:33:57,352][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"c:/Install/6.2.2/logstash-6.2.2/modules/netflow/configuration"}
[2018-03-18T10:33:57,633][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-03-18T10:33:58,368][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.2.2"}
[2018-03-18T10:33:59,243][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-03-18T10:34:06,743][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-03-18T10:34:07,322][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://win-ej0hvlfu246:9200/]}}
[2018-03-18T10:34:07,337][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://win-ej0hvlfu246:9200/, :path=>"/"}
[2018-03-18T10:34:07,555][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://win-ej0hvlfu246:9200/"}
[2018-03-18T10:34:07,618][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>nil}
[2018-03-18T10:34:07,634][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-03-18T10:34:07,650][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-03-18T10:34:07,681][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "
match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"
}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-03-18T10:34:07,727][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://win-ej0hvlfu246:9200/"]}
[2018-03-18T10:34:07,774][INFO ][logstash.inputs.http_poller] Registering http_poller Input {:type=>nil, :schedule=>{"cron"=>"* * * * * UTC"}, :timeout=>nil}
[2018-03-18T10:34:07,837][INFO ][logstash.pipeline        ] Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x27973c7f sleep>"}
[2018-03-18T10:34:07,962][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}
{
                    "tags" => [
        [0] "_http_request_failure"
    ],
    "http_request_failure" => {
                   "name" => "solarwinds",
              "backtrace" => nil,
        "runtime_seconds" => 0.234,
                  "error" => "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
                "request" => {
             "cacert" => "C:\\cert\\downloaded_cert.pem",
             "method" => "get",
               "auth" => {
                 "pass" => "Zxasqw12",
                 "user" => "admin",
                "eager" => true
            },
                "url" => "https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT%20N.NodeID,N.IPAddress,N.Caption,N.NodeDescription,N.Description,N.DNS,N.SysName,N.Vendor,N.Location,N.IOSVersion,N.MachineType,N.NodeName,I.InterfaceID,I.ObjectSubType,I.Name,I.Index,I.
TypeName,I.TypeDescription,I.PhysicalAddress,I.Caption%20as%20Caption2,I.FullName,I.Alias,I.InterfaceCaption,I.MAC,I.InterfaceIndex%20FROM%20Orion.Nodes%20as%20N%20LEFT%20JOIN%20Orion.NPM.Interfaces%20AS%20I%20ON%20I.NodeID%20=%20N.NodeID",
            "headers" => {
                "Accept" => "application/json"
            }
        }
    },
    "http_poller_metadata" => {
                   "name" => "solarwinds",
        "runtime_seconds" => nil,
                   "host" => "WIN-EJ0HVLFU246",
                "request" => {
             "cacert" => "C:\\cert\\downloaded_cert.pem",
             "method" => "get",
               "auth" => {
                 "pass" => "Zxasqw12",
                 "user" => "admin",
                "eager" => true
            },
                "url" => "https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT%20N.NodeID,N.IPAddress,N.Caption,N.NodeDescription,N.Description,N.DNS,N.SysName,N.Vendor,N.Location,N.IOSVersion,N.MachineType,N.NodeName,I.InterfaceID,I.ObjectSubType,I.Name,I.Index,I.
TypeName,I.TypeDescription,I.PhysicalAddress,I.Caption%20as%20Caption2,I.FullName,I.Alias,I.InterfaceCaption,I.MAC,I.InterfaceIndex%20FROM%20Orion.Nodes%20as%20N%20LEFT%20JOIN%20Orion.NPM.Interfaces%20AS%20I%20ON%20I.NodeID%20=%20N.NodeID",
            "headers" => {
                "Accept" => "application/json"
            }
        }
    },
                "@version" => "1",
              "@timestamp" => 2018-03-18T08:35:00.540Z
}

i tried changing the certificate directory to something simpler like c:\cert* , and also tried to open the cmd as administrator in case this was a permission issue.
nothing worked,

what can be the problem ?
also, is there a way to ignore the certificate, just as i can do on a browser ?

The error appears to be more of a "cannot build certificate chain from the remote server back to any of the CA's you've provided" than a "cannot load the provided CA".

Let's simplify by cutting Logstash out of the mix; do you have the command line utilty curl installed? If so, the following should validate that the given PEM is an appropriate CA for the certificate:

curl --cacert $PATH_TO_CERT --request GET https://solarwinds-orion:17778/

hi,
i get this:

C:\Program Files\curl-7.59.0-win64-mingw\bin>curl --cacert "C:\Program Files\curl-7.59.0-win64-mingw\bin\downloaded_cert.pem" --request GET https://solarwinds-orion:17778/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Not Found</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Not Found</h2>
<hr><p>HTTP Error 404. The requested resource is not found.</p>
</BODY></HTML>

i dont know what to do, can i just ignore the certificate ?

There is presently no option in the http poller to ignore the certificate; apparently one existed in the past, but it never worked correctly and was deprecated.

We've seen some issues in other plugins where the presence of a Windows drive letter prevented things from working correctly (after all, when one works exclusively on Linux and POSIX-compliant systems, it's easy to incorrectly assume that absolute paths always start with a forward slash); I would attempt removing it (and if that works, report it as a bug on the plugin).

thanks, ill try and update.

hi,
i tried installing logstash on ubuntu,
when i try the openssl and curl commands i get the same error...
look at this screenshot, does this give you any clues?

thanks again for your help, i really appreciate it.

The curl command looks like a success -- the TLS connection is being made successfully, and the result is a 404 (which is okay here, since we were attempting to validate the connection itself)

hi,
this is the logstash output on the same machine:

dudug@elastic-ubuntu:/usr/share/logstash/bin$ sudo ./logstash -f logstashPipeLine_SolarWinds.conf 
[INFO ] 2018-03-21 13:44:54.081 [main] scaffold - Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[INFO ] 2018-03-21 13:44:54.120 [main] scaffold - Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[WARN ] 2018-03-21 13:44:54.807 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-03-21 13:44:55.119 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.2.3"}
[INFO ] 2018-03-21 13:44:55.356 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2018-03-21 13:44:56.903 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-03-21 13:44:56.988 [[main]-pipeline-manager] http_poller - Registering http_poller Input {:type=>nil, :schedule=>{"cron"=>"* * * * * UTC"}, :timeout=>nil}
[INFO ] 2018-03-21 13:44:57.073 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x1c198798@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}
[INFO ] 2018-03-21 13:44:57.180 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Pipelines running {:count=>1, :pipelines=>["main"]}
{
    "http_poller_metadata" => {
                   "name" => "solarwinds",
                   "host" => "elastic-ubuntu",
                "request" => {
                         "method" => "get",
                     "truststore" => "/home/dudug/Downloads/downloaded_truststore.jks",
                         "cacert" => "/home/dudug/Downloads/downloaded_cert.pem",
            "truststore_password" => "Zxasqw1212",
                        "headers" => {
                "Accept" => "application/json"
            },
                           "auth" => {
                "eager" => true,
                 "pass" => "Zxasqw12",
                 "user" => "admin"
            },
                            "url" => "https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT%20N.NodeID,N.IPAddress,N.Caption,N.NodeDescription,N.Description,N.DNS,N.SysName,N.Vendor,N.Location,N.IOSVersion,N.MachineType,N.NodeName,I.InterfaceID,I.ObjectSubType,I.Name,I.Index,I.TypeName,I.TypeDescription,I.PhysicalAddress,I.Caption%20as%20Caption2,I.FullName,I.Alias,I.InterfaceCaption,I.MAC,I.InterfaceIndex%20FROM%20Orion.Nodes%20as%20N%20LEFT%20JOIN%20Orion.NPM.Interfaces%20AS%20I%20ON%20I.NodeID%20=%20N.NodeID"
        },
        "runtime_seconds" => nil
    },
                    "tags" => [
        [0] "_http_request_failure"
    ],
              "@timestamp" => 2018-03-21T11:45:00.711Z,
    "http_request_failure" => {
                "request" => {
                         "method" => "get",
                     "truststore" => "/home/dudug/Downloads/downloaded_truststore.jks",
                         "cacert" => "/home/dudug/Downloads/downloaded_cert.pem",
            "truststore_password" => "Zxasqw1212",
                        "headers" => {
                "Accept" => "application/json"
            },
                           "auth" => {
                "eager" => true,
                 "pass" => "Zxasqw12",
                 "user" => "admin"
            },
                            "url" => "https://solarwinds-orion:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT%20N.NodeID,N.IPAddress,N.Caption,N.NodeDescription,N.Description,N.DNS,N.SysName,N.Vendor,N.Location,N.IOSVersion,N.MachineType,N.NodeName,I.InterfaceID,I.ObjectSubType,I.Name,I.Index,I.TypeName,I.TypeDescription,I.PhysicalAddress,I.Caption%20as%20Caption2,I.FullName,I.Alias,I.InterfaceCaption,I.MAC,I.InterfaceIndex%20FROM%20Orion.Nodes%20as%20N%20LEFT%20JOIN%20Orion.NPM.Interfaces%20AS%20I%20ON%20I.NodeID%20=%20N.NodeID"
        },
                   "name" => "solarwinds",
              "backtrace" => nil,
        "runtime_seconds" => 0.466907,
                  "error" => "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
    },
                "@version" => "1"
}

this is a proof of the files location:

dudug@elastic-ubuntu:/usr/share/logstash/bin$ ls /home/dudug/Downloads/
downloaded_cert.pem  downloaded_truststore.jks  logstash-6.2.3.deb
dudug@elastic-ubuntu:/usr/share/logstash/bin$

what is this error?

  "error" => "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"

thanks again :slight_smile:

Try a config like this without setting the method and headers (this works is a config I'm using for a website).

input {
  http_poller {
    urls => {
      url => "myip"
     }
    cacert => "cert.pem"
    truststore => "truststore.jks"
    user => "admin"
    password => "pass"
    truststore_password => "pass"
    schedule => { cron => "50 * * * * UTC"}
    codec => "json"
  }
}
1 Like

i can see that there is no path specified for the .pem and .jks files, is that how you use it too ? if so , where do you actually put the files, so that they will be found ?

thanks,
David.

I edited out the path but you need to set something like /path/cert.pem and /path/truststore.jks. Though if you put them in the logstash root directory you might not need to set a path. Didn't test that so just set a path.

this as aline not two

thank you very much,
works like a charm !