# Logstash http\_poller plugin help

**URL:** https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957
**Category:** Logstash
**Created:** [February 4, 2017, 8:10pm UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957 "2017-02-04T20:10:18Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![FrankC](https://avatars.discourse-cdn.com/v4/letter/f/ed655f/32.png) [@FrankC](https://discuss.elastic.co/u/FrankC)
#### Post date: [February 4, 2017, 8:10pm UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957/1 "2017-02-04T20:10:18Z")

</div>

Hello,

I am trying to use the http\_poller plugin to pull data from an API end point such as the Quandl service.

First I tested the http\_poller using ES url: [http://localhost:9200](http://localhost:9200). That worked.

Then I changed url to an quandl API. I verified that my query works using a rest client app.

I get an error message and need some help debugging.

---- my config file  
input {  
http\_poller {  
urls =\> {  
method =\> get  
q1 =\> "[https://www.quandl.com/api/v3/datatables/WIKI/PRICES.json?date.gte=20170201&date.lt=20170204&ticker=MSFT&api\_key=my\_key\_goes\_here](https://www.quandl.com/api/v3/datatables/WIKI/PRICES.json?date.gte=20170201&date.lt=20170204&ticker=MSFT&api_key=my_key_goes_here)"  
headers =\> {  
Accept =\> "application/json"  
}  
}  
codec =\> "json"  
request\_timeout =\> 60  
metadata\_target =\> "http\_poller\_metadata"  
schedule =\> { cron =\> "\* \* \* \* \* UTC"}  
}  
}

output {  
stdout {  
codec =\> rubydebug  
}  
}

Snippet of the error message:  
2017-02-04T13:47:46,978][INFO][logstash.inputs.http\_poller] Registering http\_poller Input {:type=\>nil, :urls=\>{"method"=\>"get", "q1"=\>"[https://www.quandl.com/api/v3/datatables/WIKI/PRICES.json?date.gte=20170201&date.lt=20170204&ticker=MSFT&api\_key=my\_key\_goes\_here](https://www.quandl.com/api/v3/datatables/WIKI/PRICES.json?date.gte=20170201&date.lt=20170204&ticker=MSFT&api_key=my_key_goes_here)", "headers"=\>{"Accept"=\>"application/json"}}, :interval=\>nil, :schedule=\>{"cron"=\>"\* \* \* \* \* UTC"}, :timeout=\>nil}  
[2017-02-04T13:47:47,013][ERROR][logstash.agent] Pipeline aborted due to error {:exception=\>#\<LogStash::ConfigurationError: Invalid URL get\>, :backtrace=\>[....

Thanks,  
Frank

---

<div class="post-metadata">

### Author: ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)
#### Post date: [February 4, 2017, 10:42pm UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957/2 "2017-02-04T22:42:31Z")

</div>

The back trace is the most interesting thing.  
Could you provide this ?

---

<div class="post-metadata">

### Author: ![FrankC](https://avatars.discourse-cdn.com/v4/letter/f/ed655f/32.png) [@FrankC](https://discuss.elastic.co/u/FrankC)
#### Post date: [February 5, 2017, 12:36am UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957/3 "2017-02-05T00:36:42Z")

</div>

Hi Fabien,

Here is the error message with traceback:

[2017-02-04T18:29:00,410][INFO][logstash.inputs.http\_poller] Registering http\_poller Input {:type=\>nil, :urls=\>{"method"=\>"get", "q1"=\>"[https://www.quandl.com/api/v3/datatables/WIKI/PRICES.json?date.gte=20170201&date.lt=20170204&ticker=MSFT&api\_key=my\_api\_key”](https://www.quandl.com/api/v3/datatables/WIKI/PRICES.json?date.gte=20170201&date.lt=20170204&ticker=MSFT&api_key=my_api_key%E2%80%9D), "headers"=\>{"Accept"=\>"application/json"}}, :interval=\>nil, :schedule=\>{"cron"=\>"\* \* \* \* \* UTC"}, :timeout=\>nil}  
[2017-02-04T18:29:00,484][ERROR][logstash.agent] Pipeline aborted due to error {:exception=\>#\<LogStash::ConfigurationError: Invalid URL get\>, :backtrace=\>["/Users/es\_usr/Downloads/logstash-5.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-http\_poller-3.1.0/lib/logstash/inputs/http\_poller.rb:163:in `validate_request!'", "/Users/es_usr/Downloads/logstash-5.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-http_poller-3.1.0/lib/logstash/inputs/http_poller.rb:155:in`normalize\_request'", "/Users/es\_usr/Downloads/logstash-5.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-http\_poller-3.1.0/lib/logstash/inputs/http\_poller.rb:132:in `setup_requests!'", "org/jruby/RubyHash.java:1342:in`each'", "org/jruby/RubyEnumerable.java:757:in `map'", "/Users/es_usr/Downloads/logstash-5.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-http_poller-3.1.0/lib/logstash/inputs/http_poller.rb:132:in`setup\_requests!'", "/Users/es\_usr/Downloads/logstash-5.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-http\_poller-3.1.0/lib/logstash/inputs/http\_poller.rb:122:in `register'", "/Users/es_usr/Downloads/logstash-5.1.1/logstash-core/lib/logstash/pipeline.rb:353:in`start\_inputs'", "org/jruby/RubyArray.java:1613:in `each'", "/Users/es_usr/Downloads/logstash-5.1.1/logstash-core/lib/logstash/pipeline.rb:352:in`start\_inputs'", "/Users/es\_usr/Downloads/logstash-5.1.1/logstash-core/lib/logstash/pipeline.rb:228:in `start_workers'", "/Users/es_usr/Downloads/logstash-5.1.1/logstash-core/lib/logstash/pipeline.rb:183:in`run'", "/Users/es\_usr/Downloads/logstash-5.1.1/logstash-core/lib/logstash/agent.rb:292:in `start\_pipeline'"]}  
[2017-02-04T18:29:00,541][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}  
[2017-02-04T18:29:03,498][WARN][logstash.agent] stopping pipeline {:id=\>"main"}

---

<div class="post-metadata">

### Author: ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)
#### Post date: [February 5, 2017, 7:10am UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957/4 "2017-02-05T07:10:14Z")

</div>

It is because your "urls" config is wrong.  
Up to the plugin doc, it should be :

```
urls => {
    q1 => {
        method => get
        url => "https://www.quandl.com/api/v3/datatables/WIKI/PRICES.json?date.gte=20170201&date.lt=20170204&ticker=MSFT&api_key=my_key_goes_here"
        headers => {
            Accept => "application/json"
        }
    }
}
```

---

<div class="post-metadata">

### Author: ![FrankC](https://avatars.discourse-cdn.com/v4/letter/f/ed655f/32.png) [@FrankC](https://discuss.elastic.co/u/FrankC)
#### Post date: [February 5, 2017, 11:54am UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957/5 "2017-02-05T11:54:40Z")

</div>

Thanks Fabien!

---

<div class="post-metadata">

### Author: ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)
#### Post date: [February 5, 2017, 3:37pm UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957/6 "2017-02-05T15:37:00Z")

</div>

You're welcome 🙂

If your problem is solved, could you mark my previous comment as "solution" to close the topic ?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 5, 2017, 3:37pm UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957/7 "2017-03-05T15:37:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.