# Reading logs from URL **URL:** \ **Category:** Logstash\ **Created:** [August 6, 2018, 4:40am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089 "2018-08-06T04:40:13Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Post date:** [August 6, 2018, 4:40am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/1 "2018-08-06T04:40:13Z")
Logstash setup should read from beat and http url so the configuration that am using is like : ``` input { beats { port=>5044 } http { method => "POST" host => "localhost" port => 8080 path => "/context/atest" } } ``` 1. Do i need a license to use this plugin ? 2. I found path in some example so is this configuration going to work ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 6, 2018, 5:41am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/2 "2018-08-06T05:41:36Z")
> Do i need a license to use this plugin ? No. > I found path in some example so is this configuration going to work ? According to the [http input documentation](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html) `path` is not a valid option, nor is `method`. Are you sure you shouldn't be using the http\_poller plugin instead? ---
**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Post date:** [August 6, 2018, 7:12am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/3 "2018-08-06T07:12:34Z")
So as i understand if i want input from a specific path and method then i will have to use a http\_poller ? ``` http_poller { # List of urls to hit # URLs can either have a simple format for a get request # Or use more complex HTTP features urls => { uiservice => "http://localhost:8080/envliven/path" } # Maximum amount of time to wait for a request to complete request_timeout => 30 # How far apart requests should be interval => 60 # Decode the results as JSON codec => "json" # Store metadata about the request in this key metadata_target => "http_poller_metadata" } ``` Dooes this configuration looks ok for basic setup ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 6, 2018, 7:40am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/4 "2018-08-06T07:40:33Z")
Looks good OTOH, but try it out. http\_poller is for when you want Logstash to make the HTTP requests. The http input sets up Logstash as an HTTP server so that others can post data. ---
**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Post date:** [August 6, 2018, 7:45am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/5 "2018-08-06T07:45:40Z")
UI logs its logs to a specific rest url and that data is getting read by filebeat. Also i have to read this URL continuously so that i can send data to logstash. So as per my requirement i was suggest http\_poller is the better option. Please let me know if i am missing something here. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 6, 2018, 7:50am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/6 "2018-08-06T07:50:23Z")
How does the endpoint ([http://localhost:8080/envliven/path](http://localhost:8080/envliven/path)) work? Does it return a batch of log entries each time or does it stream logs indefinitely? ---
**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Post date:** [August 6, 2018, 8:41am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/7 "2018-08-06T08:41:52Z")
It streams logs indefinitely. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 6, 2018, 8:46am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/8 "2018-08-06T08:46:44Z")
Okay. I'm not sure the http\_poller input supports such endpoints. Try it out, but you'll want to use the json\_lines codec rather than the json codec. ---
**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Post date:** [August 6, 2018, 8:51am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/9 "2018-08-06T08:51:05Z")
ok then what kind of streaming we have to do so that it can be supported. Can you suggest so tool, just want to get some clarity on that. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 6, 2018, 9:39am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/10 "2018-08-06T09:39:48Z")
If http\_poller doesn't work you could for example - investigate the plugin to see if it could be made to work for the streaming case, or - write a separate input plugin that covers this use case, or - write a separate program that makes the HTTP requests and somehow transmits the results to Logstash. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 3, 2018, 9:47am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089/11 "2018-09-03T09:47:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.