Reading logs from URL

Elastic Stack Logstash
1

Logstash setup should read from beat and http url so the configuration that am using is like :

input {
beats {
	port=>5044
}

http {
	method => "POST"
	host => "localhost"
	port => 8080
	path => "/context/atest"
}
}
  1. Do i need a license to use this plugin ?
  2. I found path in some example so is this configuration going to work ?
2

Do i need a license to use this plugin ?

No.

I found path in some example so is this configuration going to work ?

According to the http input documentation path is not a valid option, nor is method. Are you sure you shouldn't be using the http_poller plugin instead?

3

So as i understand if i want input from a specific path and method then i will have to use a http_poller ?

http_poller {
    # List of urls to hit
    # URLs can either have a simple format for a get request
    # Or use more complex HTTP features
    urls => {
      uiservice => "http://localhost:8080/envliven/path"
     
    }
    # Maximum amount of time to wait for a request to complete
    request_timeout => 30
    # How far apart requests should be
    interval => 60
    # Decode the results as JSON
    codec => "json"
    # Store metadata about the request in this key
    metadata_target => "http_poller_metadata"
  }

Dooes this configuration looks ok for basic setup ?

4

Looks good OTOH, but try it out.

http_poller is for when you want Logstash to make the HTTP requests. The http input sets up Logstash as an HTTP server so that others can post data.

5

UI logs its logs to a specific rest url and that data is getting read by filebeat. Also i have to read this URL continuously so that i can send data to logstash. So as per my requirement i was suggest http_poller is the better option. Please let me know if i am missing something here.

6

How does the endpoint (http://localhost:8080/envliven/path) work? Does it return a batch of log entries each time or does it stream logs indefinitely?

7

It streams logs indefinitely.

8

Okay. I'm not sure the http_poller input supports such endpoints. Try it out, but you'll want to use the json_lines codec rather than the json codec.

9

ok then what kind of streaming we have to do so that it can be supported. Can you suggest so tool, just want to get some clarity on that.

10

If http_poller doesn't work you could for example

  • investigate the plugin to see if it could be made to work for the streaming case, or
  • write a separate input plugin that covers this use case, or
  • write a separate program that makes the HTTP requests and somehow transmits the results to Logstash.
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.