Http_poller input giving read time out

Elastic Stack Logstash
1

Http_poller input giving read time out when data is more 100000. please help me on this


{
             "@timestamp" => 2018-04-04T19:17:10.278Z,
                   "tags" => [
       [0] "_http_request_failure",
       [1] "_split_type_failure"
   ],
   "http_request_failure" => {
               "request" => {
           "headers" => {
               "Accept" => "application/json"
           },
               "url" => "http://3.204.110.149:5004/fetchAll",
            "method" => "get"
       },
                 "error" => "Read timed out",
       "runtime_seconds" => 10.021353000000001,
                  "name" => "test1",
             "backtrace" => nil
   }
2

The connect timeout by default is 10s, and it looks like you're hitting it. This means that the remote host isn't responding to open the connection. Do you have a route to the host?

3

thanks for replying. it works fine if the data is less.
i have kept timeout more than 10s enventhough the same issue occurs . please refer the input below

input {
  http_poller {
    urls => {
      test1 => {
        # Supports all options supported by ruby's Manticore HTTP client
        method => get
        url => "http://3.204.110.149:5004/fetchAll"
        headers => {
          Accept => "application/json"
        }
     }

    }
    connect_timeout => 1000000
    request_timeout => 900000
    automatic_retries => 10
    # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
    #schedule => { cron => "* * * * * UTC"}
    codec => "json"
    # A hash of request metadata info (timing, response headers, etc.) will be sent here
    metadata_target => "http_poller_metadata"
  }

``
4

When you increase the timeout, does it take longer to timeout? When you observe the network throughput on the machine, is it actually pulling data? If you curl the same URL manually from the Logstash box, how big is the payload, and how long does it take to arrive?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.