Logstash http_poller - fetching big files

Elastic Stack Logstash
1

Hello.

I have 1GB json file that changes and it is on the remote server. I am able to do the fetch with the file size approx 500 MB, but not 1GB.

input {
 http_poller {
  urls => { 
        url1 => "https://externalserver:80/all.json"
  }
  request_timeout => 9000
  socket_timeout => 9000
  tags => ["urls"]
  codec => "json"
  validate_after_inactivity => 200
  schedule => { cron => "43 * * * *" }
 }
}

My machine is is very capable of handling any processing and hardware is top-notch.
Does anyone knows what is the top limit for http_poller for fetching files?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.