How does logstash read multiple gunzipped files

Elastic Stack Logstash
1

I have set up a simple logstash pipeline with a single file input (mode= 'read'). I have given a regex pattern in the path attribute. I am trying to read a number of gunzipped files matching the regex in the path. The thing I wanted to understand is how logstash reads and processes multiple files (especially when they .gz files), as some of them could be of the order of 100s of MBs and I am worried about logstash crashing if it tried to load multiple of them in the memory simultaneously and then uncompressing and processing them.

input {
          file {
                    path => "/tmp/a/*/*/*.log.gz" #path to .gz files contained in subfolders
                    mode => "read"
                    file_completed_action => "log"
                    file_completed_log_path => "/tmp/log.txt"
            }
    }
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.