Gz file handling by interaction as XML files

Elastic Stack Logstash
1

Hi
How Can I configure input for reading per only one file. It is very important that logstash reads one file at a time as soon as it finishes parsing it and reads another one. This is related to the xml structure.
I have tried something like this


input {
    file {
        path =>"/data/data_1/A20211029*.gz"
        codec => multiline { pattern => "</measCollecFile>" negate => true what => next max_lines => 300000 max_bytes => 6000000 auto_flush_interval => 1 }
        start_position => "beginning"
        sincedb_path => "/dev/null"
        max_open_files => 1
        type => "xml"
    }
}

but max_open_files didn't cover above my requirements.

2

You cannot read gzipped files in tail mode.

The file input documentation says it will read up to 4 exabytes from each file before it processes the next. It should be working the way you want.

3

ok, so other things can we set up the pauseinterval between EOF and start to read next one?

4

stat_interval I'm right?

5

I'm sorry, I really do not understand what you are trying to do.

6

I wanted to introduce an interval for loading a files

7

stat_interval tells the input how often to wait between checks for new data. The wait and the time taken to read the files determine the actual time between checks.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.