I am using logstash to read .gz files. The gzip files are around 10 to 15GB for which logstash was unable to process and crashing. So I have split the .gz files using 7zip split based on size of 3GB .gz file chunks. i see logstash easily processing so far except the issue that it cannot determine the EOF which is significant in read mode to process .gz files. and logstash keep processing the same file and not reading the other files specified in path.
Error: Unexpected end of ZLIB input stream
- Exception: Java::JavaIo::EOFException*
- Stack: java.util.zip.InflaterInputStream.fill(java/util/zip/InflaterInputStream.java:245)*
Any help is greatly appreciated. Thanks.