# Logstash gzip\_pattern matching

**URL:** <https://discuss.elastic.co/t/logstash-gzip-pattern-matching/233026>\
**Category:** Logstash\
**Created:** [May 18, 2020, 3:29am UTC](https://discuss.elastic.co/t/logstash-gzip-pattern-matching/233026 "2020-05-18T03:29:50Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ckough](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ckough/32/61892_2.png) [@ckough](https://discuss.elastic.co/u/ckough)\
**Post date:** [May 18, 2020, 3:29am UTC](https://discuss.elastic.co/t/logstash-gzip-pattern-matching/233026/1 "2020-05-18T03:29:51Z")

</div>

Hi there,

I'm using ELK 7.7. I need to ingest s3 objects with description exactly like this

> <https://github.com/logstash-plugins/logstash-input-s3/issues/165>
>
> We are using S3 input plugin to pull data from S3, we are using kinesis firehose… to push data to S3, which does not adds 'gz' extension to the files, but data is in zipped format. 
> 
> S3 input plugin is pulling data from S3 it can only decompress the data which has gz as extension since while code walk through we we notice there is a condition checking 'gz' as extension then only it decompress the file data.
> 
> We do not want to maintain the custom plugin at our end just for small change.
> 
> I just wanted to ask can we add a flag over there, which can be set as configuration and used with that condition, so that even files which use can set intentionally to decompress files does not have 'gz' extension or with any extension.
> 
> Can you provide such feature as same plugin or can we contribute to this input plugin, if yes how and what is the process for the same?

> We are using S3 input plugin to pull data from S3, we are using kinesis firehose to push data to S3, which does not adds 'gz' extension to the files, but data is in zipped format. S3 input plugin is pulling data from S3 it can only decompress the data which has gz as extension since while code walk through we we notice there is a condition checking 'gz' as extension then only it decompress the file data.

I know of this gzip\_pattern flag in logstash s3 plugin that resolves this.

How exactly do I use it to match all files (no gz or gzip extention) in my S3 folder (s3bucket/cloudwatchlogs/yyyy/mm/dd/hh/filename) which consists only of files pulled from cloudwatch?

Thanks in advance  
ck

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 15, 2020, 3:40am UTC](https://discuss.elastic.co/t/logstash-gzip-pattern-matching/233026/2 "2020-06-15T03:40:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.