# Logstash s3-sqs input plugin to differentiate 2 types of log

**URL:** <https://discuss.elastic.co/t/logstash-s3-sqs-input-plugin-to-differentiate-2-types-of-log/247971>\
**Category:** Logstash\
**Created:** [September 9, 2020, 7:49am UTC](https://discuss.elastic.co/t/logstash-s3-sqs-input-plugin-to-differentiate-2-types-of-log/247971 "2020-09-09T07:49:43Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vinodjv\_3](https://avatars.discourse-cdn.com/v4/letter/v/f17d59/32.png) [@vinodjv\_3](https://discuss.elastic.co/u/vinodjv_3)\
**Post date:** [September 9, 2020, 7:49am UTC](https://discuss.elastic.co/t/logstash-s3-sqs-input-plugin-to-differentiate-2-types-of-log/247971/1 "2020-09-09T07:49:43Z")

</div>

Hi all,

I have an use case where application logs are pushed to s3 bucket, log types are json and plain. Am using [logstash-input-s3-sns-sqs] plugin to read logs from S3 bucket. Both log types are pushed to different path in same bucket  
Json log path : \<my\_bucket\>/jsonlog/  
plain log path: \<my\_bucket\>/plainlog/

I am using below input filter and trying to separate these logs but it does not seem to work. Any thoughts on this is welcomed.

```auto
    input {
      s3snssqs {
        region => "us-east-1"
        s3_default_options => { "endpoint_discovery" => true }
        queue => "<sqs_url>"
        from_sns => false
        s3_options_by_bucket => [
            { bucket_name => "<my_bucket>"
            folders => [
            { key => ".*app1.log"
            codec => "json"
            type => "myjsonlog"},
            { key => ".*app2.log"
    	codec => "plain"
            type => "myplainlog"}]
            }
            ]
    }
    }

```

Thanks,  
Vinod kumar

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 7, 2020, 7:49am UTC](https://discuss.elastic.co/t/logstash-s3-sqs-input-plugin-to-differentiate-2-types-of-log/247971/2 "2020-10-07T07:49:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.