# Logstash S3 input plugin prefix with full path is ignored

**URL:** <https://discuss.elastic.co/t/logstash-s3-input-plugin-prefix-with-full-path-is-ignored/267236>\
**Category:** Logstash\
**Created:** [March 15, 2021, 9:17am UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-prefix-with-full-path-is-ignored/267236 "2021-03-15T09:17:48Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![noako](https://avatars.discourse-cdn.com/v4/letter/n/3ec8ea/32.png) [@noako](https://discuss.elastic.co/u/noako)\
**Post date:** [March 15, 2021, 9:17am UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-prefix-with-full-path-is-ignored/267236/1 "2021-03-15T09:17:48Z")

</div>

I am using logstash docker image with s3 input plugin for input,  
the issue I am having is:  
if I set the prefix to the full file path, then the file is ignored, with this log:

```auto
2021-03-14T11:43:56,233][DEBUG][logstash.inputs.s3][main][8e3974da4fe5e313ca0fe57223ea9bc483a895aa4b458a3cc0d24e67a9a64d6e] Found key {:key=>"issues/actual-logs-test/SPSServer.0.log.gz"}
**[2021-03-14T11:43:56,236][DEBUG][logstash.inputs.s3][main][8e3974da4fe5e313ca0fe57223ea9bc483a895aa4b458a3cc0d24e67a9a64d6e] Ignoring {:key=>"issues/actual-logs-test/SPSServer.0.log.gz"}**
[2021-03-14T11:43:57,267][DEBUG][logstash.inputs.s3][main][8e3974da4fe5e313ca0fe57223ea9bc483a895aa4b458a3cc0d24e67a9a64d6e] Closing {:plugin=>"LogStash::Inputs::S3"}

```

looking at the source code, it seems to be casued by this logic:  
[logstash-input-s3/s3.rb at 1314a75b8c190f87e69ad0232065550fac612d64 · logstash-plugins/logstash-input-s3 · GitHub](https://github.com/logstash-plugins/logstash-input-s3/blob/1314a75b8c190f87e69ad0232065550fac612d64/lib/logstash/inputs/s3.rb#L359)),

```auto
def ignore_filename?(filename)
    if @prefix == filename
      return true
    elsif filename.end_with?("/")
      return true
    elsif (@backup_add_prefix && @backup_to_bucket == @bucket && filename =~ /^#{backup_add_prefix}/)
      return true
    elsif @exclude_pattern.nil?
      return false
    elsif filename =~ Regexp.new(@exclude_pattern)
      return true
    else
      return false
    end
  end

```

but it still doesn't seem like this the correct behaviour

this is the conf template:

```auto
input {
    s3 {
        "bucket" => "{{bucketName}}"
        "region" => "{{region}}"
        "prefix" => "{{logsPath}}"
        "access_key_id" => "{{accessKeyId}}"
        "secret_access_key" => "{{secretAccessKey}}"
        "session_token" => "{{sessionToken}}"
        "additional_settings" => {
            "force_path_style" => true
            "follow_redirects" => false
            }
        "watch_for_new_files" => false
    }
}
output { elasticsearch {
    hosts => ["{{eksEndpoint}}"]
    index => "{{index}}"
    user => "{{eksUser}}"
    password => "{{eksPassword}}"
    ilm_enabled => false
    ssl => true
    }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 12, 2021, 9:18am UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-prefix-with-full-path-is-ignored/267236/2 "2021-04-12T09:18:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.