# Logstash S3 output plugin fails to upload txt files in S3

**URL:** <https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352>\
**Category:** Logstash\
**Created:** [February 15, 2024, 9:53am UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352 "2024-02-15T09:53:44Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aniket\_Chakrabarty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aniket_chakrabarty/32/131775_2.png) [@Aniket\_Chakrabarty](https://discuss.elastic.co/u/Aniket_Chakrabarty)\
**Post date:** [February 15, 2024, 9:53am UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352/1 "2024-02-15T09:53:44Z")

</div>

Hi,  
We have tried to upload a txt file through logstash s3 output plugin but having issues it says:  
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console

My logstash.conf config is like:

```auto
input {
  file {
    path => ["/var/log/test/abc.txt"]
    sincedb_path => "/dev/null"
    start_position => "beginning"
  }
}

filter {
  # Your filters if needed
}

output {
  s3 {
    access_key_id => "ID"
    secret_access_key => "key"
    region => "us-east-1"
    bucket => "qa-alb-logs"
    #size_file => 1
    time_file => 1
    prefix => "test/"
  }
}

```

need to get a step by step guide for this.

---

<div class="post-metadata">

**Author:** ![ersalil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ersalil/32/131773_2.png) [@ersalil](https://discuss.elastic.co/u/ersalil)\
**Post date:** [February 15, 2024, 12:21pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352/2 "2024-02-15T12:21:36Z")

</div>

Have you tried reinstalling Logstash service?

---

<div class="post-metadata">

**Author:** ![Aniket\_Chakrabarty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aniket_chakrabarty/32/131775_2.png) [@Aniket\_Chakrabarty](https://discuss.elastic.co/u/Aniket_Chakrabarty)\
**Post date:** [February 15, 2024, 12:31pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352/3 "2024-02-15T12:31:15Z")

</div>

yes, freshly installed on another VM, but the same issue persists.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [February 15, 2024, 12:47pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352/4 "2024-02-15T12:47:19Z")

</div>

> [@Aniket\_Chakrabarty](#):
>
> Using default config which logs errors to the console

What are the errors? You didn't share any log, without it is not possible to know what may be happening.

---

<div class="post-metadata">

**Author:** ![Aniket\_Chakrabarty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aniket_chakrabarty/32/131775_2.png) [@Aniket\_Chakrabarty](https://discuss.elastic.co/u/Aniket_Chakrabarty)\
**Post date:** [February 15, 2024, 3:42pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352/5 "2024-02-15T15:42:33Z")

</div>

Hi leandrojmp,

Here are the logs:

```auto
tail -f /var/log/logstash/logstash-plain.log.log
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.20.1.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?]
[2024-02-15T10:05:06,590][INFO][logstash.runner] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2024-02-15T10:05:06,599][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.17.18", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.20+8 on 11.0.20+8 +indy +jit [linux-aarch64]"}
[2024-02-15T10:05:06,601][INFO][logstash.runner] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[2024-02-15T10:05:07,830][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2024-02-15T10:05:07,833][INFO][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/*.conf"}
[2024-02-15T10:05:07,842][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.

```

and when we running it the console output is:

```auto
/usr/share/logstash/bin/logstash -f logstash.conf
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO] 2024-02-15 15:39:00.748 [main] runner - Starting Logstash {"logstash.version"=>"7.17.18", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.20+8 on 11.0.20+8 +indy +jit [linux-aarch64]"}
[INFO] 2024-02-15 15:39:00.754 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[WARN] 2024-02-15 15:39:01.006 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO] 2024-02-15 15:39:01.976 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9601, :ssl_enabled=>false}
[INFO] 2024-02-15 15:39:03.067 [Converge PipelineAction::Create<main>] Reflections - Reflections took 73 ms to scan 1 urls, producing 119 keys and 419 values 
[WARN] 2024-02-15 15:39:03.869 [Converge PipelineAction::Create<main>] plain - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[WARN] 2024-02-15 15:39:03.920 [Converge PipelineAction::Create<main>] file - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[WARN] 2024-02-15 15:39:14.056 [Converge PipelineAction::Create<main>] line - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[INFO] 2024-02-15 15:39:35.926 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/root/logstash.conf"], :thread=>"#<Thread:0x40381236 run>"}
[INFO] 2024-02-15 15:39:36.558 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>0.63}
[INFO] 2024-02-15 15:39:36.610 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
[INFO] 2024-02-15 15:39:36.649 [[main]<file] observingtail - START, creating Discoverer, Watch with file and sincedb collections
[INFO] 2024-02-15 15:39:36.672 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}

```

Thanks

---

<div class="post-metadata">

**Author:** ![ersalil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ersalil/32/131773_2.png) [@ersalil](https://discuss.elastic.co/u/ersalil)\
**Post date:** [February 15, 2024, 4:35pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352/6 "2024-02-15T16:35:28Z")

</div>

Can you please share the ls -al for the etc/logstash and ./conf.d  
Just want to know if the file listed under is config or conf, as \*.conf is the regex.

Also I found the similar ticket resolved: [Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties](https://discuss.elastic.co/t/could-not-find-log4j2-configuration-at-path-usr-share-logstash-config-log4j2-properties/111857)

If you are not version specific try installing new version for the Logstash

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [February 15, 2024, 5:38pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352/7 "2024-02-15T17:38:25Z")

</div>

Most likely you have permission issue. The "logstash" user which run the service, doesn't have rights to read conf directory or files.

If I'm not wrong, your .conf file is in the /usr/share/logstash/bin/ directory. It should be in /etc/logstash/conf.d

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 14, 2024, 5:39pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-fails-to-upload-txt-files-in-s3/353352/8 "2024-03-14T17:39:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.