# Help with Logstash plug-in development

**URL:** <https://discuss.elastic.co/t/help-with-logstash-plug-in-development/236937>\
**Category:** Logstash\
**Created:** [June 12, 2020, 7:23pm UTC](https://discuss.elastic.co/t/help-with-logstash-plug-in-development/236937 "2020-06-12T19:23:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![byoungman](https://avatars.discourse-cdn.com/v4/letter/b/838e76/32.png) [@byoungman](https://discuss.elastic.co/u/byoungman)\
**Post date:** [June 12, 2020, 7:23pm UTC](https://discuss.elastic.co/t/help-with-logstash-plug-in-development/236937/1 "2020-06-12T19:23:13Z")

</div>

I am trying to develop a new logstash plugin in my development Elasticsearch stack environment. I am new to this so this is all a learning experience for me. I will try to be as clear and concise as I can be including as much information and code snippets as I can so here goes.

I started with generating the skeleton for my new plug-in using the logstash-plugin generate utility. Once I had this in place I then was able to build my plug-in using the gem build utility which ran without error and created a gem for me. I then ran the logstash-plugin install utility which successfully installed my plug-in.

The entry in my logstash.gem looks like this--  
gem "logstash-output-s3-4.3.3", "0.1.0", :path =\> "./vendor/bundle/jruby/2.5.0/gems/logstash-output-s3-4.3.3"

I then took the code from the actual logstash plug-in that I'm hoping to modify and copied it into my new plug-in directory and then created a new logstash.conf file to test with using my new plug-in and started the logstash service. It ran as expected moving data from the servers that were being watched in the appropriate s3 buckets.

logstash.conf output section--

output {  
s3{  
region =\> "us-east-1"  
bucket =\> "vertexsmb-osplogs-lower"  
prefix =\> "Dev/Request-Logs/us-east-1/%{+YYYY}/%{+MM}/%{+dd}/"  
server\_side\_encryption =\> true  
server\_side\_encryption\_algorithm =\> "aws:kms"  
ssekms\_key\_id =\> "arn:aws:kms:us-east-1:409148389496:alias/LowerEnvironmentS3"  
validate\_credentials\_on\_root\_bucket =\> false  
codec =\> "json\_lines"  
size\_file =\> 1024000  
time\_file =\> 5  
rotation\_strategy =\> "size\_and\_time"  
temporary\_directory =\> "../../LogstashS3OutputData/temp/"  
canned\_acl =\> "private"   
}

```
s3{
	region => "us-east-1"
	bucket => "vertexsmb-osplogs-lower"
	prefix => "Dev/Response-Logs/us-east-1/%{+YYYY}/%{+MM}/%{+dd}/"
	server_side_encryption => true
	server_side_encryption_algorithm => "aws:kms"
	ssekms_key_id => "arn:aws:kms:us-east-1:409148389496:alias/LowerEnvironmentS3"
	validate_credentials_on_root_bucket => false
	codec => "json_lines"
	size_file => 1024000
	time_file => 5
	rotation_strategy => "size_and_time"
	temporary_directory => "../../LogstashS3OutputData/temp/"
	canned_acl => "private"

```

}  
}

My next step was and this is where I am having issues was to try and create a new property for the plug-in called file\_prefix--

```
s3{
	region => "us-east-1"
	bucket => "vertexsmb-osplogs-lower"
	prefix => "Dev/Request-Logs/us-east-1/%{+YYYY}/%{+MM}/%{+dd}/"

```

=\> file\_prefix =\> "test"  
server\_side\_encryption =\> true  
server\_side\_encryption\_algorithm =\> "aws:kms"  
ssekms\_key\_id =\> "arn:aws:kms:us-east-1:409148389496:alias/LowerEnvironmentS3"  
validate\_credentials\_on\_root\_bucket =\> false  
codec =\> "json\_lines"  
size\_file =\> 1024000  
time\_file =\> 5  
rotation\_strategy =\> "size\_and\_time"  
temporary\_directory =\> "../../LogstashS3OutputData/temp/"  
canned\_acl =\> "private"   
}

I modified the following files in my solution  
\vendor\bundle\jruby\2.5.0\gems\logstash-output-s3-4.3.3\spec\outputs\s3\_spec.rb

- added "file\_prefix" =\> file\_prefix at line 15

\vendor\bundle\jruby\2.5.0\gems\logstash-output-s3-4.3.3\spec\supports\helpers.rb

- added let(:file\_prefix) {} at line line 12
- added "file\_prefix" =\> file\_prefix, at line 19

\vendor\bundle\jruby\2.5.0\gems\logstash-output-s3-4.3.3\lib\logstash\outputs\s3.rb

- added config :file\_prefix, :validate =\> :string right after the statement config :prefix, :validate =\> :string, :default =\> '' at line 154
- added @file\_prefix = file\_prefix at line 226

Now when I start the logstash service I get this error message  
Unknown setting 'file\_prefix' for s3

I've searched and I've tried everything I can think but I can't get past this hurdle.

TIA,  
Bill Youngman

---

<div class="post-metadata">

**Author:** ![byoungman](https://avatars.discourse-cdn.com/v4/letter/b/838e76/32.png) [@byoungman](https://discuss.elastic.co/u/byoungman)\
**Post date:** [June 18, 2020, 1:15pm UTC](https://discuss.elastic.co/t/help-with-logstash-plug-in-development/236937/2 "2020-06-18T13:15:24Z")

</div>

This can be considered closed as I am using the Feature Request #215 ( Added `filename` as a configuration option.) for the s3 Output plugin as the starting point for my project and so far haven't run into any problems.

-b

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 16, 2020, 1:15pm UTC](https://discuss.elastic.co/t/help-with-logstash-plug-in-development/236937/3 "2020-07-16T13:15:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.