Logstash save/ modify configuration in environment


(Manoj Hettiarachchi) #1

Hello,

In my system, I use logstash, filebeat and elasticsearch
Filebeat reads the logs, required fields in the logs are filtered with logstash and saved in elasticsearch.

I have a customer requirement to switch on/off saving some fields in the log by a single config change by the customer.

My planned approach is to keep the switch variable as an environment variable in "/etc/default/logstash" location and let the customer change the variables with a file operation.

But I have found out that the logtash config is not reloaded when we change that file even if we set the "config.reload.automatic: true". So I cannot continue my planned approach.
Also letting customer edit the logstast ".conf" files is not a good approach either because the code is so complex.

Please advice on this issue.

Thanks,


(Manoj Hettiarachchi) #2

I have found a alternative approach to complete this requirement.
Instead of using the environment variables I have used a text file to save my switch variables.
Then used a ruby code to read file and add the variable values to the event.

ruby {
code => "event.set( 'variable1',IO.readlines('/etc/logstash/input.txt')[0])"
}

This has fixed my problem. But I would like to know is there a performance impact in executing file operation in each event