How to increase the number of filebeat servers in stages

[its-ogawa]

We are currently trying to increase the number of servers we log using filebeat and logstash in stages.

Filebeat can be installed on a new logging server, and logstash can send logs to the installed server.
logstash is already installed on the server, we can send the logs to it.
What is the recommended procedure for a logstash that is already running?

Just editing the logstash config file will not enable the newly added filebeat.
I ran logstash again, specifying the edited config file.

/usr/share/logstash/bin/logstash -f /etc/logstash/logstash-sample.conf &

At this time, I get the message "logstash is already running", so the logstash with the added configuration will not simply run.
I've tried stopping logstash, killing the process, and then running it again, but
I don't think this is a good idea.

Please let me know what I can do.

[warkolm (Mark Walkom)]

Can you elaborate more on why you need to restart Logstash to enable Filebeat?

[its-ogawa]

I felt that logstash was not able to reflect the new settings.

I wrote the configuration for the newly added filebeat logs in the logstash configuration file that was already running and saved it.

By itself, it did not reflect the new settings and I could not get any logs.

Is it not also necessary to restart logstash?

[warkolm (Mark Walkom)]

What config changes did you make?

[its-ogawa]

This is the addition of the index for sending to elasticsearch.
Specifically, I added the following description.

output {
  ... snip ...
  else if( [tags][0] == "cas" or "cas" in [log][file][path] ) {
    elasticsearch {
      hosts => ["localhost"]
      index => "cas-error-%{+YYYY.MM.dd}"
    }
  }
  ... snip ...
}

We plan to add more filter descriptions, but at this stage we have not yet done so.

[warkolm (Mark Walkom)]

Righto, ok then.
You might want to look at the -r flag Running Logstash from the Command Line | Logstash Reference [7.12] | Elastic.

[its-ogawa]

Does your suggestion indicate that if I run the following command line, the edits to the config file will be reflected in logstash after it is saved?

/usr/share/logstash/bin/logstash -f /etc/logstash/logstash-sample.conf --config.reload.automatic &

If so, that's what I'm hoping for!

[its-ogawa]

How can I add the --config.reload.automatic option to a logstash that is already running?

[warkolm (Mark Walkom)]

You need to stop it and rerun it with that flag.