Newbie in Logstash need suggestion on setting up Logstash on GCP


I have just started using Logstash, and need some guidelines and best practices of running a Logstash pipeline on production.
I am familiar with the pipeline.yml and logstash.yml and also have written a basic configuration file. To connect Kafka and Elasticsearch and have explored various configuration options.

I need help with the below question:

  1. Where do you actually keep the config files?
    a. Should they be present in Github and the pipeline.yml should read from there or
    directly on the machine where the logstash is running.
    b. Or use docker and make it a part of the image.
    c. Any other suggestions
  2. Are environment variables the best option to store values like,
    a. Kafka/Elasticsearch connection string
    b. Topic, Index names

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.