Reindexing/Updating Elasticsearch using Logstash on Jenkins

Elastic Stack Logstash
3

@magnusbaeck Thanks for your reply. I am going to take a fresh look at this problem and do stuff from scratch. Please advice on the following :

  • What's the recommended way of running Logstash - as a service or using -e flag with the config file? We are using AWS centos box for running ElasticSearch and Logstash.

  • We would like to manage 3 environments, one for each dev, qa, prod. Each of these environment will have 2 config files, fetching data from 2 different data sources and feeding it to elasticsearch. We were thinking of running 3 different logstash processes, one for each environment. Does it sound like a good approach?

  • With above step, we would like to use scheduler that can be configured in logstash config file instead of using crontab or any other scheduler. Does the scheduler config ensure that new data gets fed into elasticsearch periodically?

Thanks