I have a logstash csv output report generation config, that I run as a separate logstash instance as and when it is required. I have many csv report to be generated and for each, I have one config file. Everytime I run the logstash instance, it takes its own time to start and generates report and shuts down for each csv report. My ask is, is there a way to keep a separate instance running and only trigger as many csv generator config, so as to minimize the time to start the logstash instance? TIA
example logstash config follows here.
``Preformatted text`
input {
Elasticsearch {
hosts => "localhost:9200"
index => "test-log_2021-11-17"
query => '
{
"query": {
"bool": {
"must": [
{
"exists": {
"field": "respCode"
}
}
]
'
}
}
output {
csv {
fields => ["uuid", "gtid", "respTxURI", "txTime", "respCode", "dataSize", "userId"]
path => "/tmp/csv_report_1.csv"
}
}
`
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/report_1.conf Preformatted text (I have 100s of reports to be generated)