Logstash is running via systemctl. It is loading the configuration file automatically after certain interval.So the Configuration file which contains an SQL query is loaded multiple times. JDBC plugin does not contain any schedule parameter.
I want my configuration file will be run only one time.(It will fetch all the data from database table and store it to elasticsearch).
What should I put in logstash.yml so that it will run once
If you want to run logstash exactly once, why use systemctl. Why not just run it on the command line?
All the pipeline are running on same configuration.
Regards
Avinaba
I agree with @Badger.
If you want the pipeline to be run once, use Logstash as a batch, not like a service always started.
1 Like
If i run from the command line logs are printing on the console.i want logs are getting printed on a log file. How to achieve this while running from console.
Regards
Avinaba
you have a few flags you can set. Like -l, --path.logs PATH Directory to write Logstash internal logs to.
you can find all of them here: https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html
there you can modify all you want.
Greetings
Malte
Need logstash config file to run with the configurations mentioned in pipeline.yml and logstash.yml
--path.settings SETTINGS_DIR
Set the directory containing the logstash.yml settings file as well as the log4j logging configuration. This can also be set through the LS_SETTINGS_DIR environment variable. The default is the config directory under Logstash home.
In this directory you have also your pipeline.yml file
1 Like
If you need both "run once" database loading and "continuous" log loading, I advice you to have 2 separate Logstash instances.
The first one is run once as a batch when you want. So when processing is finished, Logstash stops.
The second one is run as a server to continuously stream log processing.
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.