How to keep one exec plugin waiting for another?

Given the following configuration:

input {
  exec {
    command => "python3 ${LOGSTASH_HOME}/script/firmware/backend/"
    schedule => "${SCHEDULER}"
    type => "Backend"
  exec {
    command => "python2 ${LOGSTASH_HOME}/script/firmware/error_parser/parsers/log_parser/ -pp true -lf ${LOGSTASH_HOME}/firmware_data"
    schedule => "${FIRMWARE_SCHEDULER}"
    type => "Firmware"
  file {
	 path => "${LOGSTASH_HOME}/firmware_data/errorcode_diagnostics/*.diag"
	 codec => json
	 start_position => "beginning"
	 sincedb_path => "${LOGSTASH_HOME}/file/.sincedb*"
	 type => "S3"

Is there any way I can tell the exec input plugin to wait until command => "python3 ${LOGSTASH_HOME}/script/..." is done before it starts executing command => "python2 ${LOGSTASH_HOME}/script/firmware/error_parser/..."?

Note: The second exec command requires the first exec command to be finished first!

Possible solution 1: Set the time interval for the second command anywhere from 5-10m after the first command to ensure it is processed successfully before it starts.

Problem: This approach can sometimes fail if there is a lot of data being processed by the first command, so while the first command is still in process, the second command can kick in and fail.

Possible solution 2: Run the second command in the filter plugin.

Problem: This approach doesn't generate any extra fields to use after in the filter plugin or output plugin, only running the script in the input plugin does this.

There nothing in logstash that will help you do that. You would have to solve it in the Python scripts.

Yes, but I'm not the owner of the second script, so I'm not allowed to make a change.

Solution 2 that I suggested can work, but it doesn't generate any fields such as [message]. Do you know any way that I can get those fields as a result of running the second script in the filter plugin?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.