Hanging shutdown with multiple pipelines. Shutdown of pipelines in wrong order

[asp]

Hi,

I have a logstash config with multiple pipelines. One of the pipeline is a central output which ships to elasticsearch. The other pipelines have their own inputs and filters and having that central output pipeline as output.

I noticed a hanging shutdown because the output pipeline was shut down before the other pipelines. My only solution was to kill the process. I don't want to lose log messages.

Is there a way to configure a pipeline as start first, stop last?
Currently I am using a memory queue. Would a persistent queue help?

Thanks, Andreas

[Badger]

Are you using pipeline-to-pipeline communications? If so, the pipeline shutdown calls should be ordered. If you are using (for example) tcp-to-tcp then pipeline shutdown order is not guaranteed.

[asp]

Thanks for the reply.
I am using pipeline to pipeline configuration.

The output of the parsing pipelines looks like the following:

output
{
	pipeline
	{
		send_to => [commonOut]
 	}
}

The pipeline which sends to elasticsearch is using this configuration:

input
{
  # read from logstash pipeline commonOut
  pipeline
  {
    address => commonOut
  }
}
# encrypted output with TLS and authorization
output
{
	elasticsearch
	{
		hosts 		=> ["${ES_HOSTS}"]
		ssl 			=> "${USE_ES_SSL}"
		cacert		=> "${ES_CA_CERT_PATH}"
		ssl_certificate_verification	=>	"${USE_ES_OUTPUT_SSL_CERT_VERIFICATION}"

		# credentials are fetched from envrionment or logstash-keystore

		user			=> "${LOGSTASH_USER}"
		password	=> "${LOGSTASH_PASSWORD}"

		index			=> "%{[@metadata][indexName]}"
	}
}

This is the relevant part of the pipelines.yml:

# new json format. Contains all logTypes
- pipeline.id: generic_json
  pipeline.workers: 8
  queue.type: memory
  path.config: "/usr/share/logstash/pipelines/generic_json"

- pipeline.id: commonOutElasticsearch
  pipeline.workers: 8
  queue.type: memory
  path.config: "/usr/share/logstash/pipelines/output_es_common_https"