Pipeline disabling

Elastic Stack Logstash
1

Hello !

I'm currently working on the logstash 7.5.1.
I'm using the multiple pipelines file with the auto reload mode to create and delete pipelines when logstash is running.

I noticed that Logstash doesn't create a pipeline if I do the following steps :

  1. I add a pipeline with a specific ID
  2. I remove this pipeline
  3. I add a new pipeline with the same ID

So, I want to know if this behavior is normal or not. Maybe I missed something, so any help is welcomed :slight_smile:

2

I'm looking in the code and I find something strange. The method terminate_pipeline of the class PipelinesRegistry doesn't remove pipelines from memory.

def terminate_pipeline(pipeline_id, &stop_block)
  lock = get_lock(pipeline_id)
  lock.lock

  state = @states.get(pipeline_id)
  if state.nil?
    logger.error("Attempted to terminate a pipeline that does not exists", :pipeline_id => pipeline_id)
    @states.remove(pipeline_id)
  else
    yield(state.pipeline)
    @states.put(pipeline_id, state)
  end
ensure
  lock.unlock
end

So, Logstash always keep the pipeline in memory even if the pipeline is removed from the piplines.yml file. It's the expected behavior ?

For now, I did a patch to change the behavior of this method.