Conditional on elasticsearch output not working as expected

Elastic Stack Logstash
1

Hi folks,
I'm using the elasticsearch output plugin, and I want to define two alternatives configurations for it, depending on a certain condition (see code below).

...
output {
  if [condition] {
    elasticsearch {
      hosts => [ "condition-met-host" ]
      index => "an-index-name"
      ...
    }
  } else {
    elasticsearch {
      hosts => [ "condition-not-met-host" ]
      index => "an-index-name"
      ...
    }
  }
}

Everything seems to work as expected as long as the hosts (or proxy) property of BOTH the output plugins refers to a valid Elasticsearch host. However, in practice, Logstash attempts to establish a connection also with the host of the branch that doesn't satisfy the condition. If such a host exists, no problems, you wouldn't even notice. On the contrary, if such a host doesn't exist (and this is exactly what my condition is there for), Logstash keeps trying to resurrect the connection, preventing the pipeline to terminate:

[WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://condition-not-met-host:9200/", :error_type=>LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://condition-not-met-host:9200/][Manticore::ResolutionFailure] condition-not-met-host: Name or service not known"}

Am I doing something wrong? Am I missing something important?

  • thanks in advance
2

That's exactly what I would expect. logstash establishes a connection during initialization, and if initialization never completes then shutdown will hang.

3

Thanks for the reply. Then, the expected behaviour is to initialise all the plugins, regardless of whether they are defined in a conditional branch that will never be executed. Is that correct?

If that's the case, I'll need a different solution for my problem, maybe separated pipelines.

4

That is correct.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.