Why logstash cannot start after define new pipeline

Elastic Stack Logstash
1

I was created pipeline1 in logstash and it was working excellent but I added a new pipeline now when I want to start logstash it shows follow error and cannot start

icsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>2}
[2023-11-19T16:31:32,536][ERROR][logstash.javapipeline    ][pipeline1] Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"pipeline1", :error=>"", :exception=>Java::OrgJrubyExceptions::ThreadKill, :backtrace=>[], :thread=>"#<Thread:0x451e6af5 /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-11-19T16:31:32,641][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>2}
[2023-11-19T16:31:34,162][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>4}
[2023-11-19T16:31:34,220][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>4}
[2023-11-19T16:31:34,303][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>4}
[2023-11-19T16:31:34,476][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>4}
[2023-11-19T16:31:34,546][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>4}
[2023-11-19T16:31:34,648][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>4}
[2023-11-19T16:31:38,172][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>8}
[2023-11-19T16:31:38,229][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>8}
[2023-11-19T16:31:38,310][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>8}
[2023-11-19T16:31:38,485][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>8}
[2023-11-19T16:31:38,555][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>8}
[2023-11-19T16:31:38,656][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>8}
[2023-11-19T16:31:46,183][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>16}
[2023-11-19T16:31:46,262][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>16}
[2023-11-19T16:31:46,316][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>16}
[2023-11-19T16:31:46,492][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>16}
[2023-11-19T16:31:46,563][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>16}
[2023-11-19T16:31:46,665][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>16}
[2023-11-19T16:32:02,193][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>32}
[2023-11-19T16:32:02,269][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>32}
[2023-11-19T16:32:02,324][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>32}
[2023-11-19T16:32:02,501][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>32}
[2023-11-19T16:32:02,572][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>32}
[2023-11-19T16:32:02,675][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>32}
[2023-11-19T16:32:34,202][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:32:34,278][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:32:34,334][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:32:34,508][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:32:34,579][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:32:34,683][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:33:38,224][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:33:38,288][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:33:38,342][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:33:38,517][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:33:38,601][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}
[2023-11-19T16:33:38,695][ERROR][logstash.outputs.elasticsearch][pipeline1][1d705bebc465c6f2954dfc0731576a2b1c889f4746ae5d1dec61206e4d3ddf62] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>64}

but when comment one of the pipeline logstash can start and work excellent

This is my pipeline.yml file before comment one of the pipelines :

- pipeline.id: pipeline1
  path.config: "/etc/logstash/conf.d/pipeline1.conf"

- pipeline.id: pipeline2
  path.config: "/etc/logstash/conf.d/pipeline2.conf"
2

This error says that the elasticsearch output in your pipeline1 cannot connect to Elasticsearch as there are no avaiable hosts.

You need to check it.

You have two pipelines, which one you comment and Logstash works without any issue?

What does your configurationlooks like? Please share your configurations for both pipelines.

3

There is no difference it just cannot work with 2 pipeline simultaneously.

pipeline1:

input{
   beats {

    port => 5070

}

}


filter{

  grok{
    patterns_dir  => ["/etc/logstash/conf.d/testeod"]
    match  =>  { "message" => "%{custom_exception:dmerror}" }

}

}



output{

  stdout{}
  elasticsearch {
   index => eodtest
   hosts => ["https://IP:9200"]
   cacert => '/etc/logstash/certs/http_ca.crt'
   user => "elastic"
   password => "password"

}

}

testeod

custom_exception %{WORD:exception1} %{WORD:exception1} %{WORD:exception1} %{WORD:exception1} %{WORD:exception1} %{WORD:exception1} %{WORD:exception1} %{WORD:exception1}

pipeline2:

input{
   beats {

    port => 5070

}

}


filter{

  grok{
    patterns_dir  => ["/etc/logstash/conf.d/open"]
    match  =>  { "message" => "%{custom_open:response_time}" }

}

}



output{

  stdout{}
  elasticsearch {
   index => newopenbank
   hosts => ["https://IP:9200"]
   cacert => '/etc/logstash/certs/http_ca.crt'
   user => "elastic"
   password => "password"

}

}

open

custom_open %{COMMONAPACHELOG:apache}
4

You are using the same port in both pipelines, this is not possible, you need to use a different port.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.