Too many socket connection to coordinating nodes?

[Travis]

Hello,

While investigating on "[main] Marking url as dead." errors, I noticed that I have lots of socket connection established from my logstash to the two coordinating nodes :

netstat -nat | grep 9200 | wc -l
93

Logstash output conf :

output {
if [identifiant] == "CUSTOMER1" {
  elasticsearch {
      ilm_enabled => true
      hosts => ["https://192.168.145.7:9200", "https://192.168.145.8:9200"]
      user => "elastic"
      password => "pasword"
      ssl => true
      cacert => "/etc/logstash/certs/ca.pem"
      index => "ims_customer1_logs"
  }
}
}


output {
if [identifiant] == "CUSTOMER2" {
  elasticsearch {
      ilm_enabled => true
      hosts => ["https://192.168.145.7:9200", "https://192.168.145.8:9200"]
      user => "elastic"
      password => "pasword"
      ssl => true
      cacert => "/etc/logstash/certs/ca.pem"
      index => "ims_customer2_logs"
  }
}
}


output {
if [identifiant] == "CUSTOMER3" {
  elasticsearch {
      ilm_enabled => true
      hosts => ["https://192.168.145.7:9200", "https://192.168.145.8:9200"]
      user => "elastic"
      password => "pasword"
      ssl => true
      cacert => "/etc/logstash/certs/ca.pem"
      index => "ims_customer3_logs"
  }
}
}


output {
if [identifiant] == "CUSTOMER4" {
  elasticsearch {
      ilm_enabled => true
      hosts => ["https://192.168.145.7:9200", "https://192.168.145.8:9200"]
      user => "elastic"
      password => "pasword"
      ssl => true
      cacert => "/etc/logstash/certs/ca.pem"
      index => "ims_customer4_logs"
  }
}
}

Do you think this is a normal behavior ? maybe this is related to the lost connection to elasticsearch errors ?

Thanks for your help !

[Travis]

Does anybody with a similar setup can check if he has the same behavior ?

Thank you !

[Travis]

@Christian_Dahlqvist @DavidTurner any chance to have a quick feedback ?

[Badger]

I cannot speak to the error, but if the only difference between the elasticsearch outputs is the index name then I would combine them...

filter {
    if [identifiant] == "CUSTOMER1" {
        mutate { add_field => { "[@metadata][index]" => "ims_customer1_logs"
    }
    if [identifiant] == "CUSTOMER2" {
        mutate { add_field => { "[@metadata][index]" => "ims_customer2_logs"
    }
    if [identifiant] == "CUSTOMER3" {
        mutate { add_field => { "[@metadata][index]" => "ims_customer3_logs"
    }
    if [identifiant] == "CUSTOMER4" {
        mutate { add_field => { "[@metadata][index]" => "ims_customer4_logs"
    }
}

Then reference that in your elasticsearch output

     index => "%{[@metadata][index]}"

More details in the best practices section of the elasticsearch output documentation.

[Travis]

Hello @Badger, yes that's a good practice indeed. I will try this and see if it's better. Thank you

[Travis]

@Badger's solution has solved the issue : from 93 connections to cooordinating nodes, I'm around 20