Address already in use - Logstash tried to bind to port range 9600..9700, but all the ports are already in use

Version:5.2.0
OS:CentOS6.5

The OS restarts every day so I have to restart the logstash every day.The logstash was running normally, but it was suddenly unable to start today.
It cannot work after I change the http.port to 9700-9800.

This is the log:

[gold@goldencis ~]$ /usr/local/logstash-5.2.0/bin/logstash -f /usr/local/logstash-5.2.0/A_r.conf
Sending Logstash's logs to /usr/local/logstash-5.2.0/logs which is now configured via log4j2.properties
[2017-03-13T15:36:11,452][INFO ][logstash.inputs.stdin    ] Automatically switching from plain to line codec {:plugin=>"stdin"}
[2017-03-13T15:36:11,460][INFO ][logstash.inputs.redis    ] Registering Redis 
...
The stdin plugin is now waiting for input:
[2017-03-13T15:36:11,665][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://10.10.9.16:9200/]}}
[2017-03-13T15:36:11,665][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://10.10.9.16:9200/, :path=>"/"}
[2017-03-13T15:36:11,734][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x1572e889 URL:http://10.10.9.16:9200/>}
[2017-03-13T15:36:11,735][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-03-13T15:36:11,773][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2017-03-13T15:36:11,777][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0x77a729f9 URL://10.10.9.16:9200>]}
[2017-03-13T15:36:11,786][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://10.10.9.16:9200/]}}
[2017-03-13T15:36:11,786][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://10.10.9.16:9200/, :path=>"/"}
[2017-03-13T15:36:11,789][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x655e6e9d URL:http://10.10.9.16:9200/>}
[2017-03-13T15:36:11,790][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-03-13T15:36:11,795][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2017-03-13T15:36:11,799][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0x7e6147fc URL://10.10.9.16:9200>]}
[2017-03-13T15:36:11,808][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://10.10.9.16:9200/]}}
[2017-03-13T15:36:11,810][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://10.10.9.16:9200/, :path=>"/"}
[2017-03-13T15:36:11,813][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x1b4ca0cf URL:http://10.10.9.16:9200/>}
[2017-03-13T15:36:11,814][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-03-13T15:36:11,819][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2017-03-13T15:36:11,823][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0x4ec7430b URL://10.10.9.16:9200>]}
[2017-03-13T15:36:11,825][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>250, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000}
[2017-03-13T15:36:11,835][INFO ][logstash.pipeline        ] Pipeline main started
[2017-03-13T15:36:12,230][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<Errno::EADDRINUSE: Address already in use - Logstash tried to bind to port range 9600..9700, but all the ports are already in use. You can specify a new port by launching logtash with the --http.port option.">, :backtrace=>["/usr/local/logstash-5.2.0/logstash-core/lib/logstash/webserver.rb:52:in `run'", "org/jruby/RubyRange.java:479:in `each'", "org/jruby/RubyEnumerable.java:974:in `each_with_index'", "/usr/local/logstash-5.2.0/logstash-core/lib/logstash/webserver.rb:39:in `run'", "/usr/local/logstash-5.2.0/logstash-core/lib/logstash/agent.rb:198:in `start_webserver'"]}

The problem is solved.The network has changed during last weekend and I changed the IP of the server then it works.
But how can I closed the topic?

1 Like

Just mark it as solved :slight_smile:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.