Hi, logstash-filter-elasticsearch throws this error on high load:
[2017-06-27T11:58:03,714][WARN ][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=>"combine-inbound-2017.06.27", :query=>"ip:10.10.10.10", :event=>2017-06-27T09:57:40.000Z 172.16.1.4 %{message}, :error=>#<Elasticsearch::Transport::Transport::Error: Cannot get new connection from pool.>}
Logstash config:
...
elasticsearch {
hosts => ["localhost:9200"]
index => "combine-inbound-2017.06.27"
enable_sort => false
periodic_flush => true
query => "ip:%{[netflow][ipv4_src_addr]}"
fields => {"notes" => "alert"}
}
...
The server is already "tuned" for elasticsearch:
net.core.somaxconn=65535
vm.max_map_count=262144
fs.file-max=518144
elasticsearch soft nofile 65535
elasticsearch hard nofile 65535
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
As this is a setup for testing, i am running just one node.
Has anyone an idea how to solve this or what to search for?