Reached open files limit

Elastic Stack Logstash
1

Hi, I am new to ELK stack.
I have been working on it for a while and it was working fine and suddenly the logs stopped loading into kibana dashboard and when i checked the log files in the logstash log file i am getting this error

> message=>"Reached open files limit: 4095, set by the 'max_open_files' option or default, files yet to open: 16159"

I couldn't find anything with the name 'max_open_files'

2

Check out the section on Open File Descriptors on this page:

https://www.elastic.co/guide/en/elasticsearch/reference/2.2/setup-configuration.html#file-descriptors

It is likely that you have not set a high enough value for open files on your operating system for the user that is running ELK.

On Red Hat Linux this is set by adding soft and hard values for the nofile parameter in the /etc/security/limits.conf file, e.g.

username        soft     nofile          4096
username        hard     nofile          63536
3

What's your config look like?

4

Thank you It worked for me

5

File plugin of Logstash has max_open_files parameter with default value 4095. See here https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-max_open_files

Then to increase number of input files you need to

  1. increase ulimit (operation system level) as described in one of the previous answers
  2. increase LS_OPEN_FILES on logstash config in /etc/init.d/logstash
  3. increase number of max_open_files on input file plugin level. max_open_files => 46000