Hello World!
I'm using Elastic stack 6.6.1 and my metricbeat worked fine for a while, then it stopped on it's own and I'm seeing following error in /var/log/metricbeat/metricbeat:
2019-02-16T04:36:05.176Z WARN transport/tcp.go:53 DNS lookup failure "X.X.X": lookup X.X.X on [::1]:53: dial udp [::1]:53: socket: too many open files
2019-02-16T04:36:05.178Z ERROR [centralmgmt] management/manager.go:193 error retrieving new configurations, will use cached ones: fail to execute the HTTP GET request: Get https://X.X.X:443/api/beats/agent/62e347c7-09e5-4fbf-b131-23f158f8d8a3/configuration?validSetting=true: lookup X.X.X on [::1]:53: dial udp [::1]:53: socket: too many open files
2019-02-16T04:36:25.575Z ERROR instance/metrics.go:91 Error while getting memory usage: error retrieving process stats: cannot find matching process for pid=7959
2019-02-16T04:36:25.584Z ERROR instance/metrics.go:135 Error retrieving CPU percentages: error retrieving process stats: cannot find matching process for pid=7959
2019-02-16T04:36:25.584Z ERROR instance/metrics_file_descriptors.go:39 Error while retrieving FD information: error retrieving process stats: cannot find matching process for pid=7959
I've used: systemctl restart metricbeat to resolve issue, however this isn't the first time it happens...
I'm using Google's DNS, pretty sure they're up)
# ulimit
unlimited
#
Please advise.