I configured metricbeat to send system metrics to elasticsearch. I'm using the exact same configuration on 13 Servers but I'm seeing only data from 12 Servers.
Metricbeat Log from the missing server shows no Errors:
./metricbeat -e -d "*" -c "/etc//metricbeat/metricbeat.yml" -v
2018-05-16T14:14:55.116+0200 INFO instance/beat.go:468 Home path: [/usr/share/metricbeat/bin] Config path: [/usr/share/metricbeat/bin] Data path: [/usr/share/metricbeat/bin/data] Logs path: [/usr/share/metricbeat/bin/logs]
2018-05-16T14:14:55.116+0200 DEBUG [beat] instance/beat.go:495 Beat metadata path: /usr/share/metricbeat/bin/data/meta.json
2018-05-16T14:14:55.116+0200 INFO instance/beat.go:475 Beat UUID: 18e77707-04f9-4fdd-b327-f52db12c308f
2018-05-16T14:14:55.116+0200 INFO instance/beat.go:213 Setup Beat: metricbeat; Version: 6.2.4
2018-05-16T14:14:55.116+0200 DEBUG [beat] instance/beat.go:230 Initializing output plugins
2018-05-16T14:14:55.116+0200 DEBUG [processors] processors/processor.go:49 Processors:
2018-05-16T14:14:55.117+0200 INFO elasticsearch/client.go:145 Elasticsearch url: http://itoclmelk3p.int.tds.rz:9200
2018-05-16T14:14:55.119+0200 INFO pipeline/module.go:76 Beat name: tdsuc4oxy1vp
2018-05-16T14:14:55.119+0200 INFO instance/beat.go:301 metricbeat start running.
2018-05-16T14:14:55.120+0200 WARN [cfgwarn] cfgfile/reload.go:77 BETA: Dynamic config reload is enabled.
2018-05-16T14:14:55.120+0200 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-05-16T14:14:55.120+0200 INFO cfgfile/reload.go:127 Config reloader started
2018-05-16T14:15:00.120+0200 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-05-16T14:15:00.120+0200 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
2018-05-16T14:15:05.120+0200 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-05-16T14:15:05.120+0200 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
2018-05-16T14:15:10.120+0200 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-05-16T14:15:10.120+0200 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
2018-05-16T14:15:15.121+0200 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-05-16T14:15:15.121+0200 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
2018-05-16T14:15:20.121+0200 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-05-16T14:15:20.121+0200 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
2018-05-16T14:15:25.121+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":0,"time":4},"total":{"ticks":20,"time":24,"value":20},"user":{"ticks":20,"time":20}},"info":{"ephemeral_id":"b1309959-70d1-4ce3-8e4f-150c5d79e8eb","uptime":{"ms":30019}},"memstats":{"gc_next":4194304,"memory_alloc":1977824,"memory_total":3867096,"rss":17309696}},"libbeat":{"config":{"module":{"running":0},"reloads":5},"output":{"type":"elasticsearch"},"pipeline":{"clients":0,"events":{"active":0}}},"system":{"cpu":{"cores":1},"load":{"1":0.02,"15":0.06,"5":0.06,"norm":{"1":0.02,"15":0.06,"5":0.06}}}}}}
2018-05-16T14:15:25.121+0200 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-05-16T14:15:25.121+0200 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
2018-05-16T14:15:30.121+0200 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-05-16T14:15:30.122+0200 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
2018-05-16T14:15:35.122+0200 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-05-16T14:15:35.122+0200 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
2018-05-16T14:15:40.122+0200 DEBUG [cfgfile] cfgfile/reload.go:151 Scan for new config files
2018-05-16T14:15:40.122+0200 DEBUG [cfgfile] cfgfile/reload.go:170 Number of module configs found: 0
^C2018-05-16T14:15:42.440+0200 DEBUG [service] service/service.go:33 Received sigterm/sigint, stopping
2018-05-16T14:15:42.440+0200 INFO cfgfile/reload.go:147 Dynamic config reloader stopped
2018-05-16T14:15:42.440+0200 INFO instance/beat.go:308 metricbeat stopped.
2018-05-16T14:15:42.441+0200 INFO [monitoring] log/log.go:132 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":0,"time":4},"total":{"ticks":20,"time":24,"value":20},"user":{"ticks":20,"time":20}},"info":{"ephemeral_id":"b1309959-70d1-4ce3-8e4f-150c5d79e8eb","uptime":{"ms":47338}},"memstats":{"gc_next":4194304,"memory_alloc":2191248,"memory_total":4080520,"rss":17571840}},"libbeat":{"config":{"module":{"running":0},"reloads":9},"output":{"type":"elasticsearch"},"pipeline":{"clients":0,"events":{"active":0}}},"system":{"cpu":{"cores":1},"load":{"1":0.02,"15":0.05,"5":0.06,"norm":{"1":0.02,"15":0.05,"5":0.06}}}}}}
2018-05-16T14:15:42.441+0200 INFO [monitoring] log/log.go:133 Uptime: 47.339377015s
2018-05-16T14:15:42.441+0200 INFO [monitoring] log/log.go:110 Stopping metrics logging.
metricbeat.yml has only one output configured:
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["itoclmelk3p.int.tds.rz:9200"]
Any idea why the sent data isnt showing up in elasticsearch?