2020-09-16T01:04:38.158Z INFO instance/beat.go:640 Home path: [/usr/share/metricbeat] Config path: [/usr/share/metricbeat] Data path: [/usr/share/metricbeat/data] Logs path: [/usr/share/metricbeat/logs] 2020-09-16T01:04:38.178Z INFO instance/beat.go:648 Beat ID: 2557e296-54f1-4260-a6b3-ca5f515489c7 2020-09-16T01:04:38.179Z INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed 2020-09-16T01:04:38.179Z INFO [beat] instance/beat.go:976 Beat info {"system_info": {"beat": {"path": {"config": "/usr/share/metricbeat", "data": "/usr/share/metricbeat/data", "home": "/usr/share/metricbeat", "logs": "/usr/share/metricbeat/logs"}, "type": "metricbeat", "uuid": "2557e296-54f1-4260-a6b3-ca5f515489c7"}}} 2020-09-16T01:04:38.179Z INFO [beat] instance/beat.go:985 Build info {"system_info": {"build": {"commit": "ad823eca4cc74439d1a44351c596c12ab51054f5", "libbeat": "7.9.1", "time": "2020-09-01T20:03:55.000Z", "version": "7.9.1"}}} 2020-09-16T01:04:38.179Z INFO [beat] instance/beat.go:988 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":8,"version":"go1.14.7"}}} 2020-09-16T01:04:38.192Z INFO [beat] instance/beat.go:992 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-09-09T02:27:18Z","containerized":true,"name":"k8s-dev-node01","ip":["127.0.0.1/8","::1/128","192.168.51.113/24","fe80::250:56ff:fe90:1b0a/64","172.17.0.1/16","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","10.64.21.133/32","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64"],"kernel_version":"3.10.0-1062.18.1.el7.x86_64","mac":["00:50:56:90:1b:0a","02:42:38:40:7f:2b","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee"," ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":8,"patch":2003,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0}}} 2020-09-16T01:04:38.193Z INFO [beat] instance/beat.go:1021 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/metricbeat", "exe": "/usr/share/metricbeat/metricbeat", "name": "metricbeat", "pid": 1, "ppid": 0, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2020-09-16T01:04:37.110Z"}}} 2020-09-16T01:04:38.193Z INFO instance/beat.go:299 Setup Beat: metricbeat; Version: 7.9.1 2020-09-16T01:04:38.193Z INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'metricbeat-7.9.1' as ILM is enabled. 2020-09-16T01:04:38.193Z INFO eslegclient/connection.go:99 elasticsearch url: http://192.168.10.145:9200 2020-09-16T01:04:38.194Z INFO [publisher] pipeline/module.go:113 Beat name: k8s-dev-node01 2020-09-16T01:04:38.197Z INFO add_kubernetes_metadata/kubernetes.go:71 add_kubernetes_metadata: kubernetes env detected, with version: v1.17.4 2020-09-16T01:04:38.197Z INFO [kubernetes] kubernetes/util.go:99 kubernetes: Using node k8s-dev-node01 provided in the config {"libbeat.processor": "add_kubernetes_metadata"} 2020-09-16T01:04:38.224Z INFO [autodiscover.pod] kubernetes/util.go:114 kubernetes: Using pod name k8s-dev-node01 and namespace kube-system to discover kubernetes node 2020-09-16T01:04:38.242Z ERROR [autodiscover.pod] kubernetes/util.go:117 kubernetes: Querying for pod failed with error: pods "k8s-dev-node01" not found 2020-09-16T01:04:38.242Z INFO [monitoring] log/log.go:118 Starting metrics logging every 30s 2020-09-16T01:04:38.242Z INFO instance/beat.go:450 metricbeat start running. 2020-09-16T01:04:38.261Z INFO [autodiscover] autodiscover/autodiscover.go:113 Starting autodiscover manager 2020-09-16T01:04:38.261Z INFO cfgfile/reload.go:164 Config reloader started 2020-09-16T01:04:38.275Z INFO cfgfile/reload.go:224 Loading of config files completed. 2020-09-16T01:04:39.663Z INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to backoff(elasticsearch(http://192.168.10.145:9200)) 2020-09-16T01:04:39.663Z INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer 2020-09-16T01:04:39.664Z INFO [publisher] pipeline/retry.go:223 done 2020-09-16T01:04:39.681Z INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.9.1 2020-09-16T01:04:39.714Z INFO [license] licenser/es_callback.go:51 Elasticsearch license: Basic 2020-09-16T01:04:39.727Z INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.9.1 2020-09-16T01:04:39.758Z INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success. 2020-09-16T01:04:39.768Z INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false 2020-09-16T01:04:39.768Z INFO [index-management] idxmgmt/std.go:274 ILM policy successfully loaded. 2020-09-16T01:04:39.768Z INFO [index-management] idxmgmt/std.go:407 Set setup.template.name to '{metricbeat-7.9.1 {now/d}-000001}' as ILM is enabled. 2020-09-16T01:04:39.768Z INFO [index-management] idxmgmt/std.go:412 Set setup.template.pattern to 'metricbeat-7.9.1-*' as ILM is enabled. 2020-09-16T01:04:39.768Z INFO [index-management] idxmgmt/std.go:446 Set settings.index.lifecycle.rollover_alias in template to {metricbeat-7.9.1 {now/d}-000001} as ILM is enabled. 2020-09-16T01:04:39.768Z INFO [index-management] idxmgmt/std.go:450 Set settings.index.lifecycle.name in template to {metricbeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled. 2020-09-16T01:04:39.779Z INFO template/load.go:89 Template metricbeat-7.9.1 already exists and will not be overwritten. 2020-09-16T01:04:39.780Z INFO [index-management] idxmgmt/std.go:298 Loaded index template. 2020-09-16T01:04:39.788Z INFO [index-management] idxmgmt/std.go:309 Write alias successfully generated. 2020-09-16T01:04:39.796Z INFO [publisher_pipeline_output] pipeline/output.go:151 Connection to backoff(elasticsearch(http://192.168.10.145:9200)) established 2020-09-16T01:04:42.224Z WARN [elasticsearch] elasticsearch/client.go:407 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbfd077099075516c, ext:232842531, loc:(*time.Location)(0x7f53700)}, Meta:null, Fields:{"agent":{"ephemeral_id":"a0278998-7c57-4883-873a-4881ad87e6e1","hostname":"k8s-dev-node01","id":"2557e296-54f1-4260-a6b3-ca5f515489c7","name":"k8s-dev-node01","type":"metricbeat","version":"7.9.1"},"ecs":{"version":"1.5.0"},"event":{"dataset":"kubernetes.service","duration":383203478,"module":"kubernetes"},"host":{"name":"k8s-dev-node01"},"kubernetes":{"labels":{"app_kubernetes_io_name":"node-exporter","app_kubernetes_io_version":"v0.18.1"},"namespace":"monitoring","service":{"cluster_ip":"None","created":"2020-07-30T07:36:50.000Z","name":"node-exporter","type":"ClusterIP"}},"metricset":{"name":"state_service","period":10000},"service":{"address":"kube-state-metrics:8080","type":"kubernetes"}}, Private:interface {}(nil), TimeSeries:true}, Flags:0x0, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [kubernetes.service.cluster_ip] of type [ip] in document with id 'GrhxlHQBui0kzgpyL7SF'. Preview of field's value: 'None'","caused_by":{"type":"illegal_argument_exception","reason":"'None' is not an IP string literal."}} 2020-09-16T01:04:42.224Z WARN [elasticsearch] elasticsearch/client.go:407 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbfd077099075516c, ext:232842531, loc:(*time.Location)(0x7f53700)}, Meta:null, Fields:{"agent":{"ephemeral_id":"a0278998-7c57-4883-873a-4881ad87e6e1","hostname":"k8s-dev-node01","id":"2557e296-54f1-4260-a6b3-ca5f515489c7","name":"k8s-dev-node01","type":"metricbeat","version":"7.9.1"},"ecs":{"version":"1.5.0"},"event":{"dataset":"kubernetes.service","duration":383418425,"module":"kubernetes"},"host":{"name":"k8s-dev-node01"},"kubernetes":{"labels":{"k8s_app":"kubelet"},"namespace":"kube-system","service":{"cluster_ip":"None","created":"2020-07-29T02:07:53.000Z","name":"kubelet","type":"ClusterIP"}},"metricset":{"name":"state_service","period":10000},"service":{"address":"kube-state-metrics:8080","type":"kubernetes"}}, Private:interface {}(nil), TimeSeries:true}, Flags:0x0, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [kubernetes.service.cluster_ip] of type [ip] in document with id 'H7hxlHQBui0kzgpyL7SF'. Preview of field's value: 'None'","caused_by":{"type":"illegal_argument_exception","reason":"'None' is not an IP string literal."}}
I see that this problem exists in early versions of metribeat, but I am using version 7.9.1 now. Why is this problem still present?
7.10 and 8.0 have not been released yet.
As stated in https://github.com/elastic/beats/issues/17447#issuecomment-673983218 it will part of the 7.10 release. You can try to run and build the master release or wait for the official release or try with snapshot: docker.elastic.co/beats/metricbeat:7.10.0-SNAPSHOT.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.