Metricbeat setup to return localhost metrics

Running 7.11 Elastic on self managed eks 1.19 with Metricbeats 7.9 running on EC2 instance

This is the below configuration of the metricbeat.yaml

metricbeat.modules:
- module: system
  metricsets:
   # - cpu             # CPU usage
    - cpu             # CPU usage
    - filesystem     # filesystem averages
   
- module: http
  metricsets: ["server"]
  period: 10s
  host: ["localhost"]
  port: 8080
  enabled: true

Iam not able to get the http module working. Plan is to capture the response code as per this documentation "HTTP fields | Metricbeat Reference [master] | Elastic" and
HTTP server metricset | Metricbeat Reference [master] | Elastic

The metricbeat is failing to start and Iam not able to get the right config to return the response code. Can someone please help

Please share your Metricbeat logs :slight_smile:

Also please also consider upgrading, 7.15 is latest and you are running slightly older versions.

2021-11-19T22:37:36.716Z	INFO	instance/beat.go:665	Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]
2021-11-19T22:37:36.717Z	DEBUG	[beat]	instance/beat.go:723	Beat metadata path: /var/lib/metricbeat/meta.json
2021-11-19T22:37:36.717Z	INFO	instance/beat.go:673	Beat ID: b7c91577-81b9-4a1d-a035-0fa2eba5ea93
2021-11-19T22:37:36.717Z	DEBUG	[add_cloud_metadata]	add_cloud_metadata/providers.go:128	add_cloud_metadata: starting to fetch metadata, timeout=3s
2021-11-19T22:37:36.719Z	DEBUG	[add_cloud_metadata]	add_cloud_metadata/providers.go:165	add_cloud_metadata: received disposition for digitalocean after 1.08787ms. result=[provider:digitalocean, error=failed with http status code 404, metadata={}]
2021-11-19T22:37:36.719Z	DEBUG	[add_cloud_metadata]	add_cloud_metadata/providers.go:165	add_cloud_metadata: received disposition for azure after 1.151991ms. result=[provider:azure, error=failed with http status code 404, metadata={}]
2021-11-19T22:37:36.719Z	DEBUG	[add_cloud_metadata]	add_cloud_metadata/providers.go:165	add_cloud_metadata: received disposition for gcp after 1.95014ms. result=[provider:gcp, error=failed with http status code 404, metadata={}]
2021-11-19T22:37:36.720Z	DEBUG	[add_cloud_metadata]	add_cloud_metadata/providers.go:165	add_cloud_metadata: received disposition for aws after 2.13501ms. result=[provider:aws, error=<nil>, metadata={"cloud":{"account":{"id":"622494574705"},"availability_zone":"us-east-1a","image":{"id":"ami-00521671dfc1e0100"},"instance":{"id":"i-0921bed04d4c5ab67"},"machine":{"type":"r5.large"},"provider":"aws","region":"us-east-1","service":{"name":"EC2"}}}]
2021-11-19T22:37:36.720Z	DEBUG	[add_cloud_metadata]	add_cloud_metadata/providers.go:131	add_cloud_metadata: fetchMetadata ran for 2.247235ms
2021-11-19T22:37:36.720Z	INFO	[add_cloud_metadata]	add_cloud_metadata/add_cloud_metadata.go:105	add_cloud_metadata: hosting provider type detected as aws, metadata={"cloud":{"account":{"id":"622494574705"},"availability_zone":"us-east-1a","image":{"id":"ami-00521671dfc1e0100"},"instance":{"id":"i-0921bed04d4c5ab67"},"machine":{"type":"r5.large"},"provider":"aws","region":"us-east-1","service":{"name":"EC2"}}}
2021-11-19T22:37:36.720Z	DEBUG	[processors]	processors/processor.go:120	Generated new processors: add_host_metadata=[netinfo.enabled=[true], cache.ttl=[5m0s]], add_cloud_metadata={"cloud":{"account":{"id":"622494574705"},"availability_zone":"us-east-1a","image":{"id":"ami-00521671dfc1e0100"},"instance":{"id":"i-0921bed04d4c5ab67"},"machine":{"type":"r5.large"},"provider":"aws","region":"us-east-1","service":{"name":"EC2"}}}
2021-11-19T22:37:36.720Z	DEBUG	[seccomp]	seccomp/seccomp.go:117	Loading syscall filter	{"seccomp_filter": {"no_new_privs":true,"flag":"tsync","policy":{"default_action":"errno","syscalls":[{"names":["accept","accept4","access","arch_prctl","bind","brk","chmod","chown","clock_gettime","clone","close","connect","dup","dup2","epoll_create","epoll_create1","epoll_ctl","epoll_pwait","epoll_wait","exit","exit_group","fchdir","fchmod","fchmodat","fchown","fchownat","fcntl","fdatasync","flock","fstat","fstatfs","fsync","ftruncate","futex","getcwd","getdents","getdents64","geteuid","getgid","getpeername","getpid","getppid","getrandom","getrlimit","getrusage","getsockname","getsockopt","gettid","gettimeofday","getuid","inotify_add_watch","inotify_init1","inotify_rm_watch","ioctl","kill","listen","lseek","lstat","madvise","mincore","mkdirat","mmap","mprotect","munmap","nanosleep","newfstatat","open","openat","pipe","pipe2","poll","ppoll","pread64","pselect6","pwrite64","read","readlink","readlinkat","recvfrom","recvmmsg","recvmsg","rename","renameat","rt_sigaction","rt_sigprocmask","rt_sigreturn","sched_getaffinity","sched_yield","sendfile","sendmmsg","sendmsg","sendto","set_robust_list","setitimer","setsockopt","shutdown","sigaltstack","socket","splice","stat","statfs","sysinfo","tgkill","time","tkill","uname","unlink","unlinkat","wait4","waitid","write","writev"],"action":"allow"}]}}}
2021-11-19T22:37:36.720Z	INFO	[seccomp]	seccomp/seccomp.go:124	Syscall filter successfully installed
2021-11-19T22:37:36.720Z	INFO	[beat]	instance/beat.go:1014	Beat info	{"system_info": {"beat": {"path": {"config": "/etc/metricbeat", "data": "/var/lib/metricbeat", "home": "/usr/share/metricbeat", "logs": "/var/log/metricbeat"}, "type": "metricbeat", "uuid": "b7c91577-81b9-4a1d-a035-0fa2eba5ea93"}}}
2021-11-19T22:37:36.720Z	INFO	[beat]	instance/beat.go:1023	Build info	{"system_info": {"build": {"commit": "e127fc31fc6c00fdf8649808f9421d8f8c28b5db", "libbeat": "7.14.0", "time": "2021-07-29T21:09:05.000Z", "version": "7.14.0"}}}
2021-11-19T22:37:36.720Z	INFO	[beat]	instance/beat.go:1026	Go runtime info	{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.16.6"}}}
2021-11-19T22:37:36.721Z	INFO	[beat]	instance/beat.go:1030	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-08-16T20:12:39Z","containerized":false,"name":"ip-10-128-0-50.ec2.internal","ip":["127.0.0.1/8","::1/128","10.128.0.50/26","fe80::101b:d7ff:fe53:7d4d/64"],"kernel_version":"3.10.0-1160.15.2.el7.x86_64","mac":["12:1b:d7:53:7d:4d"],"os":{"type":"linux","family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":9,"patch":2009,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0,"id":"3d5c05376530a2eb49e3e90576f83c5b"}}}
2021-11-19T22:37:36.721Z	INFO	[beat]	instance/beat.go:1059	Process info	{"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null}, "cwd": "/", "exe": "/usr/share/metricbeat/bin/metricbeat", "name": "metricbeat", "pid": 21825, "ppid": 1, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2021-11-19T22:37:36.420Z"}}}
2021-11-19T22:37:36.721Z	INFO	instance/beat.go:309	Setup Beat: metricbeat; Version: 7.14.0
2021-11-19T22:37:36.721Z	DEBUG	[beat]	instance/beat.go:335	Initializing output plugins
2021-11-19T22:37:36.722Z	DEBUG	[publisher]	pipeline/consumer.go:148	start pipeline event consumer
2021-11-19T22:37:36.722Z	INFO	[publisher]	pipeline/module.go:113	Beat name: ip-10-128-0-50.ec2.internal
2021-11-19T22:37:36.739Z	DEBUG	[modules]	beater/metricbeat.go:151	Available modules and metricsets: Register [ModuleFactory:[aws, awsfargate, azure, beat, cloudfoundry, docker, elasticsearch, kibana, kubernetes, linux, logstash, mongodb, mssql, mysql, oracle, postgresql, system, uwsgi], MetricSetFactory:[aerospike/namespace, apache/status, appsearch/stats, aws/billing, aws/cloudwatch, awsfargate/task_stats, azure/app_insights, azure/billing, azure/monitor, azure/storage, beat/state, beat/stats, ceph/cluster_disk, ceph/cluster_health, ceph/cluster_status, ceph/mgr_cluster_disk, ceph/mgr_cluster_health, ceph/mgr_osd_perf, ceph/mgr_osd_pool_stats, ceph/mgr_osd_tree, ceph/mgr_pool_disk, ceph/monitor_health, ceph/osd_df, ceph/osd_tree, ceph/pool_disk, cloudfoundry/container, cloudfoundry/counter, cloudfoundry/value, consul/agent, coredns/stats, couchbase/bucket, couchbase/cluster, couchbase/node, couchdb/server, docker/container, docker/cpu, docker/diskio, docker/event, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, docker/network_summary, dropwizard/collector, elasticsearch/ccr, elasticsearch/cluster_stats, elasticsearch/enrich, elasticsearch/index, elasticsearch/index_recovery, elasticsearch/index_summary, elasticsearch/ml_job, elasticsearch/node, elasticsearch/node_stats, elasticsearch/pending_tasks, elasticsearch/shard, envoyproxy/server, etcd/leader, etcd/metrics, etcd/self, etcd/store, gcp/billing, gcp/metrics, golang/expvar, golang/heap, graphite/server, haproxy/info, haproxy/stat, http/json, http/server, istio/citadel, istio/galley, istio/mesh, istio/mixer, istio/pilot, jolokia/jmx, kafka/consumergroup, kafka/partition, kibana/stats, kibana/status, kubernetes/apiserver, kubernetes/container, kubernetes/controllermanager, kubernetes/event, kubernetes/node, kubernetes/pod, kubernetes/proxy, kubernetes/scheduler, kubernetes/state_container, kubernetes/state_cronjob, kubernetes/state_daemonset, kubernetes/state_deployment, kubernetes/state_node, kubernetes/state_persistentvolume, kubernetes/state_persistentvolumeclaim, kubernetes/state_pod, kubernetes/state_replicaset, kubernetes/state_resourcequota, kubernetes/state_service, kubernetes/state_statefulset, kubernetes/state_storageclass, kubernetes/system, kubernetes/volume, kvm/dommemstat, kvm/status, linux/conntrack, linux/iostat, linux/ksm, linux/memory, linux/pageinfo, logstash/node, logstash/node_stats, memcached/stats, mongodb/collstats, mongodb/dbstats, mongodb/metrics, mongodb/replstatus, mongodb/status, mssql/performance, mssql/transaction_log, munin/node, mysql/galera_status, mysql/query, mysql/status, nats/connection, nats/connections, nats/route, nats/routes, nats/stats, nats/subscriptions, nginx/stubstatus, openmetrics/collector, oracle/performance, oracle/tablespace, php_fpm/pool, php_fpm/process, postgresql/activity, postgresql/bgwriter, postgresql/database, postgresql/statement, prometheus/collector, prometheus/query, prometheus/remote_write, rabbitmq/connection, rabbitmq/exchange, rabbitmq/node, rabbitmq/queue, redis/info, redis/key, redis/keyspace, sql/query, stan/channels, stan/stats, stan/subscriptions, statsd/server, system/core, system/cpu, system/diskio, system/entropy, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/network_summary, system/process, system/process_summary, system/raid, system/service, system/socket, system/socket_summary, system/uptime, system/users, traefik/health, uwsgi/status, vsphere/datastore, vsphere/host, vsphere/virtualmachine, zookeeper/connection, zookeeper/mntr, zookeeper/server], LightModules:[LightModules:[mysql/performance, redisenterprise/node, redisenterprise/proxy, kafka/producer, kafka/consumer, kafka/broker, aws/vpn, aws/natgateway, aws/sqs, aws/lambda, aws/dynamodb, aws/rds, aws/s3_daily_storage, aws/s3_request, aws/transitgateway, aws/kinesis, aws/ec2, aws/ebs, aws/usage, aws/sns, aws/elb, ibmmq/qmgr, iis/webserver, iis/website, tomcat/cache, tomcat/threading, tomcat/memory, tomcat/requests, gcp/compute, gcp/pubsub, gcp/loadbalancing, gcp/storage, cockroachdb/status, istio/proxy, istio/istiod, azure/container_registry, azure/container_instance, azure/container_service, azure/database_account, azure/app_state, azure/compute_vm, azure/compute_vm_scaleset, googlecloud/compute, googlecloud/pubsub, googlecloud/loadbalancing, googlecloud/storage, activemq/broker, activemq/queue, activemq/topic]]]
2021-11-19T22:37:36.739Z	INFO	filesystem/filesystem.go:57	Ignoring filesystem types: squashfs, nsfs, nfs, smbfs, autofs
2021-11-19T22:37:36.740Z	INFO	instance/beat.go:442	metricbeat stopped.
2021-11-19T22:37:36.741Z	ERROR	instance/beat.go:989	Exiting: 1 error: can not convert 'object' into 'string' accessing 'metricbeat.modules.1.host' (source:'/etc/metricbeat/metricbeat.yml')

Can you please share your entire metricbeat.yml.

Updated above

If you don't want to put the port directly in the host name (eg hosts: ["localhost:80"] ) then your port line needs to be quoted too (eg port: "80" )

Thanks the port change didnt help the Metric beat still doesnt return any http metrics