# Kubernetes data collection error with metricbeat!

**URL:** https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511
**Category:** Beats
**Tags:** metricbeat
**Created:** [August 8, 2018, 12:19pm UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511 "2018-08-08T12:19:14Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 8, 2018, 12:19pm UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/1 "2018-08-08T12:19:14Z")

</div>

Hello all!  
I am trying to get kubernetes describing data (kubernetes.) via metricbeat. For installation i used metricbeat-kubernetes.yaml template. I have installed locally single-node kubernetes via docker, and many-node kubernetes on specifical host. So, MAIN problem is that local kubernetes cluster sends all data, including kubernetes., system. and so on, but another cluster sends only system.\* data. When i found error why kubernetes data is not sending: `"error making http request: Get http://localhost:10255/stats/summary: dial tcp localhost:10255: getsockopt: connection refused"`. So i changed localhost:10255 to (InternalIIP(master node) which i found in kubernetes dashboard):10255. In result, from local kubenetes cluster data about is still sending, but external cluster shows same error about "connection refused".  
Can someone help with this problem?

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 8, 2018, 12:24pm UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/2 "2018-08-08T12:24:35Z")

</div>

Problem connected with this code part (HOSTS) ==\>

kubernetes.yml: |-  
- module: kubernetes  
metricsets:  
- node  
- system  
- pod  
- container  
- volume  
period: 10s  
hosts: ["localhost / IntrernalIP(master node) \<== tried both:10255"]

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 8, 2018, 12:31pm UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/3 "2018-08-08T12:31:27Z")

</div>

External Kubernetes cluster (master node/all nodes) response:  
 ![Screenshot%20(186)](https://us1.discourse-cdn.com/elastic/original/3X/8/8/887ce01895623d0314420d760743c7ef89890bfb.png)

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 8, 2018, 12:36pm UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/4 "2018-08-08T12:36:49Z")

</div>

Local Kubernetes cluster (master node) response:

 ![Screenshot%20(187)](https://us1.discourse-cdn.com/elastic/original/3X/6/3/639668ba5802339e3dd6224ab5394a1fc66b8587.png)

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [August 8, 2018, 4:01pm UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/5 "2018-08-08T16:01:00Z")

</div>

Hi @b22565875b81d856c5b0,

In `localhost:10255` it is expected to find the kubelet read-only API, not to be mistaken by the node master or the apiserver. This kubelet API allows to collect info and metrics of the node where this kubelet is running. It should be available on all nodes of your cluster, you can check it by running `curl localhost:10255` on any node, if it is not available there please check your kubelets status and configuration.

If kubelet is running, and it is reachable on `localhost:10255`, then there must be some connectivity problem between the metricbeat pod and the host network, please check that the metricbeat pods started by the daemon set on every node include the setting `hostNetwork: true`.

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 9, 2018, 8:01am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/6 "2018-08-09T08:01:04Z")

</div>

Thanks for response! I found that curl localhost:10255 is not available on my nodes. So i checked netstat and found that port 10255 dont exist at all. ☹ How could i add needed port?

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 9, 2018, 8:01am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/7 "2018-08-09T08:01:52Z")

</div>

Kubelet config file:  
 ![Screenshot%20(190)](https://us1.discourse-cdn.com/elastic/original/3X/e/b/ebecbe1b6b5f586657f143ec58e8271f3e042bbd.png)  
NetStat output:

 ![Screenshot%20(191)](https://us1.discourse-cdn.com/elastic/original/3X/0/b/0b99d7c617c3d458c8661700c54125c6b80d3c04.png)

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [August 9, 2018, 10:55am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/8 "2018-08-09T10:55:39Z")

</div>

What version of kubernetes are you using?

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 9, 2018, 11:03am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/10 "2018-08-09T11:03:02Z")

</div>

1.10.3.

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 9, 2018, 11:05am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/11 "2018-08-09T11:05:47Z")

</div>

{  
"clientVersion": {  
"major": "1",  
"minor": "10",  
"gitVersion": "v1.10.3",  
"gitCommit": "2bba0127d85d5a46ab4b778548be28623b32d0b0",  
"gitTreeState": "clean",  
"buildDate": "2018-05-21T09:17:39Z",  
"goVersion": "go1.9.3",  
"compiler": "gc",  
"platform": "linux/amd64"  
},  
"serverVersion": {  
"major": "1",  
"minor": "10",  
"gitVersion": "v1.10.3",  
"gitCommit": "2bba0127d85d5a46ab4b778548be28623b32d0b0",  
"gitTreeState": "clean",  
"buildDate": "2018-05-21T09:05:37Z",  
"goVersion": "go1.9.3",  
"compiler": "gc",  
"platform": "linux/amd64"  
}  
}

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [August 9, 2018, 11:31am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/12 "2018-08-09T11:31:10Z")

</div>

It should be there then, kubelet opens this port unless it is modified or disabled with the `--read-only-port` flag. Can you share also the arguments you are running `kubelet` with?

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 9, 2018, 11:36am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/14 "2018-08-09T11:36:21Z")

</div>

I have found, that --read-only-port has been deprecated. Do you know how could i get it available?

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 9, 2018, 2:02pm UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/15 "2018-08-09T14:02:18Z")

</div>

Or how to get same information, if if would use 10250 (secure port) instead localhost:10255/stats/summary?

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 15, 2018, 7:35am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/16 "2018-08-15T07:35:42Z")

</div>

up...

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [August 15, 2018, 8:57am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/17 "2018-08-15T08:57:28Z")

</div>

Umm, yes, it seems that the flag is deprecated, but it could still be in use. This port is open by default, I have just checked in two different 1.10 clusters. Did you check if this flag was being used (`ps aux | grep kubelet`)?

How are you configuring your cluster? It seems that for example `kubeadm` disables the read-only port by default.

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 15, 2018, 9:37am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/18 "2018-08-15T09:37:47Z")

</div>

> [@jsoriano](#):
>
> ps aux | grep kubelet

Entering "ps aux | grep kubelet" i have found that read-only-port is defined as 0.

 ![Screenshot%20(197)_LI](https://us1.discourse-cdn.com/elastic/original/3X/b/4/b4bd930fb39164297d4df03d09f63d2f54d717bf.jpeg)

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 15, 2018, 9:40am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/19 "2018-08-15T09:40:41Z")

</div>

How and which config file should i change to get available this port? etc/kubernetes/kubelet.conf?

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [August 15, 2018, 9:44am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/20 "2018-08-15T09:44:01Z")

</div>

> [@b22565875b81d856c5b0](#):
>
> How and which config file should i change to get available this port? etc/kubernetes/kubelet.conf?

It depends on what is adding this flag 🙂 This is why I asked about how you are managing the configuration of your cluster.

---

<div class="post-metadata">

### Author: ![b22565875b81d856c5b0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b22565875b81d856c5b0/32/33762_2.png) [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)
#### Post date: [August 15, 2018, 9:53am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/22 "2018-08-15T09:53:15Z")

</div>

Honestly, i am newbie in kubernetes, so it is possible what i something can not understand, but when i check kubeadm, it shows what "not found". This cluster configured by my colegue from Ukraine, so many things about cluster i don`t know. ![:frowning:](https://emoji.discourse-cdn.com/twitter/frowning.png?v=5) ![Screenshot%20(199)_LI](https://us1.discourse-cdn.com/elastic/original/3X/d/f/df623e2ae68928d50d5e727e4e085016f31a35a0.jpeg)

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [August 15, 2018, 10:27am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511/23 "2018-08-15T10:27:12Z")

</div>

You'd need to check how `kubelet` is started, but probably better to wait for your colleague 🙂

In the meantime if you want you can try to [configure ssl](https://www.elastic.co/guide/en/beats/metricbeat/6.3/metricbeat-module-kubernetes.html) to use the secure port.

[Next page](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511.md?page=2)
