# Unable to flow kuberbenetes logs to elasticsearch **URL:** \ **Category:** Elasticsearch\ **Created:** [October 27, 2020, 3:35pm UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459 "2020-10-27T15:35:45Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![arunreddy00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arunreddy00/32/78054_2.png) [@arunreddy00](https://discuss.elastic.co/u/arunreddy00)\ **Post date:** [October 27, 2020, 3:35pm UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/1 "2020-10-27T15:35:45Z")
Hello Elastic, Really need your help struggling to get logs to ElasticSearch 1. Created EKS cluster in AWS with two nodes 2. Later integrated VPN (site to site connection) from EKS cluster's VPC to connect to Office network. 3. Established Vpn connection to flow kuberbenetes logs from Eks to Elasticsearch which is in office network. - Able to ping and telnet from EKS cluster nodes,but Kuberbenetes logs are not going to Elasticsearch. 1. Deployed kube-state-metrics on EKS cluster 2. later deployed metricbeat-kubernetes.yaml from below link [https://github.com/elastic/beats/blob/7.9.3/deploy/kubernetes/metricbeat-kubernetes.yaml](https://github.com/elastic/beats/blob/7.9.3/deploy/kubernetes/metricbeat-kubernetes.yaml) •kubectl create -f metricbeat-kubernetes.yaml changed below fields: `output.elasticsearch:` `hosts: ['10.10.10.11:9200']` ` username: ${ELASTICSEARCH_USERNAME}` ` password: ${ELASTICSEARCH_PASSWORD}` `env:` ` -name: ELASTICSEARCH_HOST` `value: 10.10.10.11` `- name: ELASTICSEARCH_PORT` `value: "9200"` ` - name: ELASTICSEARCH_USERNAME` `value: elastic` ` - name: ELASTICSEARCH_PASSWORD` `value: jkjgshfaytioutgsaifug` Error logs: please find attachment ![Screenshot (26)](https://us1.discourse-cdn.com/elastic/original/3X/9/d/9d85a334d8378f9f9411816757f1e44b6a588a0b.png) ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [November 1, 2020, 6:25pm UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/2 "2020-11-01T18:25:08Z")
1. Is that really HTTP (as in the connection attempt) or HTTPS? 2. Since you say that you can telnet, what happens when you run a cURL request? And that is from the EKS cluster, right? ---
**Author:** ![arunreddy00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arunreddy00/32/78054_2.png) [@arunreddy00](https://discuss.elastic.co/u/arunreddy00)\ **Post date:** [November 2, 2020, 6:42am UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/3 "2020-11-02T06:42:02Z")
Thank you xeraa, for response .Yes, this is from eks cluster _Figured out what went wrong._ replaced with below fields with earlier fields `output.elasticsearch:` `hosts: ['${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}']` `username: ${ELASTICSEARCH_USERNAME}` `password: ${ELASTICSEARCH_PASSWORD}` `ssl.verification_mode: none` `env:` `- name: ELASTICSEARCH_HOST` `value: https://10.10.10.11` `- name: ELASTICSEARCH_PORT` `value: "9200"` `- name: ELASTICSEARCH_USERNAME` `value: elastic` `- name: ELASTICSEARCH_PASSWORD` `value: jkjgshfaytioutgsaifug` Now, Logs are visible in the ElasticSearch dashboard. But, In Controller Manager & API server ECS,&Scheduler no metrics are shown, dashboards are empty . I don't understand why these dashboards are empty, Can i know why does the dashboards are not filled with metrics. ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [November 3, 2020, 1:21am UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/4 "2020-11-03T01:21:17Z")
So what data do you have in the Metricbeat index then? ---
**Author:** ![arunreddy00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arunreddy00/32/78054_2.png) [@arunreddy00](https://discuss.elastic.co/u/arunreddy00)\ **Post date:** [November 3, 2020, 2:13am UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/5 "2020-11-03T02:13:59Z")
1.In Elasticsearch dashboard , have detailed metrics of **overview of kubernetes cluster metrics** and **kubernetes proxy metrics**. 2.But **Controller Manager & API server ECS,&Scheduler** dashboards are empty. - Is this happening because of EKS cluster..? ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [November 3, 2020, 4:18pm UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/6 "2020-11-03T16:18:08Z")
I've never used EKS, so "maybe". The docs know what data and fields that should be collecting — do you get that data in your cluster? ---
**Author:** ![arunreddy00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arunreddy00/32/78054_2.png) [@arunreddy00](https://discuss.elastic.co/u/arunreddy00)\ **Post date:** [November 3, 2020, 5:57pm UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/7 "2020-11-03T17:57:58Z")
where do i get docs.? To compare with.. As i didn't find any docs about EKScluster metrics dashboard's in Elasticsearch ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [November 7, 2020, 7:34pm UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/8 "2020-11-07T19:34:32Z")
[https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-kubernetes-controllermanager.html](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-kubernetes-controllermanager.html) has the expected fields. In Kibana's Discover you can filter the data down to `metricset.name : "controllermanager"`. The first step will be to figure out if you are collecting the necessary data. Based on that the next step will be: Either look at the Metricbeat logs to see why they are not being collected; or debug the dashboard why they are not being displayed correctly. PS: I assume you have enabled the required module, right? ``` - module: kubernetes enabled: true metricsets: - state_node - state_deployment - state_replicaset - state_statefulset - state_pod - state_container - state_cronjob - state_resourcequota - state_service - state_persistentvolume - state_persistentvolumeclaim - state_storageclass ``` ---
**Author:** ![arunreddy00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arunreddy00/32/78054_2.png) [@arunreddy00](https://discuss.elastic.co/u/arunreddy00)\ **Post date:** [November 7, 2020, 8:20pm UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/9 "2020-11-07T20:20:43Z")
_Yes_, I enabled that module For reference, i used below link to deploy > ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [November 9, 2020, 3:32am UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/10 "2020-11-09T03:32:14Z")
Good. So can you find the data from that module in Kibana's Discover? Otherwise we'll need to take a look at the logs. ---
**Author:** ![arunreddy00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arunreddy00/32/78054_2.png) [@arunreddy00](https://discuss.elastic.co/u/arunreddy00)\ **Post date:** [November 18, 2020, 6:05am UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/11 "2020-11-18T06:05:07Z")
Dear Xeraa, After uncomment below field I see data in the dashboard. But the data is wrong which i see in the dashboard. `- module: kubernetes` ` metricsets:` ` - apiserver` `hosts: ["https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}"] bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token` `ssl.certificate_authorities:` ` - /var/run/secrets/kubernetes.io/serviceaccount/ca.crt` ` period: 30s` The data i see in the dashboard is , please find attachment ![Screenshot (45)](https://us1.discourse-cdn.com/elastic/original/3X/3/c/3c6cc0acb27a60480ba77ba4ba719892e0fd6342.png) But in EKS cluster i only have - node :1 - secrets : 1 - services: 1 - namespaces: 4 - configmaps: 1 - leases: 0 - endpoints: 1 Can you please look into the issue . ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [November 18, 2020, 10:32am UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/12 "2020-11-18T10:32:50Z")
Let's see why: - Click "Edit" on your dashboard. - Open the visualization (each visualization will have an icon in the top right corner that you can open and then edit it) - On the visualization there should be an "Inspect" button that shows you the actual query and response. With that information we can figure out, how it is getting to that result. ---
**Author:** ![arunreddy00](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arunreddy00/32/78054_2.png) [@arunreddy00](https://discuss.elastic.co/u/arunreddy00)\ **Post date:** [December 7, 2020, 12:51am UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/13 "2020-12-07T00:51:30Z")
When i went through the steps you mentioned, to "Inspect" dashboard, it shows Disabled. We assume this type of visualization does not support inspect. Please find below attachment ![Screenshot (83)](https://us1.discourse-cdn.com/elastic/original/3X/f/a/fa38dc7c3434d107985879e1074daf65b2704bc6.png) Thank you Xeraa, For helping. ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [December 7, 2020, 5:51am UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/14 "2020-12-07T05:51:56Z")
Sorry, on some visualizations this isn't available (and I always forget which ones and in which version), so we'll have to look at the configuration for that one. And actually the visualization without data would be the interesting one. What's the index and setting a little further down on the visualization? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 4, 2021, 5:52am UTC](https://discuss.elastic.co/t/unable-to-flow-kuberbenetes-logs-to-elasticsearch/253459/15 "2021-01-04T05:52:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.