# Filebeat not sending logs to EKS in a Microk8s Kubernetes environment

**URL:** <https://discuss.elastic.co/t/filebeat-not-sending-logs-to-eks-in-a-microk8s-kubernetes-environment/345221>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [October 17, 2023, 2:20pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-eks-in-a-microk8s-kubernetes-environment/345221 "2023-10-17T14:20:43Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![NashF](https://avatars.discourse-cdn.com/v4/letter/n/c5a1d2/32.png) [@NashF](https://discuss.elastic.co/u/NashF)\
**Post date:** [October 17, 2023, 2:20pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-eks-in-a-microk8s-kubernetes-environment/345221/1 "2023-10-17T14:20:43Z")

</div>

I'm at my wits end with this ELK stack and Filebeat. Currently using Elasticsearch, Kibana ,Filebeat all version 8.10.2 and logstash logstash-logback-encoder:7.4 for converting my springboot microservice logs into json objects

Currently the Elasticsearch, Kibana are working just fine with each other, I'm able to access the Kibana dashboard. No issue.

The real issue is Filebeat, for all intents and purposes it seems to be functioning well at least from the logs, I'm not seeing any errors. From this I was assuming that Filebeat was able to send log data to Elasticsearch to be indexed. But no log data was being sent nor any indexes being created for Filebeat in Elastic Search.

I'm adding the manifests and Filebeat yml configuration below, Does filebeat not work nicely with ELK in a microk8 kubernetes environment?

================== Filebeat.yml start ================================  
filebeat.autodiscover:  
providers:  
- type: kubernetes  
include\_labels:  
- "app"  
node: kvm-u-node0  
templates:  
- condition:  
contains:  
kubernetes.container.labels.collect\_logs\_with\_filebeat: "true"  
config:  
- type: container  
paths:  
- "/var/lib/docker/containers/${data.kubernetes.namespace}_${data.kubernetes.pod.name}_${data.kubernetes.pod.uid}/${data.kubernetes.container.name}/**/\*.log"  
json.keys\_under\_root: true  
json.add\_error\_key: true  
json.message\_key: message  
- condition:  
contains:  
kubernetes.container.labels.decode\_log\_event\_to\_json\_object: "true"  
config:  
- type: container  
paths:  
- "/var/lib/docker/containers/${data.kubernetes.namespace}_${data.kubernetes.pod.name}_${data.kubernetes.pod.uid}/${data.kubernetes.container.name}/**/\*.log"  
processors:  
- decode\_json\_fields:  
fields: ["message"]  
target: ""  
overwrite\_keys: true

# filebeat.inputs:

# - type: log

# enabled: true

# paths:

# - "/var/lib/docker/containers/${data.kubernetes.namespace}_${data.kubernetes.pod.name}_${data.kubernetes.pod.uid}/${data.kubernetes.container.name}/\*\*/\*.log"

output.elasticsearch:  
enabled: true  
hosts: ["elasticsearch-service-for-springboot:9200"]  
username: "elastic"  
password: "password\_elastic"  
index: "filebeat-%{+yyyy.MM.dd}"  
pipeline: "filebeat-import-example-pipeline"

# setup.kibana:

# host: "10.152.183.79:5601"

# username: "kibana\_system"

# password: "password\_kibana"

# 

setup.ilm.enabled: true  
setup.ilm.rollover\_alias: "filebeat"  
setup.ilm.pattern: "filebeat-{now/d}-000001"

setup.template.name: "filebeat"  
setup.template.pattern: "filebeat-\*"

## logging.level: debug logging.metrics.enabled: false ================== Filebeat.yml end ================================ ==================Filebeat manifest start ============================

## apiVersion: v1 kind: Service metadata: name: filebeat-service namespace: dev spec: selector: app: filebeat ports: - protocol: TCP port: 5044 targetPort: 5044 type: ClusterIP

apiVersion: apps/v1  
kind: DaemonSet  
metadata:  
name: filebeat  
namespace: dev  
labels:  
app: filebeat  
spec:  
selector:  
matchLabels:  
app: filebeat  
template:  
metadata:  
labels:  
app: filebeat  
spec:  
hostNetwork: true  
volumes:  
- name: config  
hostPath:  
path: /home/nash/Documents/food-ordering-demo-microservices/elasticsearch/kubernetes/logging/filebeat/filebeat.yml  
type: File  
- name: varlibdockercontainers  
hostPath:  
path: /var/log/pods  
- name: kuberenetes-sock  
hostPath:  
path: /var/run/kubernetes.sock  
containers:  
- name: filebeat  
image: [docker.elastic.co/beats/filebeat:8.10.2](http://docker.elastic.co/beats/filebeat:8.10.2)  
env:  
- name: ELASTICSEARCH\_HOST  
value: elasticsearch-service-for-springboot  
- name: ELASTICSEARCH\_PORT  
value: "9200"  
- name: ELASTICSEARCH\_USERNAME  
value: elastic  
- name: ELASTICSEARCH\_PASSWORD  
value: password\_elastic  
volumeMounts:  
- name: config  
mountPath: /usr/share/filebeat/filebeat.yml  
readOnly: true  
- name: varlibdockercontainers  
mountPath: /var/lib/docker/containers  
readOnly: true  
- name: kuberenetes-sock  
mountPath: /var/run/kubernetes.sock  
readOnly: true  
==================Filebeat manifest end ============================

## ==================Elasticsearch and Kibana manifests start ====================

apiVersion: apps/v1  
kind: StatefulSet  
metadata:  
name: elasticsearch  
namespace: dev  
spec:  
serviceName: elasticsearch  
replicas: 1  
selector:  
matchLabels:  
app: elasticsearch  
template:  
metadata:  
labels:  
app: elasticsearch  
spec:

# initContainers:

# - name: init-reset-password

# image: [docker.elastic.co/elasticsearch/elasticsearch:8.10.2](http://docker.elastic.co/elasticsearch/elasticsearch:8.10.2)

# command: ["/bin/sh", "-c", "cp /config/reset-passwords.sh /usr/share/elasticsearch/bin/ && chmod +x /usr/share/elasticsearch/bin/reset-passwords.sh && /usr/share/elasticsearch/bin/reset-passwords.sh"]

```
  containers:
    - name: elasticsearch
      image: docker.elastic.co/elasticsearch/elasticsearch:8.10.2
      imagePullPolicy: Always

```

# command: ["/bin/sh", "-c", "cp /config/reset-passwords.sh /usr/share/elasticsearch/bin/ && chmod +x /usr/share/elasticsearch/bin/reset-passwords.sh && /usr/share/elasticsearch/bin/reset-passwords.sh"]

```
      resources:
        requests:
          memory: "600Mi"
          cpu: ".5"
        limits:
          memory: "2Gi"
          cpu: "1.5"
      env:
        - name: cluster.routing.allocation.disk.threshold_enabled
          value: "false"
        - name: xpack.security.enabled
          value: "true"
        - name: xpack.security.enrollment.enabled
          value: "true"
        - name: xpack.security.transport.ssl.enabled
          value: "false"
        - name: xpack.security.http.ssl.enabled
          value: "false"
        - name: discovery.type
          value: single-node
        - name: node.name
          value: elasticsearch
        - name: bootstrap.memory_lock
          value: "false"
        - name: ES_JAVA_OPTS
          value: -Xms1g -Xmx1g
        - name: ELASTICSEARCH_PASSWORD
          value: password_elastic
        - name: KIBANA_PASSWORD
          value: password_kibana
      ports:
        - containerPort: 9200
          name: http
        - containerPort: 9300
          name: transport
      volumeMounts:
        - name: data
          mountPath: /home/nash/elasticsearch/data

```

# - name: init-config

# mountPath: /config

# volumes:

# - name: init-config

# configMap:

# name: elasticsearch-init-config

```
  securityContext:
    runAsUser: 1000 # Set the desired user ID
    fsGroup: 1000 # Set the desired group ID

```

volumeClaimTemplates:  
- metadata:  
name: data  
labels:  
app: elasticsearch  
spec:  
accessModes: ["ReadWriteOnce"]  
resources:  
requests:  
storage: 5Gi

* * *

## apiVersion: v1 kind: Service metadata: name: elasticsearch-service-for-springboot labels: app: elasticsearch-service-for-springboot namespace: dev spec: ports: - name: elasticsearch-port1 port: 9200 targetPort: 9200 - name: elasticsearch-port2 port: 9300 targetPort: 9300 type: ClusterIP selector: app: elasticsearch

## apiVersion: apps/v1 kind: Deployment metadata: name: kibana namespace: dev spec: selector: matchLabels: app: kibana replicas: 1 template: metadata: labels: app: kibana spec: containers: - name: kibana image: [docker.elastic.co/kibana/kibana:8.10.2](http://docker.elastic.co/kibana/kibana:8.10.2) # Specify the appropriate Kibana version imagePullPolicy: Always env: - name: server.port value: "5601" - name: xpack.security.enabled value: "false" - name: xpack.ingestManager.fleet.tlsCheckDisabled value: "false" - name: xpack.fleet.enabled value: "true" - name: xpack.security.transport.ssl.enabled value: "false" - name: xpack.security.http.ssl.enabled value: "false" - name: ELASTICSEARCH\_HOSTS value: [http://elasticsearch-service-for-springboot:9200](http://elasticsearch-service-for-springboot:9200) # URL of the Elasticsearch service - name: xpack.fleet.registryUrl value: [https://epr.elastic.co/](https://epr.elastic.co/) - name: ELASTICSEARCH\_USERNAME value: kibana\_system - name: ELASTICSEARCH\_PASSWORD value: password\_kibana ports: - containerPort: 5601

apiVersion: v1  
kind: Service  
metadata:  
name: kibana-service-for-springboot  
labels:  
app: kibana-service-for-springboot  
namespace: dev  
spec:  
ports:  
- name: kibana-port1  
port: 5601  
targetPort: 5601  
type: ClusterIP  
selector:  
app: kibana

==================Elasticsearch and Kibana manifests end ====================

Snippet of the podlogs in Filebeat :

{"log.level":"debug","@timestamp":"2023-10-17T04:26:03.628Z","log.logger":"autodiscover","log.origin":{"file.name":"autodiscover/autodiscover.go","file.line":274},"message":"Got a stop event: map[config: host:10.1.83.198 id:33ace020-4df8-4ce5-b94c-27b9ff902d59 kubernetes:{"annotations":{"cni":{"projectcalico":{"org/containerID":"9f026e0bfccff823e12dfb06384a6737147d28a9e77d30fb6f1764fda1521f01","org/podIP":"10.1.83.198/32","org/podIPs":"10.1.83.198/32"}}},"deployment":{"name":"food-ordering-demo-springboot"},"labels":{"app":"food-ordering-demo-springboot","pod-template-hash":"574945b945"},"namespace":"dev","namespace\_labels":{"kubernetes\_io/metadata\_name":"dev"},"namespace\_uid":"4d4e3005-c960-49ad-acf2-c06c29adc590","node":{"hostname":"kvm-u-node0","labels":{"beta\_kubernetes\_io/arch":"amd64","beta\_kubernetes\_io/os":"linux","kubernetes\_io/arch":"amd64","kubernetes\_io/hostname":"kvm-u-node0","kubernetes\_io/os":"linux","microk8s\_io/cluster":"true","node\_kubernetes\_io/microk8s-controlplane":"microk8s-controlplane"},"name":"kvm-u-node0","uid":"561630e1-6656-4f8e-9cd7-310fe6469883"},"pod":{"ip":"10.1.83.198","name":"food-ordering-demo-springboot-574945b945-smchb","uid":"33ace020-4df8-4ce5-b94c-27b9ff902d59"},"replicaset":{"name":"food-ordering-demo-springboot-574945b945"}} meta:{"kubernetes":{"deployment":{"name":"food-ordering-demo-springboot"},"labels":{"app":"food-ordering-demo-springboot"},"namespace":"dev","namespace\_labels":{"kubernetes\_io/metadata\_name":"dev"},"namespace\_uid":"4d4e3005-c960-49ad-acf2-c06c29adc590","node":{"hostname":"kvm-u-node0","labels":{"beta\_kubernetes\_io/arch":"amd64","beta\_kubernetes\_io/os":"linux","kubernetes\_io/arch":"amd64","kubernetes\_io/hostname":"kvm-u-node0","kubernetes\_io/os":"linux","microk8s\_io/cluster":"true","node\_kubernetes\_io/microk8s-controlplane":"microk8s-controlplane"},"name":"kvm-u-node0","uid":"561630e1-6656-4f8e-9cd7-310fe6469883"},"pod":{"ip":"10.1.83.198","name":"food-ordering-demo-springboot-574945b945-smchb","uid":"33ace020-4df8-4ce5-b94c-27b9ff902d59"},"replicaset":{"name":"food-ordering-demo-springboot-574945b945"}}} ports:{"port-tcp":8080,"port-udp":8080} provider:7e8787b4-0745-445b-a44d-a3cd74765d0f stop:true]","service.name":"filebeat","ecs.version":"1.6.0"}  
{"log.level":"debug","@timestamp":"2023-10-17T04:26:03.628Z","log.logger":"autodiscover","log.origin":{"file.name":"autodiscover/autodiscover.go","file.line":274},"message":"Got a stop event: map[config: host:10.1.83.198 id:33ace020-4df8-4ce5-b94c-27b9ff902d59.food-ordering-demo-springboot kubernetes:{"annotations":{"cni":{"projectcalico":{"org/containerID":"9f026e0bfccff823e12dfb06384a6737147d28a9e77d30fb6f1764fda1521f01","org/podIP":"10.1.83.198/32","org/podIPs":"10.1.83.198/32"}}},"container":{"id":"f12b9c5555a28cccbfb202d5fc910f108a3b2faf8fda22d3865f2e35ccc9b91a","image":"192.168.20.222:8443/digitalbiohumans/poc/food-ordering-axoniq-demo-springboot:0.0.7.2-spring-cloud-config-secure-client-SNAPSHOT","name":"food-ordering-demo-springboot","runtime":"containerd"},"deployment":{"name":"food-ordering-demo-springboot"},"labels":{"app":"food-ordering-demo-springboot","pod-template-hash":"574945b945"},"namespace":"dev","namespace\_labels":{"kubernetes\_io/metadata\_name":"dev"},"namespace\_uid":"4d4e3005-c960-49ad-acf2-c06c29adc590","node":{"hostname":"kvm-u-node0","labels":{"beta\_kubernetes\_io/arch":"amd64","beta\_kubernetes\_io/os":"linux","kubernetes\_io/arch":"amd64","kubernetes\_io/hostname":"kvm-u-node0","kubernetes\_io/os":"linux","microk8s\_io/cluster":"true","node\_kubernetes\_io/microk8s-controlplane":"microk8s-controlplane"},"name":"kvm-u-node0","uid":"561630e1-6656-4f8e-9cd7-310fe6469883"},"pod":{"ip":"10.1.83.198","name":"food-ordering-demo-springboot-574945b945-smchb","uid":"33ace020-4df8-4ce5-b94c-27b9ff902d59"},"replicaset":{"name":"food-ordering-demo-springboot-574945b945"}} meta:{"container":{"id":"f12b9c5555a28cccbfb202d5fc910f108a3b2faf8fda22d3865f2e35ccc9b91a","image":{"name":"192.168.20.222:8443/digitalbiohumans/poc/food-ordering-axoniq-demo-springboot:0.0.7.2-spring-cloud-config-secure-client-SNAPSHOT"},"runtime":"containerd"},"kubernetes":{"container":{"name":"food-ordering-demo-springboot"},"deployment":{"name":"food-ordering-demo-springboot"},"labels":{"app":"food-ordering-demo-springboot"},"namespace":"dev","namespace\_labels":{"kubernetes\_io/metadata\_name":"dev"},"namespace\_uid":"4d4e3005-c960-49ad-acf2-c06c29adc590","node":{"hostname":"kvm-u-node0","labels":{"beta\_kubernetes\_io/arch":"amd64","beta\_kubernetes\_io/os":"linux","kubernetes\_io/arch":"amd64","kubernetes\_io/hostname":"kvm-u-node0","kubernetes\_io/os":"linux","microk8s\_io/cluster":"true","node\_kubernetes\_io/microk8s-controlplane":"microk8s-controlplane"},"name":"kvm-u-node0","uid":"561630e1-6656-4f8e-9cd7-310fe6469883"},"pod":{"ip":"10.1.83.198","name":"food-ordering-demo-springboot-574945b945-smchb","uid":"33ace020-4df8-4ce5-b94c-27b9ff902d59"},"replicaset":{"name":"food-ordering-demo-springboot-574945b945"}}} provider:7e8787b4-0745-445b-a44d-a3cd74765d0f stop:true]","service.name":"filebeat","ecs.version":"1.6.0"}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 14, 2023, 4:21pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-eks-in-a-microk8s-kubernetes-environment/345221/2 "2023-11-14T16:21:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.