# Filebeat to elasticsearch in Kubernetes

**URL:** <https://discuss.elastic.co/t/filebeat-to-elasticsearch-in-kubernetes/124155>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 15, 2018, 5:49pm UTC](https://discuss.elastic.co/t/filebeat-to-elasticsearch-in-kubernetes/124155 "2018-03-15T17:49:59Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![clule404](https://avatars.discourse-cdn.com/v4/letter/c/f05b48/32.png) [@clule404](https://discuss.elastic.co/u/clule404)\
**Post date:** [March 15, 2018, 5:49pm UTC](https://discuss.elastic.co/t/filebeat-to-elasticsearch-in-kubernetes/124155/1 "2018-03-15T17:49:59Z")

</div>

Hi everyone,

I have some problems with Filebeat -\> Elasticsearch -\> Kibana in Kubernetes.  
I get my logs to show up in Kibana, that's not the problem.  
The problem is, that every Log that shows up just has the following information:

```
beat.hostname
beat.name
beat.version 	
kubernetes.container.name 	
kubernetes.namespace 	
kubernetes.node.name 	
kubernetes.pod.name 	
message

```

Now I want to use the Filebeat Dashboard for NGINX logs but it needs fields like nginx.access.geoip.location and so on. But I guess that is only possible if the nginx is not running in Kubernetes.  
Is that right or do I just need to change some configuration in filebeat?

These two ConfigMaps I am using in Kuberentes:

```
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: kube-system
  labels:
    app: filebeat
    kubernetes.io/cluster-service: "true"
data:
  filebeat.yml: |-
    filebeat.config:
      prospectors:
        # Mounted `filebeat-prospectors` configmap:
        path: ${path.config}/prospectors.d/*.yml
        # Reload prospectors configs as they change:
        reload.enabled: false
      modules:
        path: ${path.config}/modules.d/*.yml
        # Reload module configs as they change:
        reload.enabled: false

    processors:
      - add_cloud_metadata:

              #cloud.id: ${ELASTIC_CLOUD_ID}
              #cloud.auth: ${ELASTIC_CLOUD_AUTH}

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      #bulk_max_size: 1000
      #username: ${ELASTICSEARCH_USERNAME}
      #password: ${ELASTICSEARCH_PASSWORD}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-prospectors
  namespace: kube-system
  labels:
    app: filebeat
    kubernetes.io/cluster-service: "true"
data:
  kubernetes.yml: |-
    - type: docker
      containers.ids:
      - "*"
      processors:
        - add_kubernetes_metadata:
            in_cluster: true
---

```

Thanks for any help 🙂

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [March 15, 2018, 5:58pm UTC](https://discuss.elastic.co/t/filebeat-to-elasticsearch-in-kubernetes/124155/2 "2018-03-15T17:58:58Z")

</div>

Hi @clule404,

It's actually possible, have a look at autodiscover features, there is an example for redis: [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html#\_docker\_2](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html#_docker_2)

You would have to add something like this to `filebeat.yml` (I haven't tested it):

```auto
filebeat.autodiscover:
  providers:
    - type: docker
      templates:
        - condition:
            equals:
              docker.container.image: nginx
          config:
            - module: nginx
              access:
                prospector:
                  type: docker
                  containers.ids:
                  - "${data.docker.container.id}"

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 12, 2018, 5:59pm UTC](https://discuss.elastic.co/t/filebeat-to-elasticsearch-in-kubernetes/124155/3 "2018-04-12T17:59:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.