Hi everyone,
I have some problems with Filebeat -> Elasticsearch -> Kibana in Kubernetes.
I get my logs to show up in Kibana, that's not the problem.
The problem is, that every Log that shows up just has the following information:
beat.hostname
beat.name
beat.version
kubernetes.container.name
kubernetes.namespace
kubernetes.node.name
kubernetes.pod.name
message
Now I want to use the Filebeat Dashboard for NGINX logs but it needs fields like nginx.access.geoip.location and so on. But I guess that is only possible if the nginx is not running in Kubernetes.
Is that right or do I just need to change some configuration in filebeat?
These two ConfigMaps I am using in Kuberentes:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-config
namespace: kube-system
labels:
app: filebeat
kubernetes.io/cluster-service: "true"
data:
filebeat.yml: |-
filebeat.config:
prospectors:
# Mounted `filebeat-prospectors` configmap:
path: ${path.config}/prospectors.d/*.yml
# Reload prospectors configs as they change:
reload.enabled: false
modules:
path: ${path.config}/modules.d/*.yml
# Reload module configs as they change:
reload.enabled: false
processors:
- add_cloud_metadata:
#cloud.id: ${ELASTIC_CLOUD_ID}
#cloud.auth: ${ELASTIC_CLOUD_AUTH}
output.elasticsearch:
hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
#bulk_max_size: 1000
#username: ${ELASTICSEARCH_USERNAME}
#password: ${ELASTICSEARCH_PASSWORD}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-prospectors
namespace: kube-system
labels:
app: filebeat
kubernetes.io/cluster-service: "true"
data:
kubernetes.yml: |-
- type: docker
containers.ids:
- "*"
processors:
- add_kubernetes_metadata:
in_cluster: true
---
Thanks for any help