Hi there, I'm struggling to get filebeats to pick up log files in my pods using autodiscovery annotations. My filebeat daemonset config looks like this:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-config
namespace: kube-system
labels:
k8s-app: filebeat
data:
filebeat.yml: |-
filebeat.autodiscover:
providers:
- type: kubernetes
host: ${NODE_NAME}
hints.enabled: true
hints.default_config:
type: container
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log
processors:
- add_cloud_metadata:
- add_host_metadata:
- add_labels:
labels:
cluster.name: pixl8staging
output.elasticsearch:
hosts: ['https://xxx.xxx.xxx:9200']
l
This works just great in getting all the stdout
from all the pods. However, I have deployments that will have multiple log files and I wanted to configure them with annotations. I have this, in my deployment:
spec:
template:
metadata:
name: xxx
labels:
app: xxx
annotations:
co.elastic.logs/raw: "[{\"type\": \"log\", \"paths\": [\"/var/www/logs/*.log\", \"/opt/lucee/conf/lucee-web/logs/*.log\"] } ]"
spec:
containers:
...
That stringified json looks like this in full:
[
{
"type": "log",
"paths": [
"/var/www/logs/*.log",
"/opt/lucee/conf/lucee-web/logs/*.log"
]
}
]
I still see stdout
logs for my pod in Kibana, but I don't see any content from the logs that are local to the container as defined in the input configuration above.
I have a suspicion that I'm close, but missing some fundamental understanding. Is anyone able to point me in the right direction?
TIA
Dominic