Hello all,
While investigating a problem with high DNS request rate on Core DNS, I started notice DNS requests from Filebeat to a domain with duplicated .svc in the name, e.g. eck-orchestration-app-elasticsearch-es-http.eck-operator.svc.eck-operator**.svc.cluster.local**. My Filebeat configuration does not contain this duplication:
filebeat:
autodiscover:
providers:
- hints:
default_config:
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log
type: container
enabled: true
node: ${NODE_NAME}
templates: null
type: kubernetes
output:
elasticsearch:
hosts:
- https://eck-orchestration-app-elasticsearch-es-http.eck-operator.svc:9200
ssl:
certificate_authorities:
- /mnt/elastic-internal/elasticsearch-certs/ca.crt
username: [REDUCTED]
setup:
ilm:
enabled: true
policy_name: filebeat-logs
template:
settings:
index:
lifecycle:
name: filebeat-logs
rollover_alias: filebeat-logs-copy
I suspect something in the discovery, configuration or the Helm chart is appending .svc.cluster.local automatically, resulting in high number of invalid DNS queries.
Query to eck-orchestration-app-elasticsearch-es-http.eck-operator.svc does work if I try manually using curl.
Is there a discovery logic, or something else that could causing this duplication, and how can I prevent it?
Thanks!