trying to setup servicemonitor for elasticsearch exporter to expose elasticsearch cluster metrics to prometheus, but exporter fails to connect to elasticsearch cluster
in exporter we are passing the service url along with port but it seems it trying to connect localhost [we checked the logs and show that]
we logged in to pods and check the environment variable for url and it shows correct value
we have configured exporter and service exposing metrics to 9108 port and our elasticsearch url is running at 9200 port
logs
level=error ts=2025-02-20T18:23:10.343471202Z caller=clusterinfo.go:188 msg="failed to retrieve cluster info from ES" err="Get "http://localhost:9200/\": dial tcp [::1]:9200: connect: connection refused"
level=info ts=2025-02-20T18:28:10.342907259Z caller=clusterinfo.go:183 msg="providing consumers with updated cluster info label"
level=error ts=2025-02-20T18:28:10.343653485Z caller=clusterinfo.go:267 msg="failed to get cluster info" err="Get "http://localhost:9200/\": dial tcp [::1]:9200: connect: connection refused"
level=error ts=2025-02-20T18:28:10.343710811Z caller=clusterinfo.go:188 msg="failed to retrieve cluster info from ES" err="Get "http://localhost:9200/\": dial tcp [::1]:9200: connect: connection refused"
=========
exporter servicemonitor
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: elasticsearch-monitor
namespace: exporter-test
labels:
app.kubernetes.io/instance: exporter-test
openshift.io/user-monitoring: "true"
spec:
endpoints:
- interval: 30s
port: "http"
namespaceSelector:
matchNames:
- exporter-test
selector:
matchLabels:
app: exporter-test
exporter
apiVersion: apps/v1
kind: Deployment
metadata:
name: exporter-test
namespace: exporter-test
spec:
replicas: 1
selector:
matchLabels:
app: exporter-test
template:
metadata:
labels:
app: exporter-test
spec:
containers:
- name: exporter-test
image: Quay
ports:
- containerPort: 9108
env:
- name: ES_SERVER
value: "http://elasticsearch-test-es-http:9200"
- name: ES_ALL
value: "true" # Query stats for all nodes
- name: ES_INDICES
value: "true" # Query stats for all indices
- name: ES_INDICES_SETTINGS
value: "true" # Query stats for index settings
- name: ES_INDICES_MAPPINGS
value: "true" # Query stats for index mappings
- name: ES_SHARDS
value: "true" # Query stats for shards
- name: ES_SNAPSHOTS
value: "true" # Query stats for snapshots
- name: ES_CLUSTER_SETTINGS
value: "false" # Query stats for cluster settings (disabled)
- name: ES_TIMEOUT
value: "30s" # Timeout for Elasticsearch queries
- name: ES_SSL_SKIP_VERIFY
value: "false" # Do not skip SSL verification
- name: ES_SSL_ENABLED
value: "false" #
resources:
limits:
memory: "500Mi"
cpu: "500m"
requests:
memory: "500Mi"
cpu: "500m"
exporter service
apiVersion: v1
kind: Service
metadata:
name: elasticsearch-exporter-test
namespace: exporter-test
labels:
app: exporter-test
spec:
ports:
- port: 9108
targetPort: 9108
name: http
selector:
app: exporter-test
==============================