Configuring envoyproxy module for collecting stats

Elastic Stack Beats
,
1

Hi,

I have an envoy, metricbeat and elasticsearch container in my kubernetes cluster. I want to use the envoyproxy module of metricbeat to collect stats and send it to elasticsearch. Configured the module according this page: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-envoyproxy.html

The endpoint is available from metricbeat pod (tried with curl command from the pod, gave a valid response), but it seems that the module cannot pick up and transmit the data (the documents only contain the default fields, but not the envoyproxy). There is no error log entry, checked all the pods.

Also, I have tried with prometheus module, used the /stats/prometheus path, in this case the data has been collected, but the metricbeat index template does not contain the proper mappings for this case.

Does anyone has a tip, how to configure properly the envoyproxy module?
Documentation tells rather nothing...

Cheers!

(I am using ELK 7.3.1 and envoy 1.11.0 versions)

2

According the metricbeat logs, start up is fine, the published event is thin:

2019-09-11T09:53:02.936Z INFO instance/beat.go:421 metricbeat start running.
2019-09-11T09:53:02.937Z DEBUG [cfgfile] cfgfile/reload.go:133 Checking module configs from: /usr/share/metricbeat/conf.d/*.yml
2019-09-11T09:53:02.937Z DEBUG [cfgfile] cfgfile/cfgfile.go:175 Load config from file: /usr/share/metricbeat/conf.d/envoyproxy.yml
2019-09-11T09:53:02.937Z DEBUG [cfgfile] cfgfile/reload.go:147 Number of module configs found: 1
2019-09-11T09:53:02.938Z INFO cfgfile/reload.go:171 Config reloader started
2019-09-11T09:53:02.938Z DEBUG [cfgfile] cfgfile/reload.go:197 Scan for new config files
2019-09-11T09:53:02.938Z DEBUG [cfgfile] cfgfile/cfgfile.go:175 Load config from file: /usr/share/metricbeat/conf.d/envoyproxy.yml
2019-09-11T09:53:02.938Z DEBUG [cfgfile] cfgfile/reload.go:216 Number of module configs found: 1
2019-09-11T09:53:02.938Z DEBUG [reload] cfgfile/list.go:62 Starting reload procedure, current runners: 0
2019-09-11T09:53:02.938Z DEBUG [reload] cfgfile/list.go:80 Start list: 1, Stop list: 0
2019-09-11T09:53:02.939Z DEBUG [reload] cfgfile/list.go:101 Starting runner: envoyproxy [metricsets=1]
2019-09-11T09:53:02.939Z DEBUG [module] module/wrapper.go:118 Starting Wrapper[name=envoyproxy, len(metricSetWrappers)=1]
2019-09-11T09:53:02.940Z INFO cfgfile/reload.go:226 Loading of config files completed.
2019-09-11T09:53:02.940Z DEBUG [module] module/wrapper.go:180 Starting metricSetWrapper[module=envoyproxy, name=server, host=sb4b-envoy:9901]
2019-09-11T09:53:02.981Z DEBUG [processors] processing/processors.go:183 Publish event: {
"@timestamp": "2019-09-11T09:53:02.940Z",
"@metadata": {
"beat": "metricbeat",
"type": "_doc",
"version": "7.3.0"
},
"service": {
"address": "sb4b-envoy:9901",
"type": "envoyproxy"
},
"event": {
"module": "envoyproxy",
"duration": 40872608,
"dataset": "envoyproxy.server"
},
"agent": {
"type": "metricbeat",
"ephemeral_id": "b6995a49-c5df-473b-b964-f82b45d711e5",
"hostname": "sb4b-metricbeat-db86cfbc6-t9wls",
"id": "735168d8-aa9d-432f-a5a3-5310a1d808cb",
"version": "7.3.0"
},
"ecs": {
"version": "1.0.1"
},
"host": {
"name": "sb4b-metricbeat-db86cfbc6-t9wls"
},
"metricset": {
"name": "server"
}
}

3

I am trying to setup the metricbeat for envoyproxy also, (I am using ELK 7.3.1 and envoy 1.2.4 versions). in the logs for the metricbeat-metrics pod im getting: [metricbeat-metrics-74f488d4f4-94lqh metricbeat] 2019-09-12T18:49:21.230Z INFO module/wrapper.go:247 Error fetching data for metricset envoyproxy.server: error in http fetch: HTTP error 503 in : 503 Service Unavailable

4

Hi @ceastman-ibm,

It seems to me that you have a slightly different (authorization) problem.
Have you checked that the envoyproxy stats endpoint is available from the metricbeat pod?
Try something like:

kubectl exec -it [metricbeat pod name] -n [namespace] -- /bin/bash

When you in the metricbeat pod, ping the envoy stats endpoint.

5

Hi @somi,

can you post your configuration file here?

@ceastman-ibm

same with you, although it looks like you are reaching an unconfigured virtualhost at envoy, smells like an envoy configuration issue.

6

Hi @pmercado,
thanks for your reply, here are the anonymised configs:

metricbeat.yml

logging.level: info
logging.metrics.enabled: false
metricbeat.config:
  modules:
    path: ${path.config}/conf.d/*.yml
    reload.enabled: false
    reload.period: 10s
output.elasticsearch:
  hosts:
  - elastic-host:9200
  index: metricbeat-%{+yyyy.MM.dd}
  protocol: http
output.file:
  enabled: false
setup.dashboards.enabled: false
setup.ilm.enabled: false
setup.kibana.host: kibana-host:5601
setup.kibana.path: /monitor
setup.kibana.protocol: http
setup.template.enabled: true
setup.template.name: metricbeat-%{[agent.version]}
setup.template.pattern: metricbeat-*

envoyproxy.yml

- hosts:
  - http://envoy-host:9901
  metricsets:
  - server
  module: envoyproxy
  namespace: default
7

Hi @pmercado,
I still cannot figure it out. Have you got a solution for me?
Cheers,
somi

8

Long silence...

If you have similar issue, please vote this ticket:

9

I will not let this to be locked.

10

Hi @somi,

It looks like this related to your Envoy Proxy version. The envoyproxy module is tested with Envoy 1.7.0. I checked new versions of Envoy Proxy and it has new metrics. Because of this envoy proxy module publishes events without data. I am not sure why there is no error log entry, tested locally with envoy v1.11.0 and it shows an error message in logs.

I updated the Envoyproxy module with Envoy Proxy v1.12.0, you can follow this PR.
I hope this will be a solution to your issue.

Best regards

1 Like
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.