# Custom service monitoring and autodiscovery

**URL:** https://discuss.elastic.co/t/custom-service-monitoring-and-autodiscovery/329323
**Category:** Beats
**Tags:** docker, metricbeat
**Created:** [April 4, 2023, 1:02pm UTC](https://discuss.elastic.co/t/custom-service-monitoring-and-autodiscovery/329323 "2023-04-04T13:02:49Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![denisk](https://avatars.discourse-cdn.com/v4/letter/d/c77e96/32.png) [@denisk](https://discuss.elastic.co/u/denisk)
#### Post date: [April 4, 2023, 1:02pm UTC](https://discuss.elastic.co/t/custom-service-monitoring-and-autodiscovery/329323/1 "2023-04-04T13:02:49Z")

</div>

Hi,

I have set up metric collection of a number of custom microservices using the http module. Per microservice I've create a separate yaml file, and in each yaml file looks something like this:

```auto
# Each service has its own yaml file like below with custom setup
- module: http
  metricsets :
    - json
  period : 60s
  hosts : ["http://${MY_SERVICE1}:${SERVICE1_PORT}"]
  path: "/management/health"
  namespace : 'service1'
  fields_under_root: true
  fields :
    environment: '${ENVIRONMENT:local}'
    systemname: system-name
    servicename: service-name
    logtype : metricbeat

- module: http
  metricsets :
    - json
  period : 60s
  hosts : ["http://${MY_SERVICE1}:${SERVICE1_PORT}"]
  path: "/management/metrics/my.metric1"
  namespace : 'service1'
  fields_under_root: true
  fields :
    environment: '${ENVIRONMENT:local}'
    systemname: system-name
    servicename: service-name
    logtype : metricbeat

... etc

```

I customized the metricbeat container and put the yaml files in the right location:

```auto
# Metricbeat Dockerfile
FROM docker.elastic.co/beats/metricbeat:8.6.2
COPY my-metricbeat.yml /usr/share/metricbeat/metricbeat.yml
COPY modules.d/* /usr/share/metricbeat/modules.d/
...

```

When I enable my custome custom modules from the start, metricbeat automatically starts polling my services and everything works nicely. However that is not a solid solution for when services are not always running, or if services get replicated and I want to monitor all instances.

So I started with autodiscovery. In the `metricbeat.yml` file I enable autodiscovery:

```auto
metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 30s

metricbeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

```

In the docker-compose file I added these labels to the services:

```auto
    labels:
      - "co.elastic.metrics/hosts=$${data.host}:$${data.port}"
      - "co.elastic.metrics/period=60s"
      - "co.elastic.metrics/module=service1"

```

When I start those services, I can see that metricbeat notices the startup, but doesn't like the configuration:

```auto
{
  "log.level": "debug",
  "@timestamp": "2023-04-04T08:49:51.446Z",
  "log.logger": "autodiscover",
  "log.origin": {
    "file.name": "autodiscover/autodiscover.go",
    "file.line": 203
  },
  "message": "Generated config: {\n \"enabled\": true,\n \"hosts\": [\n \"xxxxx\"\n],\n \"metricsets\": [],\n \"module\": \"my-service1\",\n \"period\": \"60s\",\n \"processors\": [],\n \"timeout\": \"3s\"\n}",
  "service.name": "metricbeat",
  "ecs.version": "1.6.0"
}
{
  "log.level": "debug",
  "@timestamp": "2023-04-04T08:49:51.447Z",
  "log.logger": "autodiscover",
  "log.origin": {
    "file.name": "autodiscover/autodiscover.go",
    "file.line": 290
  },
  "message": "Got a meta field in the event",
  "service.name": "metricbeat",
  "ecs.version": "1.6.0"
}
{
  "log.level": "error",
  "@timestamp": "2023-04-04T08:49:51.447Z",
  "log.logger": "autodiscover",
  "log.origin": {
    "file.name": "autodiscover/autodiscover.go",
    "file.line": 236
  },
  "message": "Auto discover config check failed for config '{\n \"enabled\": true,\n \"hosts\": [\n \"xxxxx\"\n],\n \"metricsets\": [],\n \"module\": \"service1\",\n \"period\": \"60s\",\n \"processors\": [],\n \"timeout\": \"3s\"\n}', won't start runner: no metricsets configured for module 'service1'",
  "service.name": "metricbeat",
  "ecs.version": "1.6.0"
}

```

I was wondering: I have the idea that I'm stretching the setup of metricbeat, because I've created my own module, just by creating a new file in the `modules.d/` directory, with a reused of the `http` module and metricset `json`. Can this setup with autodiscovery actually work, or should I search in a different direction?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 2, 2023, 3:03pm UTC](https://discuss.elastic.co/t/custom-service-monitoring-and-autodiscovery/329323/2 "2023-05-02T15:03:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.