Configuration of heartbeat on Scale

Elastic Stack Beats
1

Hello,

I'm evaluating the option of configuring heartbeat to monitor URL's at scale. I have 1000's of endpoints to monitor with a check interval ranging between 30 seconds to a few minutes . I don't have an idea of how many endpoint a single heartbeat agent can handle. So I was looking for a solution where I can dynamically split the list of endpoint to monitor between few nodes and the option to have a central location from where these heartbeat agent can lookup for details for endpoints to configure. Basically a dynamic, distributed and scaleable heartbeat agent configuration rather than have all endpoint configured manually and on single nodes.

I did think about have some database where the endpoint details be updated, devise some script that would lookup that info and split it across a number of nodes. But that's a really vague blueprint I have in mind currently. Anyone had to workaround a similar issue?

--Syd

2

Hi syd05,

To have multiple instances of Heartbeat, you will have to give them different URLs to monitor via a different configuration file for each of them.

Watch out for the limit of open file descriptors on your hosts and per-process, to make sure Heartbeat won't hit some OS limitations.

One thing to also consider is how you want to split the URLs into the configuration, I recommend two options:

  1. Single .yml with a unique monitor per host/URL you want to monitor
  2. Multiple .yml files, one per unique host you want to monitor.

The advantage of #2 is that you have more granular control if you need to reload the configs and the yml files will be smaller (easier to inspect/edit manually).

You can always start with one instance of Heartbeat and observe the performance/resource consumption, if you're not happy then starting adding more Heartbeat instances.

1 Like
3

Hy @syd05,

I also wanted to point out that we do have plans to work on support for lightweight monitors through "projects", which will allow you to sync those HTTP monitors you have using a single command in your local environment or CI. That would make it easier to keep monitors up-to-date.

This is being tracked on Support for lightweight monitors via projects · Issue #542 · elastic/synthetics · GitHub.

1 Like
4

Thanx @TiagoQueiroz for those suggestions
@lucasfcosta - would look up in the concept of Projects and keep an eye on the Github issue

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.