Deploy & Monitor Filebeat

Elastic Stack Beats
1

Hi all,
the question is very simple, is there a way to deploy/install Filebeat in thousand (25.000/50.000) of machines?
Solutions like puppet, ansible, chef or Docker are very expensive. Maybe with few machines the cost is reasonable, but when the number grows, the cost is disproportionate.

Another question, can I use heartbeat to check the status of Filebeat?

Thank you.

2

What is the tool you use to manage these 50k machines? Can you also share some info on why puppet etc. get very expensive?

For the monitoring best have a look at https://github.com/elastic/beats/issues/3422. Currently with heartbeat monitoring is not possible. Potentially this PR could make it possible but as I don't recommend the port to the outside, you would need heartbeat in addition on each node. https://github.com/elastic/beats/pull/3717

3

Puppet, etc. are expensive because the license is (generally) based on the number of "managed" nodes and here the number is a bit high. On the other hand, Chef for example requires an agent installed on each machine and this is not so elengant.

Currently I don't use any kind of tool, I have only to propose a solution for this problem and eventually use it. But I think it's not so easy.

Thanks.

4

I thought you mean "expensive" from a resource specific point of view and was not thinking of the commercial part of puppet :slight_smile: I have to feeling if you have that many machines you will not some sort of automation tool anyways. As soon as you decided for one, filebeat can be handled very similar to all the other applications you will install.

5

Thank you Ruflin :slight_smile:
But, any plan to integrate this kind of feature in the next versions of Beats? Maybe in the "central configuration management" (What’s brewin’ in Beats, Elastic{on} '17)?

6

The monitoring part is definitively on our agenda. So far we don't have any concrete plans to also have a way to update and ship a binary as this would require to be already on the nodes. And for this tool you would probably need again some automation ...

7

Think about that :slight_smile: because a way to distribute (and update) the Beats (or Elastic stack) components via SSH, maybe with a standalone app, would be the killer feature and it would make really full the stack.

8

This topic was automatically closed after 21 days. New replies are no longer allowed.