Metricbeat-Filebeat Install & Run as Docker Container

Elastic Stack Beats
,
1

Task Objective: using aws module (metricbeat + filebeat), I would like to pull logs and metrics from multiple aws accounts from one server.
Thus reducing the server cost of running beats in each aws account.

To achieve this: Need to install Metricbeat and Filebeat on a docker container running multiple containers of beats that pull logs/metrics from different aws accounts.

Can anyone please guide (or) provide a walk-through on how to implement on a docker
(or) is there a strategy to pull aws metrics and aws cloudtrail logs from multiple aws accounts while running the beats from one server?

2

Main question here is why placing docker in the ecuation? Just run each Beat once (Filebeat and Metricbeat) with as many configurations as you need.

The question is: strategy to run Metricbeat and Filebeat in one server. Well, the constrains are really hard :smiley: I mean, just run them :smile:

Or I don't understand the question :sweat_smile:

3

Hi @Mario_Castro ,

Reason for running multiple beats for metricbeat and filebeat:

  • Via these two agents I would like to pull aws-metrics and aws-cloudtrail logs from multiple aws accounts using IAM-Role (profile name) from one single server rather than running the beats in each aws-account.

So, when I configured metricbeat as below - it only honoured only the first profile-name of same metricset. Reference yml snippet below:

metricbeat.modules:
- module: aws
  period: 300s
  credential_profile_name: <aws-account-1 profile name>
  metricsets:
    - ec2
- module: aws
  period: 300s
  credential_profile_name: <aws-account-2 profile name>
  metricsets:
    - ec2
- module: aws
  period: 300s
  credential_profile_name: <aws-account-1 profile name>
  metricsets:
    - sqs
- module: aws
  period: 300s
  credential_profile_name: <aws-account-2 profile name>
  metricsets:
    - sqs

Here I saw the logs for ec2 and sqs metricsets being pulled only from account-1 and not from account-2

So, I thought of running multiple beats parallel. But if my understanding/config is incorrect I would sincerely request a help on this objective

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.