Don't see the metrics I expect

[mcco0l (walter)]

Hi there,
I'm trying to get some Fargate metrics using Cloudwatch namespaces, but I don't see a way to get them to Kibana.
modules.d/aws.yml

    - module: aws
      period: 5m
      access_key_id: 'xxxxxxxxxxxxx'
      secret_access_key: 'xxxxxxxxxxxxxxxxxxxxxxx'
      metricsets:
        - rds
        - usage
        - ebs
        - ec2
        - cloudwatch
      metrics:
        - namespace: "*"

in the logs output I can see this, but not sure how to check this in kibana:

      "aws": {
        "ecs": {
          "metrics": {
            "MemoryUtilization": {
              "avg": 15.336588541666666,
              "max": 15.4296875,
              "min": 15.234375,
              "sum": 153.36588541666666,
              "count": 10
            },
            "CPUUtilization": {
              "sum": 5.197281962927915,
              "count": 10,
              "avg": 0.5197281962927915,
              "max": 1.1245832656975834,
              "min": 0.2879062617197633
            }
          }
        },
        "cloudwatch": {
          "namespace": "AWS/ECS"
        },
        "dimensions": {
          "ServiceName": "web-service",
          "ClusterName": "tt-ecs"
        }
      },
      "event": {
        "dataset": "aws.cloudwatch",
        "module": "aws",
        "duration": 120682193
      },
      "metricset": {
        "name": "cloudwatch",
        "period": 300000
      }
    }

How would I be able to visualize these metrics?

[mtojek (Marcin Tojek)]

Did you integrate filebeat with Elasticsearch? See: Filebeat quick start: installation and configuration | Filebeat Reference [master] | Elastic

[mcco0l (walter)]

Hi Marcin,
so you mean I should install filebeat inside the containers running on ECS? I've installed firelens container with fluentbit as AWS recommends for log streaming to fluentd.

[mtojek (Marcin Tojek)]

Could you please post configuration for filebeat? I don't see where did you configure the connection to Elasticsearch.

[mcco0l (walter)]

not using filebeat but metricbeat to get the metrics from AWS Cloudwatch AWS/ContainerInsights namespace.
Metricbeat configuration is pretty simple,

metricbeat.config.modules:

  path: ${path.config}/modules.d/*.yml

reload.enabled: yes

setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression

setup.kibana:
output.elasticsearch:
  hosts: ["localhost:9200"]

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
  - add_observer_metadata: ~

logging.level: debug
logging.to_files: true
logging.files:
  path: /var/log/metricbeat
  name: metricbeat
  keepfiles: 7
  permissions: 0644

[mtojek (Marcin Tojek)]

Sorry misread the fact that you're using metricbeat. Actually this is the snippet I was looking for:

output.elasticsearch:
  hosts: ["localhost:9200"]

Could you please check if any data land in the ES cluster? As you said you can see outputted logs (process stdout?)