Metricbeat credentials config help

Elastic Stack Beats
,
1

HI Team,

In " metricbeat.yml" configuration file we are using the below values for estrablish the connection to AWS.

metricbeat.modules:
  - module: aws
    period: 300s
    metricsets:
      - lambda
    access_key_id: $${AWS_ACCESS_KEY_ID}
    secret_access_key: $${AWS_SECRET_ACCESS_KEY}
    role_arn: ${ROLE_ARN}

But here the "AWS_ACCESS_KEY_ID" and "AWS_SECRET_ACCESS_KEY" are expired every 90 days so it's may be problem to us to remenber and reproduce the credentils to use it.

So can you please suggest any alternatives we can use in this metricbeat configuration instead of "AWS_ACCESS_KEY_ID" and "AWS_SECRET_ACCESS_KEY".

Thanks in advance!!!

Regards,
Syed

2

Hi @sadik,

I know it sounds obvious, but you could set your access/secret key not to expire, that would be one way to solve it.

However there are other things you can also try, like using role_arn authentication.

Look at our documentation regarding AWS credentials: AWS module | Metricbeat Reference [master] | Elastic it lists all the possible options, you can try them out and see which one works best for you.

Another option would be to automate the credentials generation and restart of Metricbeat every couple of months.

3

Hi @TiagoQueiroz ,

Thanks for your reply, here i miss one step, my aws service wanna connect to K8s service, below is sample one,

processors:
  - add_cloud_metadata:

fields_under_root: true
fields:
  kubernetes.cluster: ${CLUSTER}

So, by using role_arn authentication, how we wanna connect it, please provide any sample articles so it's so helpfull to complete our task.

Thank you!!!

4

You can use the add_kubernetes_metadata processor for that, on the documentation link there is information on how to configure authentication.

5

Hi @TiagoQueiroz ,

I have go through the add_kubernetes_metadata processor documentation but not find any IAM role level authentication configuration in that document which will help to complete the task . So please suggest any other alternatives.

Thank you!!!

6

Yes, it doe not mention IAM role direct, but it does show how you can set the host and kube_config parameters to configure the access to the cluster.

I believe you have a way to access your kubernetes cluster, right? A way to run kubectl and inspect/modify the state of the cluster. Metricbeat can leverage the same configuration file as kubectl to connect to Kubernetes.

Both, host and kube_config, are documented on the link I provided as well as there is an example:

processors:
  - add_kubernetes_metadata:
      host: <hostname>
      # If kube_config is not set, KUBECONFIG environment variable will be checked
      # and if not present it will fall back to InCluster
      kube_config: ~/.kube/config
      default_indexers.enabled: false
      default_matchers.enabled: false
      indexers:
        - ip_port:
      matchers:
        - fields:
            lookup_fields: ["metricset.host"]
      #labels.dedot: true
      #annotations.dedot: true

This will allow Metricbeat to connect to Kubernetes and get the necessary metadata.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.