Metricbeat AWS RDS Module not retrieving db_instance.identifiers

we are using metricbeat (7.10.0) -> AWS module, RDS metrics. The metric data that is pulled looks incomplete. it doesn't retrieve aws.rds.db_instance.engine_name for some and for some it doesn't retrieve aws.rds.db_instance.identifier, or aws.rds.db_instance.identifier, or aws.rds.db_instance.class (see sample screenshot below). What am I missing here?

text- module: aws
  period: 1m
    - rds
    - us-west-2

Debugging for another day with out replies, looks like I am able narrow down that problem to not having the permission - rds:ListTagsForResource. Line : 192-198, It was an most likely an error in our case, though the return statement doesn't make sense. If its a warning, throw a warning, continue and if its an error, throw any error. Returning partially constructed data is incorrect, most likely resulting in the weird behavior i noticed.

reqListTags := svc.ListTagsForResourceRequest(&listTagsInput)
		outputListTags, err := reqListTags.Send(context.TODO())
		if err != nil {
			m.Logger().Warn("ListTagsForResourceRequest failed, rds:ListTagsForResource permission is required for getting tags.")
			dbDetailsMap[*dbInstance.DBInstanceIdentifier] = dbDetails
			return dbInstanceIDs, dbDetailsMap, nil

Another unrelated bug, same class, Line 180-182, looks like an error.

if dbInstance.DBInstanceClass != nil {
			dbDetails.dbStatus = *dbInstance.DBInstanceStatus

The aws.rds.db_instance.engine_name is another mystery still unsolved.

Hello! Thanks for reporting this problem. We consider these fields optional, that's why we logged a warning in the log and still published the metrics. For engine_name, this is actually a dimension from the original RDS CloudWatch metrics. Some of the CloudWatch metrics have EngineName as dimension, some have other fields as dimension. For example: DBInstanceIdentifier, DBClusterIdentifier and etc. Please see the section Amazon RDS Dimensions in for more details. Thanks!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.