Hi,
I am using logstash 6.6 docker container with elastic and kibana. for right now I am just messing with the logstash part.
I am trying to output the cloudwatch metrics for CloudFront to stdout debug and unfortunately I don't seem to be getting data. I get data for EC2 so i know in general the plugin works. However, i get nothing for CloudFront.
Here is my example config.
input {
cloudwatch {
metrics => ["Requests"]
region => "us-east-1"
namespace => "AWS/CloudFront"
filters => { DistributionId => E1LUI2I6F05BGW Region=>Global}
statistics => ["Sum"]
access_key_id => "xxxxxxxxxxxxxxxxx"
secret_access_key => "xxxxxxxxxxxxxxx"
}
}
output {
stdout { codec => rubydebug }
}
I see this in the output.
[2019-04-23T23:45:18,894][INFO ][logstash.inputs.cloudwatch] [Aws::CloudWatch::Client 200 0.197245 0 retries] get_metric_statistics(namespace:"AWS/CloudFront",metric_name:"Requests",start_time:2019-04-23 23:30:18 UTC,end_time:2019-04-23 23:45:18 UTC,period:300,statistics:["Sum"],dimensions:[{name:"DistributionId",value:"[FILTERED]"}])
[2019-04-23T23:45:18,992][INFO ][logstash.inputs.cloudwatch] [Aws::CloudWatch::Client 200 0.088867 0 retries] get_metric_statistics(namespace:"AWS/CloudFront",metric_name:"Requests",start_time:2019-04-23 23:30:18 UTC,end_time:2019-04-23 23:45:18 UTC,period:300,statistics:["Sum"],dimensions:[{name:"Region",value:"[FILTERED]"}])
But no statistics like
{
"maximum" => 968845138.0,
"metric_name" => "NetworkOut",
"unit" => "Bytes",
"average" => 728685898.6666666,
"start_time" => 2019-04-23T23:34:54.000Z,
"@version" => "1",
"end_time" => 2019-04-23T23:49:54.000Z,
"period" => 300,
"sample_count" => 6.0,
"sum" => 4372115392.0,
"timestamp" => 2019-04-23T23:34:00.000Z,
"minimum" => 513890069.0,
"InstanceId" => "i-02cb7332f4ded8e12",
"namespace" => "AWS/EC2",
"@timestamp" => 2019-04-23T23:49:54.630Z
}
I have wrote my own script to see if i can get metrics from that distribution id and I do.
cw_conn = boto3.client('cloudwatch',region_name='us-east-1')
cf_response = cw_conn.get_metric_statistics(
Namespace='AWS/CloudFront',
MetricName='Requests',
Dimensions=[
{
'Name': 'DistributionId',
'Value': 'E1LUI2I6F05BGW'
},{
'Name' : 'Region',
'Value': 'Global'
}
],
Period=300,
Statistics=[
'Sum'
],
StartTime=datetime(2019,4,23,23,1,11),
EndTime=datetime(2019,4,23,23,16,11)
)
pprint(cf_response)
{'Datapoints': [{'Sum': 159.0,
'Timestamp': datetime.datetime(2019, 4, 23, 23, 6, tzinfo=tzutc()),
'Unit': 'None'},
{'Sum': 163.0,
'Timestamp': datetime.datetime(2019, 4, 23, 23, 11, tzinfo=tzutc()),
'Unit': 'None'},
{'Sum': 181.0,
'Timestamp': datetime.datetime(2019, 4, 23, 23, 1, tzinfo=tzutc()),
'Unit': 'None'}],
'Label': 'Requests',
'ResponseMetadata': {'HTTPHeaders': {'content-length': '750',
'content-type': 'text/xml',
'date': 'Tue, 23 Apr 2019 23:31:41 GMT',
'x-amzn-requestid': 'f2e3b77c-661f-11e9-8a2a-7fbfd7b895b1'},
'HTTPStatusCode': 200,
'RequestId': 'f2e3b77c-661f-11e9-8a2a-7fbfd7b895b1',
'RetryAttempts': 0}}
but again. no metrics from the cloudwatch logstash plugin