# MetricBeat and CloudWatch Graphs not matching up

**URL:** <https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [February 6, 2017, 1:18pm UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057 "2017-02-06T13:18:06Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![geek876](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geek876/32/9329_2.png) [@geek876](https://discuss.elastic.co/u/geek876)\
**Post date:** [February 6, 2017, 1:18pm UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/1 "2017-02-06T13:18:06Z")

</div>

Hi,  
We have been using Kibana/ES since few months now and as TimeLion is now built-in in ES5 it makes a great combination. Are thinking of using Beats (metric-beats) to start with on all our servers and hence have been comparing (and learning really).

When I see MetricBeat TimeLion output v/s CloudWatch , they don't seem to match up. ![](https://us1.discourse-cdn.com/elastic/original/2X/8/892f176ccd16bf7fdd9cd5e9b7a898cea9d94a13.png) ![](https://us1.discourse-cdn.com/elastic/original/2X/e/e7504bb3789c3b19c3717469e7df26e06bcfdac1.png)

As seen above,

CloudWatch: 12:46 -\> Value is 7204  
TimeLion : 12:46 -\> Value is 13380

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [February 6, 2017, 4:10pm UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/2 "2017-02-06T16:10:51Z")

</div>

Do you have more than one host or more than one interface reporting system network info to Elasticsearch? You could select one host with `q=beat.name:somehost` and split the graph by interfaces using `s=system.network.name:5` and then see how the values compare.

---

<div class="post-metadata">

**Author:** ![geek876](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geek876/32/9329_2.png) [@geek876](https://discuss.elastic.co/u/geek876)\
**Post date:** [February 6, 2017, 4:29pm UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/3 "2017-02-06T16:29:21Z")

</div>

@andrewkroh Thanks. I only have one host at the moment reporting via metricbeat and that host has only eth0. Regardless, I have now two graphs (1) Split and (2) Without split. Both rendering same values

 ![](https://us1.discourse-cdn.com/elastic/original/2X/1/1345d142ba40f7d100bdbea063ea38cbd3561e40.png)

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [February 6, 2017, 5:15pm UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/4 "2017-02-06T17:15:00Z")

</div>

Can you also change CloudWatch to show 1 minute periods to match what Timelion is showing (or vice versa)? And then post the two graphs over the same exact 1 hour period.

What OS is Metricbeat running on?

---

<div class="post-metadata">

**Author:** ![geek876](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geek876/32/9329_2.png) [@geek876](https://discuss.elastic.co/u/geek876)\
**Post date:** [February 6, 2017, 10:12pm UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/5 "2017-02-06T22:12:30Z")

</div>

@andrewkroh. OS is Ubuntu 15.10. Graphs Attached.

 ![](https://us1.discourse-cdn.com/elastic/original/2X/7/7f2d74a40c7437cf743df72db85c36ed4dea6a80.png) ![](https://us1.discourse-cdn.com/elastic/original/2X/4/41a5d6c2502ea94148113d20816b13658fee4c08.png)

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [February 7, 2017, 4:21pm UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/6 "2017-02-07T16:21:46Z")

</div>

I'm not too familiar with CloudWatch metrics, but I've been reading their docs. The default collection interval is 5 minutes, but you can increase it to 1 min to get more resolution. Metricbeat is collecting at 10 sec intervals by default. The differences could be caused by aliasing. Maybe you can try increasing the sampling interval to 1 min for the CloudWatch.

Is there a way to get the raw samples from CloudWatch in tabular form to compare to the raw samples from Metricbeat? Metricbeat is reporting the raw counter values from /proc/net/dev. CloudWatch is collecting its metrics from EC2 (not from a host agent). Since you have only one NIC I would expect the raw values to be similar (minus differences caused by sampling intervals not being aligned).

> NetworkPacketsIn  
> The number of packets received on all network interfaces by the  
> instance. This metric identifies the volume of incoming traffic in  
> terms of the number of packets on a single instance. This metric is  
> available for basic monitoring only.

---

<div class="post-metadata">

**Author:** ![geek876](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geek876/32/9329_2.png) [@geek876](https://discuss.elastic.co/u/geek876)\
**Post date:** [February 8, 2017, 11:49am UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/7 "2017-02-08T11:49:58Z")

</div>

There is actually a cli to get cloudwatch logs.

`aws cloudwatch get-metric-statistics --metric-name NetworkPacketsIn --namespace AWS/EC2 --statistics Maximum --dimensions Name=InstanceId,Value=xxxx --start-time 2017-02-07T11:00:00 --end-time 2017-02-07T12:00:00 --period 60`

````auto
2017-02-07T11:07:00Z : 31980.0
2017-02-07T11:12:00Z : 16971.0
2017-02-07T11:17:00Z : 34274.0
2017-02-07T11:22:00Z : 46335.0
2017-02-07T11:27:00Z : 8211.0
2017-02-07T11:32:00Z : 14940.0
2017-02-07T11:37:00Z : 37422
2017-02-07T11:42:00Z : 20721
2017-02-07T11:47:00Z : 9094.0
2017-02-07T11:52:00Z : 18393.0
2017-02-07T11:57:00Z : 27123.0```

MetricBeat Graph for the same duration<img <img src="//cdck-file-uploads-us1.s3.dualstack.us-west-2.amazonaws.com/elastic/original/2X/a/aa6ee02ec2a1ddee85f87b00cd1d7a0fb5644800.png" width="690" height="226">
````

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [February 10, 2017, 8:50am UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/8 "2017-02-10T08:50:02Z")

</div>

Did you try to compare the raw values from the exact same time in the discover tab?

---

<div class="post-metadata">

**Author:** ![geek876](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geek876/32/9329_2.png) [@geek876](https://discuss.elastic.co/u/geek876)\
**Post date:** [February 13, 2017, 10:11am UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/9 "2017-02-13T10:11:22Z")

</div>

Hi,

The raw values actually do match up with Metricbeat (more closely) compared to CloudWatch.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [February 14, 2017, 11:59am UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/10 "2017-02-14T11:59:33Z")

</div>

@geek876 Does that mean they are close enough to come to the conclusion, values are the same? If yes, that means metricbeat mainly gives a more detailed picture because it collects more data points?

---

<div class="post-metadata">

**Author:** ![geek876](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geek876/32/9329_2.png) [@geek876](https://discuss.elastic.co/u/geek876)\
**Post date:** [February 14, 2017, 12:18pm UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/11 "2017-02-14T12:18:18Z")

</div>

@ruflin. Yes

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 14, 2017, 12:18pm UTC](https://discuss.elastic.co/t/metricbeat-and-cloudwatch-graphs-not-matching-up/74057/12 "2017-03-14T12:18:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
