# Make future prediction on current cpu usage data with logstash

**URL:** <https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347>\
**Category:** Logstash\
**Created:** [June 9, 2016, 3:17pm UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347 "2016-06-09T15:17:39Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\
**Post date:** [June 9, 2016, 3:17pm UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/1 "2016-06-09T15:17:39Z")

</div>

I have written a conf file to collect log file data about cpu usage. Is there anyway to do prediction of my cpu usage in the future 30 days with my current cpu usage data in the logstash conf file?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2016, 8:06pm UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/2 "2016-06-13T20:06:42Z")

</div>

Well, Logstash certainly won't predict future values. Has anything in the environment changed since last month? If not, why wouldn't the best prediction of the CPU usage for month _N_+1 be the CPU usage of month _N_?

---

<div class="post-metadata">

**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\
**Post date:** [June 14, 2016, 1:05am UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/3 "2016-06-14T01:05:19Z")

</div>

How about if I get a manual equation to make the prediction for that, then will there be a possible way of doing that with logstash or elasticsearch?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 14, 2016, 5:38am UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/4 "2016-06-14T05:38:53Z")

</div>

I'm not sure exactly what you mean by "manual equation", but neither Logstash nor Elasticsearch are equation solvers. Logstash is a simple data processing pipeline and ES is a search engine.

---

<div class="post-metadata">

**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\
**Post date:** [June 29, 2016, 2:45am UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/5 "2016-06-29T02:45:24Z")

</div>

I have found the moving average function which can be used in kibana. How about if using moving average function to predict future value?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 29, 2016, 5:36am UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/6 "2016-06-29T05:36:45Z")

</div>

Sure, that could work. [Elasticsearch's moving average aggregation documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline-movavg-aggregation.html) discusses this to some degree.

---

<div class="post-metadata">

**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\
**Post date:** [June 29, 2016, 5:43am UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/7 "2016-06-29T05:43:35Z")

</div>

But as from my query  
`.es(index='linux_cpu-*', metric='avg:CPU(%)').movingaverage(10)`

 ![](https://us1.discourse-cdn.com/elastic/original/2X/4/4bfb3c4886f8cf6aebcb4a44d9376331fbdf6baa.png)

It does not seem to work properly.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 29, 2016, 7:10am UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/8 "2016-06-29T07:10:59Z")

</div>

This can happen if the moving average model doesn't match your data or if the choice of parameters isn't optimal. In this particular case maybe the window size is too small. This is not my area of expertise so I won't be able to offer any more help.

---

<div class="post-metadata">

**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\
**Post date:** [July 4, 2016, 10:59am UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/9 "2016-07-04T10:59:57Z")

</div>

I have found from [Moving average aggregation | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline-movavg-aggregation.html) which states that moving average can do predictions. But I am not sure how should I do that.  
Should I just include the following into the logstash output json file for the related index as an output mapping?

> {  
> "the\_movavg":{  
> "moving\_avg":{  
> "buckets\_path": "the\_sum",  
> "window" : 30,  
> "model" : "simple",  
> "predict" : 10  
> }  
> }

This is my json file for logstash

> {  
> "template" : "linux\_cpu-_",  
> "settings" : {  
> "index.refresh\_interval" : "5s"  
> },  
> "mappings" : {  
> "default" : {  
> "\_all" : {"enabled" : true, "omit\_norms" : true},  
> "dynamic\_templates" : [ {  
> "message\_field" : {  
> "match" : "message",  
> "match\_mapping\_type" : "string",  
> "mapping" : {  
> "type" : "string", "index" : "analyzed", "omit\_norms" : true,  
> "fielddata" : { "format" : "disabled" }  
> }  
> }  
> }, {  
> "string\_fields" : {  
> "match" : "_",  
> "match\_mapping\_type" : "string",  
> "mapping" : {  
> "type" : "string", "index" : "analyzed", "omit\_norms" : true,  
> "fielddata" : { "format" : "disabled" },  
> "fields" : {  
> "raw" : {"type": "string", "index" : "not\_analyzed", "ignore\_above" : 256}  
> }  
> }  
> }  
> } ],  
> "properties" : {  
> "@timestamp": { "type": "date" },  
> "@version": { "type": "string", "index": "not\_analyzed" },  
> "geoip" : {  
> "dynamic": true,  
> "properties" : {  
> "ip": { "type": "ip" },  
> "location" : { "type" : "geo\_point" },  
> "latitude" : { "type" : "float" },  
> "longitude" : { "type" : "float" }  
> }  
> }  
> }  
> }  
> }  
> }

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:49am UTC](https://discuss.elastic.co/t/make-future-prediction-on-current-cpu-usage-data-with-logstash/52347/10 "2017-07-06T04:49:46Z")

</div>


