# ElasticSearch for higher frequency time series data **URL:** \ **Category:** Elasticsearch\ **Created:** [April 9, 2014, 5:30am UTC](https://discuss.elastic.co/t/elasticsearch-for-higher-frequency-time-series-data/16884 "2014-04-09T05:30:07Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![mikesam460](https://avatars.discourse-cdn.com/v4/letter/m/50afbb/32.png) [@mikesam460](https://discuss.elastic.co/u/mikesam460)\ **Post date:** [April 9, 2014, 5:30am UTC](https://discuss.elastic.co/t/elasticsearch-for-higher-frequency-time-series-data/16884/1 "2014-04-09T05:30:07Z")
1\> Let's assume the time series data comes from distributed nodes like we want to collect CPU counter on 100's of servers in the DC at high frequency(4 times a sec). Does anybody have any experience using ElasticSearch as the time series data storage and query engine? Any gotcha doing so? 2\> Also, is logstash the way to collect and send the data at each host to ES? Thank you, Mike -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 9, 2014, 5:37am UTC](https://discuss.elastic.co/t/elasticsearch-for-higher-frequency-time-series-data/16884/2 "2014-04-09T05:37:28Z")
Any reason for ES and not something like graphite? ES should be able to do it though. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 9 April 2014 15:30, [mikesam460@gmail.com](mailto:mikesam460@gmail.com) wrote: > 1\> Let's assume the time series data comes from distributed nodes like we > want to collect CPU counter on 100's of servers in the DC at high > frequency(4 times a sec). Does anybody have any experience using > Elasticsearch as the time series data storage and query engine? Any gotcha > doing so? > > 2\> Also, is logstash the way to collect and send the data at each host to > ES? > > Thank you, > > Mike > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa\_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![mikesam460](https://avatars.discourse-cdn.com/v4/letter/m/50afbb/32.png) [@mikesam460](https://discuss.elastic.co/u/mikesam460)\ **Post date:** [April 9, 2014, 5:48am UTC](https://discuss.elastic.co/t/elasticsearch-for-higher-frequency-time-series-data/16884/3 "2014-04-09T05:48:11Z")
Well, we are thinking to store both logs and metrics in one place. In theory it could work but I am wondering if anybody has actually tired that? On Tue, Apr 8, 2014 at 10:37 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > Any reason for ES and not something like graphite? > > ES should be able to do it though. > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 9 April 2014 15:30, [mikesam460@gmail.com](mailto:mikesam460@gmail.com) wrote: > > > 1\> Let's assume the time series data comes from distributed nodes like we > > want to collect CPU counter on 100's of servers in the DC at high > > frequency(4 times a sec). Does anybody have any experience using > > Elasticsearch as the time series data storage and query engine? Any gotcha > > doing so? > > > > 2\> Also, is logstash the way to collect and send the data at each host to > > ES? > > > > Thank you, > > > > Mike > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/qkPiqXFTlXw/unsubscribe](https://groups.google.com/d/topic/elasticsearch/qkPiqXFTlXw/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa\_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa\_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Thanks, Mike -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CACiRYOef%2BSf1khEowoHMTXXNzkqFxfnmhzERXEeL%2BAq9-O0S1w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CACiRYOef%2BSf1khEowoHMTXXNzkqFxfnmhzERXEeL%2BAq9-O0S1w%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 9, 2014, 5:52am UTC](https://discuss.elastic.co/t/elasticsearch-for-higher-frequency-time-series-data/16884/4 "2014-04-09T05:52:31Z")
I gave it a test a while ago and it seemed to work ok (using collectd + logstash), that was only with one server though and not at that resolution. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 9 April 2014 15:48, Mike Sam [mikesam460@gmail.com](mailto:mikesam460@gmail.com) wrote: > Well, we are thinking to store both logs and metrics in one place. In > theory it could work but I am wondering if anybody has actually tired that? > > On Tue, Apr 8, 2014 at 10:37 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > > > Any reason for ES and not something like graphite? > > > > ES should be able to do it though. > > > > Regards, > > Mark Walkom > > > > Infrastructure Engineer > > Campaign Monitor > > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > On 9 April 2014 15:30, [mikesam460@gmail.com](mailto:mikesam460@gmail.com) wrote: > > > > > 1\> Let's assume the time series data comes from distributed nodes like > > > we want to collect CPU counter on 100's of servers in the DC at high > > > frequency(4 times a sec). Does anybody have any experience using > > > Elasticsearch as the time series data storage and query engine? Any gotcha > > > doing so? > > > > > > 2\> Also, is logstash the way to collect and send the data at each host > > > to ES? > > > > > > Thank you, > > > > > > Mike > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0bc154cb-f0ad-4480-9177-8f29adb3da4c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/qkPiqXFTlXw/unsubscribe](https://groups.google.com/d/topic/elasticsearch/qkPiqXFTlXw/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa\_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa\_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEM624Zn1XTG3MHBk72Oq4xa_j9q0qkRDNYD0YRktL0%3Dy%3DRt8A%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Thanks, > Mike > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CACiRYOef%2BSf1khEowoHMTXXNzkqFxfnmhzERXEeL%2BAq9-O0S1w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CACiRYOef%2BSf1khEowoHMTXXNzkqFxfnmhzERXEeL%2BAq9-O0S1w%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CACiRYOef%2BSf1khEowoHMTXXNzkqFxfnmhzERXEeL%2BAq9-O0S1w%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CACiRYOef%2BSf1khEowoHMTXXNzkqFxfnmhzERXEeL%2BAq9-O0S1w%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624Y3XGkLcCeOw5dCqdMda%3DWxAT5h%3DSXSSaa\_L2QvzXvz2Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624Y3XGkLcCeOw5dCqdMda%3DWxAT5h%3DSXSSaa_L2QvzXvz2Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:37am UTC](https://discuss.elastic.co/t/elasticsearch-for-higher-frequency-time-series-data/16884/5 "2017-07-06T01:37:28Z")