# ES as a timeseries metrics store? **URL:** \ **Category:** Elasticsearch\ **Created:** [August 29, 2013, 1:41am UTC](https://discuss.elastic.co/t/es-as-a-timeseries-metrics-store/13379 "2013-08-29T01:41:13Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Dieter\_Plaetinck](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@Dieter\_Plaetinck](https://discuss.elastic.co/u/Dieter_Plaetinck)\ **Post date:** [August 29, 2013, 1:41am UTC](https://discuss.elastic.co/t/es-as-a-timeseries-metrics-store/13379/1 "2013-08-29T01:41:13Z")
In the most recent announcement ([http://elasticsearch.com/blog/welcome-jordan-logstash/](http://elasticsearch.com/blog/welcome-jordan-logstash/)) the following caught my eye: "Another example is a company which uses Logstash and Elasticsearch not only for all their application logs, but also for all of their application metrics. The ability to tie metrics indicating high CPU usage to a log message of “mmm, we shouldn’t really get here” has proven to be invaluable more than once." I've been looking a lot at dedicated multi-server timeseries metrics stores (opentsdb, kairosdb, blueflood etc) and I wonder how ES compares. Specifically, I'm looking for a system that does: - load distribution across servers (reads and writes), - HA (data in+out should be 100% available in case of node failures) and self healing (replication) in case nodes go down. - the ability to aggregate data in the storage system itself (i.e. given minutely datapoints, give me the per-hour max, or min, or avg, or mean. or histograms across a certain timerange) - easy or trivial to deploy (I know ES kicks ass here) nice to have: - balance data to prevent nodes with unequal disk capacities running full. I'm not sure how appropriate ES is because: - there's no need for flexible schema. datapoints are basically (timestamp, value) pairs. not "documents". no need for unique "id's". I'm concerned about storage space and performance here. (value could be float or int) - fixed intervals means it should be easy to seek to the location of the data, no real indices needed, and in fact this means timestamps don't need to be stored, they are implicit. does this work with ES? I noticed a lot of the information about ES is "top down" (what the features are and how to use them). is there any "bottom-up" material that conveys the internal design of ES, and what makes it applicable (or not) for certain use cases? - If I wanted to build a metrics system with ES, how would I go about it? any tutorials? thanks, Dieter -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [August 29, 2013, 5:56am UTC](https://discuss.elastic.co/t/es-as-a-timeseries-metrics-store/13379/2 "2013-08-29T05:56:58Z")
If you can assign your metrics to a millisecond value and not nanoseconds, you're fine with ES, because milliseconds fit into long data type. Consider your value types you want to store. In most cases you have many types of values in an event, and let it just be the source of the event. Then just create documents per event, with many fields to represent an event. You can tweak a lot about storage space. There is a tradeoff, but by default, the index is compressed. You can create an index of fixed intervals, and store whatever you like in there, a document per fixed interval. Note, ES is weak when it comes to computational aspects of your stored values (for example, performing statistical or sorting procedures on huge amount of values while indexing). Also, there is no stream processing API. Currently, this can be done by query result post processors outside of ES with a pulling client. In the future of ES 1.0, there will be an aggregation framework with many exciting features. I do not fully understand what you mean by "bottom up" material. Do you want to read about Lucene internals or Java implementations and other boring details? Or how an event pump can be built to push data into ES? Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:19am UTC](https://discuss.elastic.co/t/es-as-a-timeseries-metrics-store/13379/3 "2017-07-06T02:19:09Z")