# Advice to store measurements

**URL:** <https://discuss.elastic.co/t/advice-to-store-measurements/48394>\
**Category:** Elasticsearch\
**Created:** [April 26, 2016, 8:17am UTC](https://discuss.elastic.co/t/advice-to-store-measurements/48394 "2016-04-26T08:17:36Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![daohodac](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daohodac/32/9235_2.png) [@daohodac](https://discuss.elastic.co/u/daohodac)\
**Post date:** [April 26, 2016, 8:17am UTC](https://discuss.elastic.co/t/advice-to-store-measurements/48394/1 "2016-04-26T08:17:36Z")

</div>

Hello,  
I have some sensors that monitors an engine. I want to store the data to see trends and correlations between data. **The aim of the post is to discuss of the best way of storing the data to do this**.

Basically, a small document could be:

- a `monitored_object_id` (e.g the engine) we can have a hundred of different ids
- a `parameter_name` (e.g the vibration\_axis\_at\_point\_A) we can have thousands of different names
- a `value` (double)
- a `timestamp` (to the ms)
- a list of `tags`

`{ monitored_object_id:'engine_1', parameter_name:'VIB_PHASE_1', value:143.2, timestamp:'2016-04-26T12:21:33Z', tags: ['test', 'PW'] }`

`{ monitored_object_id:'engine_1', parameter_name:'VIB_AMP_1', value:4.2, timestamp:'2016-04-26T12:21:33Z', tags: ['test', 'PW'] }`

I could save this simple document, up to 20 million docs per month (few samples per second)  
currently I did this but not sure it is the right way of doing this. Good for writing over daily indexes like logstash does.

But is it the right way of doing things to search efficiently? Here is the queries I need:

**Parameter over time: easy**  
I want to see the values over time for a `parameter_name`. This is easy, even by making time buckets and aggregate with `stats`. The small documents are perfect

**P1 vs P2: difficult**  
Say I have parameters `VIB_AMP_1` and `VIB_PHASE_1`. I know that I have some measurements at the same timestamp. I'd like to plot `VIB_AMP_1` vs `VIB_PHASE_1`. And here is the difficulty.

What could be the query? How to do that in kibana?

In fact, I could save documents like this  
`{ monitored_object_id:'engine_1', VIB_AMP_1:4.2, VIB_PHASE_1:143.2 timestamp:'2016-04-26T12:21:33Z', tags: ['test', 'PW'] }`

but the documents are very different because at the same timestamp, we may not have the same parameters. And remember, we could have thousand of different keys! plus if we have 2 measurements at 1ms time offset, we have the same problem as before

finally, I could have nested the document. But how to query VIB\_AMP\_1 vs VIB\_PHASE\_1?

`{ monitored_object_id:'engine_1', samples: [{ name:'VIB_AMP_1', value:4.2, }, { name:'VIB_PHASE_1', value:143.2, },], timestamp:'2016-04-26T12:21:33Z', tags: ['test', 'PW'] }`

thank's

---

<div class="post-metadata">

**Author:** ![nkvoll](https://avatars.discourse-cdn.com/v4/letter/n/7993a0/32.png) [@nkvoll](https://discuss.elastic.co/u/nkvoll)\
**Post date:** [April 26, 2016, 11:07am UTC](https://discuss.elastic.co/t/advice-to-store-measurements/48394/2 "2016-04-26T11:07:02Z")

</div>

I've moved this topic to the Elasticsearch forum which is a better suit for this topic.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:56pm UTC](https://discuss.elastic.co/t/advice-to-store-measurements/48394/3 "2017-07-05T22:56:19Z")

</div>