How to store bar chart data?

Elastic Stack Elasticsearch
1

Hello,
we are in the middle of restructuring one of our products to use ES instead of MySQL as its data storage. Now we are not sure what would be the best way to model the following data:
Each element has a bar chart that visualizes a value in 10 second steps*, e.g.:

0: -1.6783
10: 20.034
20: 15.2223
30: 10.139

Should we

  1. save that as an object and use the times "0", "10", "20", "30" as the field names within this object, so we'd have "hist.0", "hist.10", "hist.20", "hist.30"?
  2. save the values as an array of nested objects like [{ "bucket":0, "value":-1.6783}, …]
  3. save the array as flattened json
  4. save the key value pairs as a linestring shape?
  5. do something different that you can recommend that we haven't thought of yet?

Thank you in advance!

(*we can't eliminate the possibility that the interval might change in the future.)

PS: I'm under the impression that option 2 might give us the most flexibility for queries or aggregations if we ever need them?

2

Can you attach the full timestamp, instead of an N second increment?

3

I think that might make the processing of the data in our software more difficult as we need to work with the times relative to the element start. What would we gain from that?

4

Let me step back - what sort of data is this? What sort of questions does it help answer/what value does it provide?

I can ask a bunch of questions about different options, but understanding what the higher level value is might make things clearer :slight_smile:

5

We've got measurements for the user reactions over the course of a playout of an audio element. These have been calculated in Python and shall be saved to decrease the loading times of the user interface. Later multiple playouts are aggregated to show the average user acceptance over the last X days on a dashboard.

Thank you for investing some of your time to help us!

6

Ahh ok, now the time offset makes more sense to me :slight_smile:

The second option seems the best, but maybe also include a "offset_value": 10 as well, so that if you do change that offset, it'll be more obvious?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.