Design structure for similar mappings with small data type differences


(Felix König) #1

This is more of a Best Practices question than a hard problem, but I still want to know what the recommended solution is. My Problem is the following:

I am trying to store multiple kinds of timeseries data. Each point consists of some metadata, a timestamp, and a value. The metadata and timestamp are identical schema-wise for all timeseries. However the values differ in data type, for example one series might consist of longs, and another of doubles.

If I want to keep the name of the value field consistent, e.g. just call it value, I can't have each timeseries' mappings live in the same index, as same fields must have the same data type per index. So I would create one index per mapping. If I do this, the fields that could be shared, aren't anymore. Also I wouldn't need any types and just call them all default or something.

The other option is to give each field a different name, e.g. value_double, value_long and put the mappings into separete types like double and long. Can this cause (performance) problems regarding the sparsity of all those fields in one index? Sounds like lots of missing fields per document might "pollute" the index, if that even is a thing.

What do you suggest?


(Felix König) #2

Turns out elastic is removing types in the long term: https://github.com/elastic/elasticsearch/issues/15613
I did not know about this before (though I have studied the documentation very rigorously the last few weeks), but that pretty much settles it. I will go with multiple indices


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.