Hello,
I'm building an Elasticsearch service for an IoT application. The purpose of it will be to store different kinds of messages coming from a lot of different devices (hear different sets of fields for each type of message).
I wonder what would the best way to design the whole indexing policy. My first thought would be to create an index dynamically each time a new device is activated for the first time and then store related messages in that index. The issue with that is that the documents in that index will have different fields with different types as a device can send different kind of data (I want the data to be searchable, so no bulk data JSON field or similar).
Would that be a problem? Is it better to have index with homogeneous documents? If yes, how could I do better?
Other options I considered were : Creating an index per message type with all messages from all devices, split per time, or different indexes for each device AND type of message.
Thanks for helping.