Best data structure for sensor data

Elastic Stack Elasticsearch
1

Hello everyone,

We are setting up a cluster for collecting of IoT/sensor data. And I'm not sure what is the best structure for this kind of data. The simplest way would be to use single index for all similar (numeric) data sources and structure data in few fixed fields:

  • timestamp
  • measure_point_location
  • value
  • unit_code
  • some_meta_data

The other option would be to use field name as measure point location and therefore split data in multiple indexes by area, sensor type or other rule:

  • timestamp
  • room_01_temerature_01
  • room_01_temerature_02
  • room_02_temerature_01
  • room_02_temerature_02
  • room_01_humidity_01
  • room_01_humidity_02
  • ...

The goal is to create different kinds of visualizations in Kibana. Which option is more suitable for that?

Thanks!

Regards

2

Hi @allatrue Welcome to the community!

Of course there's many many details to any one use case.

Personally, I like the first structure.

It will allow easier aggregations and filtering across any of the dimensions.

I'm not a big fan of doing the concatenating of field names I see potential issue as those grow and complexity It also limits your ability to do easier filtering and aggregations.

That's just my high level view.

I would certainly test

2 Likes
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.