How to index state changes?

[NiclasK]

Hi,

I have a process that I want to monitor/analyze using elastic search. The process takes place on thousand's of embedded devices. I am wondering what is the preferred format to send the data to elastic search to have flexibility to look at different properties.
My initial idea is to send json a series of documents like
{
"deviceId": "abc123",
"seqId": 1,
"state": "stat_1",
"prevState": "state_0",
"date": "2019-02-11T18:25:43",
"measuredValue": 1.27
}
Every time there is a state change in the process a message like the one above will be sent with the new and old state and some other data. Once the process finishes it starts over, but with a new seqId.
I would like to be able look (both for one device or many devices) at e.g.:

• How long time that is spent in different states.
• How long time between any two states.
• make histogram over measured values

I'm not sure how to add this data to elastic search, especially how to be able to calculate the time difference. Or is there a better format to send the data in?

Thanks!

[Mark_Harwood]

Analysing behaviour of many actors often requires an entity centric index alongside your event-centric log index

[NiclasK]

Thanks, I will have a look at that!