Detecting specific event sequences in data streams

Elastic Stack Elasticsearch
1

Hey there.

I have a datastream in Elasticsearch with mappings similar to this:

{
  "mappings": {
    "properties": {
      "@timestamp": {"type": "date"},  
      "event-type": {"type": "keyword"}, 
      "session": {"type": "keyword"},
      "message": {"type": "keyword"}
    }
  }
}

Given a sequence of documents like below, I need a term aggregation of the messages preceeding the "complete" event-type in the same session , which in this case would be ipsum: 1 and lorem: 1

{
  "event-type":  "recieving", 
  "session":   "1",
  "message":  "lorem", 
},
{
  "event-type":  "recieving", 
  "session":   "1",
  "message":  "ipsum", 
},
{
  "event-type":  "complete", 
  "session":   "1",
},
{
  "event-type":  "recieving", 
  "session":   "2",
  "message":  "lorem", 
},
{
  "event-type":  "recieving", 
  "session":   "1",
  "message":  "ipsum", 
}
{
  "event-type":  "complete", 
  "session":   "2",
}

Any ideas on how to do this within Elasticsearch? or would I need to retrieve all the events in the datastream for manual processing outside Elasticsearch.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.