Find sequences in time series data

Elastic Stack Elasticsearch
1

Hi,

I'm trying to find example queries for returning sequences of events in a time series. My dataset is rainfall values at 10-minute intervals, and I want to find all storm events. A storm event would be considered continuous rainfall for more than 12 hours. This would equate to 72 consecutive records with a rainfall value greater than zero. I could do this in code, but to do so I'd have to page through thousands of records so I'm hoping for a query-based solution.

I'm working in a University research group, so any solutions that involve premium tier licences are probably out due to budget.

Thanks!

2

I've discovered EQL is what I need and have this query

GET /rabt-rainfall-*/_eql/search
{
  "timestamp_field": "@timestamp",
  "event_category_field": "type",
  "size": 100,
  "query": """
    sequence
      [ rainfall where `rain-last-10-mins` > 0 ]
      [ rainfall where `rain-last-10-mins` > 0 ]
    until [ rainfall where `rain-last-10-mins` == 0 ]
  """
}

This returns results but only of length 2, I need to adapt it to find arbitrary sequences when the rainfall value is above zero.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.