I'm trying to write a custom filter in Kibana Discover tab using ES Query DSL. I want to classify instances within a particular time period. Assume each document has following fields, I want to find all instances between 2018/12/18 to 2018/12/19 such that its hash doesn't exist before 2018/12/18. I will classify those instances as new instances. I think I will need to do an aggregation to get the timestamp of the first/oldest document that has a particular hash and do a comparison on the timestamp. Is it possible to achieve in the Kibana custom filter? How should I properly write the ES query?
I think if each document can store the timestamp of the oldest/first document temporarily, then we can do a comparison between oldest_timestamp and its timestamp.
Not sure what kinda of query I should use to classify those instances. Any suggestions will be a great help! Thanks!
doc_ID: 123 hash: abchd (not unique) timestamp: 2018/12/18