# Elasticsearch sequence pattern mining
**URL:** https://discuss.elastic.co/t/elasticsearch-sequence-pattern-mining/88255
**Category:** Elasticsearch
**Created:** [June 5, 2017, 10:24am UTC](https://discuss.elastic.co/t/elasticsearch-sequence-pattern-mining/88255 "2017-06-05T10:24:26Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@vinay\_khandelwal](https://discuss.elastic.co/u/vinay_khandelwal)
#### Post date: [June 5, 2017, 10:24am UTC](https://discuss.elastic.co/t/elasticsearch-sequence-pattern-mining/88255/1 "2017-06-05T10:24:26Z")
I am looking for a way to search patterns in the elasticsearch events.
Let us consider two different query matches that return documents of type A and B.
Is it possible to obtain all documents where an A is followed by a B which is followed by another A? The order is based on timestamps.
From another perspective, I am looking for a way to compare the timestamp field across multiple documents.
---
### Author:  [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)
#### Post date: [June 9, 2017, 11:37am UTC](https://discuss.elastic.co/t/elasticsearch-sequence-pattern-mining/88255/2 "2017-06-09T11:37:36Z")
You can't do that without building another view of your data, ie "entity centric indexing".
To do this you'd build a document that represents all actions (eg you'd convert an index where each document represents a single click to a new index where each document represents a whole user session, with each click stored in the document as a nested object)
Take a look at [https://www.elastic.co/videos/entity-centric-indexing-mark-harwood](https://www.elastic.co/videos/entity-centric-indexing-mark-harwood) for more
---
### Author:  [@vinay\_khandelwal](https://discuss.elastic.co/u/vinay_khandelwal)
#### Post date: [June 14, 2017, 9:20am UTC](https://discuss.elastic.co/t/elasticsearch-sequence-pattern-mining/88255/3 "2017-06-14T09:20:18Z")
Hey thanks for the entity-centric indexing technique. It is almost what I was looking for.
However if the query is really large and complex (like A-\>B-\>......-\>F-\>C-\>N) [around a size of 1500], would it be effective to create entity centric indexes? Also is it efficient/useful if I create the entity centric indexes while querying (i.e not create the entity centric indexes periodically as done by mark in the video)?
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [June 14, 2017, 10:19am UTC](https://discuss.elastic.co/t/elasticsearch-sequence-pattern-mining/88255/4 "2017-06-14T10:19:43Z")
Generally the issue entity-centric is tackling is joining related data and it does so by shifting the costs involved from query time to index time.
If the key you join the data on has many unique values or the business logic in any derived properties is complex [1] then generally you will need to look at doing this to avoid overly-long or complex queries.
> [@vinay\_khandelwal](#):
>
> However if the query is really large and complex (like A-\>B-\>......-\>F-\>C-\>N) [around a size of 1500], would it be effective to create entity centric indexes?
It would certainly be simpler to search for an indexed token that was ABFCN...
> [@vinay\_khandelwal](#):
>
> is it efficient/useful if I create the entity centric indexes while querying (i.e not create the entity centric indexes periodically
When I said "periodically" I did not necessarily mean overnight consolidation jobs. The update job could be run every second to patch in just the latest events. Think about your browser loading this web page - it is a flurry of activity involving many individual requests to get HTML, CSS, javascript, images etc. I wouldn't rush to update your entity-centric `websession` document upon receipt of every individual log record containing your session cookie. I could hang back just a second and perform only one update to your session doc with a batch of maybe 20 log file entries pertaining to your latest activity. This would save 19 Lucene updates. I think of it more as "micro-batching". Of course the sensible duration of a batch will depend on the nature of your system.
Cheers
Mark
[1] For a car the `distance-driven-while-failed` is a property derived from the difference of the mileage reported on the _first_ test result failure on a car followed by the mileage on all test results up to and including a subsequent "pass" test result.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 12, 2017, 10:19am UTC](https://discuss.elastic.co/t/elasticsearch-sequence-pattern-mining/88255/5 "2017-07-12T10:19:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.