# Conditionally aggregating documents in an index?
**URL:** https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560
**Category:** Elasticsearch
**Created:** [March 8, 2019, 9:43pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560 "2019-03-08T21:43:24Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@propel](https://discuss.elastic.co/u/propel)
#### Post date: [March 8, 2019, 9:43pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/1 "2019-03-08T21:43:24Z")
I'm trying to conditionally aggregate documents in my index based on the existence of another document in the same set. Imagine the following series:
```
{"action": "pressed_submit", "user": 1, "timestamp": "2016-01-01 00:00:00", "meta": ...}
{"action": "answered_question", "user": 1, "timestamp": "2016-01-01 00:00:00", "meta": ...}
{"action": "answered_question", "user": 1, "timestamp": "2016-01-01 00:00:00", "meta": ...}
{"action": "filled_textbox", "user": 1, "timestamp": "2016-01-01 00:00:00", "meta": ...}
{"action": "pressed_submit", "user": 1, "timestamp": "2012-01-01 00:00:00", "meta": ...}
```
And then the following:
```
{"action": "answered_question", "user": 2, "timestamp": "2016-01-01 00:00:00", "meta": ...}
{"action": "answered_question", "user": 2, "timestamp": "2016-01-01 00:00:00", "meta": ...}
{"action": "filled_textbox", "user": 2, "timestamp": "2016-01-01 00:00:00", "meta": ...}
{"action": "pressed_submit", "user": 1, "timestamp": "2012-01-01 00:00:00", "meta": ...}
```
How can I find the last event done by each user that has not "pressed\_submit"? Meaning, how can I find the last action by any user that hasn't pressed\_submit? Note the timestamps.
I've been cracking my head around this for a while. While I've managed to solve the problem by querying ALL events and then filtering in Python code, it's very slow. Is there any way to use ElasticSearch's query engine to get results like that?
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [March 8, 2019, 10:02pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/2 "2019-03-08T22:02:03Z")
Behavioural analysis at scale typically requires an entity centric index - see [https://twitter.com/elasticmark/status/1009380268409610240?s=21](https://twitter.com/elasticmark/status/1009380268409610240?s=21)
---
### Author:  [@propel](https://discuss.elastic.co/u/propel)
#### Post date: [March 8, 2019, 10:26pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/3 "2019-03-08T22:26:26Z")
I'm sorry, but I didn't understand your answer. I've watched the linked video, but I still don't understand what the centric index would be in this scenario.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [March 8, 2019, 10:32pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/4 "2019-03-08T22:32:04Z")
Users and their last events?
---
### Author:  [@propel](https://discuss.elastic.co/u/propel)
#### Post date: [March 8, 2019, 11:11pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/5 "2019-03-08T23:11:56Z")
That's already there. The JSON sample I sent is a single document type, in a single index.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [March 8, 2019, 11:17pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/6 "2019-03-08T23:17:23Z")
Those documents are events.
An entity centric document would have the id field (think primary key) of the user id.
---
### Author:  [@propel](https://discuss.elastic.co/u/propel)
#### Post date: [March 8, 2019, 11:18pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/7 "2019-03-08T23:18:47Z")
Is there any tutorial on how to create entity-centric indexes?
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [March 8, 2019, 11:19pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/8 "2019-03-08T23:19:16Z")
The video I shared along with the example code.
---
### Author:  [@propel](https://discuss.elastic.co/u/propel)
#### Post date: [March 8, 2019, 11:20pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/9 "2019-03-08T23:20:22Z")
Yes, but the example implies having to re-index data. Unfortunately I do not control the application that writes to the ES index: it is proprietary third-party software.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [March 8, 2019, 11:25pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/10 "2019-03-08T23:25:00Z")
The example advocates keeping your existing event-centric index and building a secondary entity-centric index from it.
---
### Author:  [@propel](https://discuss.elastic.co/u/propel)
#### Post date: [March 8, 2019, 11:27pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/11 "2019-03-08T23:27:21Z")
But it revolves around the idea of having to recreate the "reviewers" index, or update it, with a cron job or everytime I need a report. So, whenever I need to run my query, I have to make sure that buildEntities.sh was ran recently. Correct?
That is, effectively, slower than processing the data in my programming language of choice.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [March 8, 2019, 11:32pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/12 "2019-03-08T23:32:44Z")
At 31:30 in the video I talk about incremental updates
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 6, 2019, 5:07pm UTC](https://discuss.elastic.co/t/conditionally-aggregating-documents-in-an-index/171560/14 "2019-04-06T17:07:07Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.