# Retention users in ES

**URL:** <https://discuss.elastic.co/t/retention-users-in-es/31537>\
**Category:** Elasticsearch\
**Created:** [October 2, 2015, 11:10am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537 "2015-10-02T11:10:29Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Smasell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smasell/32/43483_2.png) [@Smasell](https://discuss.elastic.co/u/Smasell)\
**Post date:** [October 2, 2015, 11:10am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/1 "2015-10-02T11:10:29Z")

</div>

Hi!!!  
I have documents with user\_id field. When new user comes I get log with :"New user\_id created"  
Is it possible in ES to calculate how many new users in my store visit it again after 7(28) days after.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 3, 2015, 11:32pm UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/2 "2015-10-03T23:32:14Z")

</div>

It should be with an aggregation, can you provide a sample document?

---

<div class="post-metadata">

**Author:** ![Smasell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smasell/32/43483_2.png) [@Smasell](https://discuss.elastic.co/u/Smasell)\
**Post date:** [October 4, 2015, 11:31am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/3 "2015-10-04T11:31:25Z")

</div>

@warkolm  
New user(this document comes only one per a user while he's registration):

```
"_source": {
           "message": "Authorization request, created new account: login: '678886946f9a80a24f407ade11d21ee8'; user_id: 1113124; name: ''; provider: GooglePlay; build_type: 'atc';",
           "@version": "1",
           "@timestamp": "2015-09-21T15:33:59.420Z",
           "host": "132.91.54.125",
           "server": "PRODUCTION",
           "HostName": "RD000D3AB11C64",
           "thread": "45",
           "level": "DEBUG",
           "user_id": 1113124,
           "event": "create_user",
           "build_type": "atc",
           "provider": "GooglePlay"
        },
        "sort": [
           1442849639420
        ]
     },

```

And after that I get other documents with user\_id field, for example:

```
"message": "Cash updated: account: 1115836; name: 'han han'; operation: UnlockArmor; delta_cash0: 0; delta_cash1: 0; delta_cash2: 0; delta_cash3: 0; delta_cash4: 0; cash0: 3000; cash1: 1000; cash2: 3; cash3: 5; cash4: 0;",
           "@version": "1",
           "@timestamp": "2015-09-23T00:01:24.886Z",
           "host": "138.91.54.148",
           "server": "PRODUCTION",
           "HostName": "RD000D3AB11C64",
           "thread": "34",
           "level": "DEBUG",
           "user_name": "han han",
           "user_id": 1115836,
           "operation": "UnlockArmor",
           "credits": 3000,
           "parts": 1000,
           "iron_runes": 3,
           "life_runes": 5,
           "boosters": 0,
           "delta_credits": 0,
           "delta_parts": 0,
           "delta_iron_runes": 0,
           "delta_life_runes": 0,
           "delta_boosters": 0,
           "event": "update_cash"

```

All I want is to calculate all new users per 1 period of time (day, week, etc.) and know how many users come back to my store also in a certain period of time (day,week,etc).

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 4, 2015, 1:29pm UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/4 "2015-10-04T13:29:19Z")

</div>

This may be a use case that could be well served through creating an [entity centric](https://www.elastic.co/elasticon/2015/sf/building-entity-centric-indexes) user index, in which a single document or hierarchy of documents contain information about the user, e.g. creation date and interaction history, and better supports the type of queries you wish to run.

---

<div class="post-metadata">

**Author:** ![Smasell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smasell/32/43483_2.png) [@Smasell](https://discuss.elastic.co/u/Smasell)\
**Post date:** [October 4, 2015, 4:41pm UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/5 "2015-10-04T16:41:57Z")

</div>

@Christian_Dahlqvist  
So without entity centric I can't do this?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 4, 2015, 8:32pm UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/6 "2015-10-04T20:32:57Z")

</div>

Although I was not able to think of any easy way to do it based on my understanding of your requirements, I would not rule out that there are other solutions.

---

<div class="post-metadata">

**Author:** ![NickLi](https://avatars.discourse-cdn.com/v4/letter/n/e495f1/32.png) [@NickLi](https://discuss.elastic.co/u/NickLi)\
**Post date:** [April 5, 2016, 9:18am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/7 "2016-04-05T09:18:58Z")

</div>

Hi, have you solved this problem and how?  
My team encounter the same situation

---

<div class="post-metadata">

**Author:** ![Smasell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smasell/32/43483_2.png) [@Smasell](https://discuss.elastic.co/u/Smasell)\
**Post date:** [April 5, 2016, 9:35am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/8 "2016-04-05T09:35:43Z")

</div>

No I haven't!  
But if you interested in Analytics. I would recommend to you R language with RStudio. They have elastic package(for working with raw data).

---

<div class="post-metadata">

**Author:** ![NickLi](https://avatars.discourse-cdn.com/v4/letter/n/e495f1/32.png) [@NickLi](https://discuss.elastic.co/u/NickLi)\
**Post date:** [April 5, 2016, 11:28am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/9 "2016-04-05T11:28:24Z")

</div>

Do you mean that RStudio can solve this problem?

---

<div class="post-metadata">

**Author:** ![raghvendra](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Post date:** [June 20, 2017, 8:12am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/10 "2017-06-20T08:12:20Z")

</div>

hey . have you guys figured out , how to do retention in Elasticsearch ?. I am also stuck with the same problem .

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [June 20, 2017, 8:58am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/11 "2017-06-20T08:58:05Z")

</div>

This isn't an elasticsearch problem, it's essentially physics.

If you are trying to do analysis of user behaviours it's easiest to do on a user-centric store where all related data is consolidated in one place. The further apart you spread the related data the more costly life becomes.  
Separating related user data on the same disk (as often happens when data is received in time-series) requires lots of disk accesses and/or RAM to link these events together as part of querying.  
Separating related data by distributing it across multiple machines incurs the costs of streaming data over slow networks to link information.

On systems with large volumes of users, each generating many log events these physical linking costs become unbearable at query time. We can't make disks and networks faster, nor can we make RAM cheaper. We have to use techniques like entity-centric indexing to keep the query costs contained.

---

<div class="post-metadata">

**Author:** ![raghvendra](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Post date:** [June 20, 2017, 9:18am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/12 "2017-06-20T09:18:23Z")

</div>

Is there tool which could be useful ?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [June 20, 2017, 9:26am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/13 "2017-06-20T09:26:29Z")

</div>

The scripts and data from the[talk on building entity-centric indexes](https://www.youtube.com/watch?v=yBf7oeJKH2Y) is [here](http://bit.ly/entcent_painless)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2022, 4:07am UTC](https://discuss.elastic.co/t/retention-users-in-es/31537/14 "2022-11-04T04:07:04Z")

</div>


