# Calculate daily new users (never seen before) to a website

**URL:** <https://discuss.elastic.co/t/calculate-daily-new-users-never-seen-before-to-a-website/234404>\
**Category:** Kibana\
**Created:** [May 26, 2020, 7:30pm UTC](https://discuss.elastic.co/t/calculate-daily-new-users-never-seen-before-to-a-website/234404 "2020-05-26T19:30:01Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sanjaysubramanian](https://avatars.discourse-cdn.com/v4/letter/s/bcef8e/32.png) [@sanjaysubramanian](https://discuss.elastic.co/u/sanjaysubramanian)\
**Post date:** [May 26, 2020, 7:30pm UTC](https://discuss.elastic.co/t/calculate-daily-new-users-never-seen-before-to-a-website/234404/1 "2020-05-26T19:30:01Z")

</div>

> [@How to calculate daily new users to a website](https://discuss.elastic.co/t/how-to-calculate-daily-new-users-to-a-website/186214):
>
> We're using Elasticsearch to visualize our website usage data. Each day there are new users coming into our website. How can I calculate the number of new users and visualize it with a line graph?

I referred the above link but cannot understand how to build a visualization in Kibana for daily unique new users (that we have never seen before.

The following is a simplified look at our mappings in Elastic search

```auto
{
    "mappings" : {
        "properties": {
                    "userid": { "type": "keyword" },
                    "yyyy_mm_dd": { "type": "date", "format": "yyyy-MM-dd" },
                    "ts": { "type": "date"}
    }
}

```

**Usecase**  
[eric@cream.com](mailto:eric@cream.com), 2020-03-28, 1585357301265  
[paul@beatles.com](mailto:paul@beatles.com), 2020-03-28, 1585382231269  
[john@beatles.com](mailto:john@beatles.com), 2020-03-28, 1585383569863  
[eric@cream.com](mailto:eric@cream.com), 2020-03-28, 1585414906564

[paul@beatles.com](mailto:paul@beatles.com), 2020-03-29, 1585466637966  
[ginger@cream.com](mailto:ginger@cream.com), 2020-03-29, 1585493882027  
[george@beatles.com](mailto:george@beatles.com), 2020-03-29, 1585486384984

[paul@beatles.com](mailto:paul@beatles.com), 2020-03-30, 1585562310061  
[jackbruce@cream.com](mailto:jackbruce@cream.com), 2020-03-30, 1585571947493  
[eric@cream.com](mailto:eric@cream.com), 2020-03-30, 1585597746891  
[john@beatles.com](mailto:john@beatles.com), 2020-03-30, 1585555104127  
[ginger@cream.com](mailto:ginger@cream.com), 2020-03-30, 1585563504459  
[freddie@queen.com](mailto:freddie@queen.com), 2020-03-30, 1585578397198

**My desired Visualization**

2020-03-28 - 3 unique new users

2020-03-29 - 2 unique new users (george , ginger)

2020-03-30 - 2 unique new users. (jackbruce and freddie)

---

<div class="post-metadata">

**Author:** ![myasonik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/myasonik/32/62369_2.png) [@myasonik](https://discuss.elastic.co/u/myasonik)\
**Post date:** [May 26, 2020, 7:47pm UTC](https://discuss.elastic.co/t/calculate-daily-new-users-never-seen-before-to-a-website/234404/2 "2020-05-26T19:47:13Z")

</div>

Hi @sanjaysubramanian!

Unfortunately, it looks like that older ticket you linked to seems to come to the conclusion that it's currently not possible to do in Kibana.

At that time, this feature wasn't available in ES and so was not possible in Kibana. The underlying feature is now merged in ES ([#43661](https://github.com/elastic/elasticsearch/pull/43661)) but it hasn't been exposed in Kibana yet.

You can track this feature in Kibana under the [Aggregations meta issue](https://github.com/elastic/kibana/issues/58628). (Search for "Cumulative cardinality")

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [May 26, 2020, 7:56pm UTC](https://discuss.elastic.co/t/calculate-daily-new-users-never-seen-before-to-a-website/234404/3 "2020-05-26T19:56:37Z")

</div>

The tricky part is to know if a user is a new user. For that you need not only the actual data point but also have to look into the past.

The issue you referenced suggests an entity centric index which you can build using a [transform](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/transforms.html). The idea is basically: you build a secondary index that pivots your original index to get another view on the data. A _continuous_ transform keeps this view up-to-date, which means it refreshes the secondary index with minimal effort. You find Transform under kibana management.

I suggest to `group_by` username or user\_id. As aggregation you use `min` to find out when the user 1st appeared. Given that, you should be able to build a visualization on top of that (there is probably one problem: you need to adjust the kibana index pattern for the transform destination index regarding the time field). Apart from the users 1st appearance you can build many other useful metrics around your user, e.g. last appearance, number of interactions, unique count (`cardinality`) of devices he used, e.t.c., whatever your data provides.

The transform documentation should get you started.

Good luck!

---

<div class="post-metadata">

**Author:** ![sanjaysubramanian](https://avatars.discourse-cdn.com/v4/letter/s/bcef8e/32.png) [@sanjaysubramanian](https://discuss.elastic.co/u/sanjaysubramanian)\
**Post date:** [May 26, 2020, 9:01pm UTC](https://discuss.elastic.co/t/calculate-daily-new-users-never-seen-before-to-a-website/234404/4 "2020-05-26T21:01:50Z")

</div>

Thank you @Hendrik_Muhs  
Let me try the transform and report back. Thanks a ton !  
warmly  
sanjay

---

<div class="post-metadata">

**Author:** ![sanjaysubramanian](https://avatars.discourse-cdn.com/v4/letter/s/bcef8e/32.png) [@sanjaysubramanian](https://discuss.elastic.co/u/sanjaysubramanian)\
**Post date:** [May 26, 2020, 9:20pm UTC](https://discuss.elastic.co/t/calculate-daily-new-users-never-seen-before-to-a-website/234404/5 "2020-05-26T21:20:42Z")

</div>

@Hendrik_Muhs

I successfully created a transform (and poited it to an index pattern) which essentially is the following query

`SELECT userid, min(ts) from USERS group by userid`

From the documentation, I understand that the transform will keep running at regular intervals to keep updating itself. I assume that visualization built on the index pattern will get updated if we refresh the Visualization ?

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [May 27, 2020, 5:32am UTC](https://discuss.elastic.co/t/calculate-daily-new-users-never-seen-before-to-a-website/234404/6 "2020-05-27T05:32:51Z")

</div>

Great!

Yes if you enabled continuous mode in transform it updates the index (on the transform overview page, mode should be continuous (not batch)). For the visualization you can configure auto-refresh(clock symbol).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 24, 2020, 5:32am UTC](https://discuss.elastic.co/t/calculate-daily-new-users-never-seen-before-to-a-website/234404/7 "2020-06-24T05:32:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
