# Custom metrics from bunch of servers

**URL:** <https://discuss.elastic.co/t/custom-metrics-from-bunch-of-servers/250209>\
**Category:** Elasticsearch\
**Created:** [September 28, 2020, 2:32pm UTC](https://discuss.elastic.co/t/custom-metrics-from-bunch-of-servers/250209 "2020-09-28T14:32:37Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rmrfchik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmrfchik/32/76332_2.png) [@rmrfchik](https://discuss.elastic.co/u/rmrfchik)\
**Post date:** [September 28, 2020, 2:32pm UTC](https://discuss.elastic.co/t/custom-metrics-from-bunch-of-servers/250209/1 "2020-09-28T14:32:37Z")

</div>

I have a number of different servers (php, java...). All of the servers uses different user tracking cookies and are behind nginx.  
I already have access.log with all cookies I need and all of them are feeded to ES with logstash.

Now I want to analyze logs to see metrics: unique visitors per page, unique user\_agents per page and so on. In RDMBS I would create view with complex SELECT to distinguish visitors. Not sure this is right way in ES.

Should I build/calculate these metrics outside ES or there is good way to do it in ES?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 28, 2020, 11:22pm UTC](https://discuss.elastic.co/t/custom-metrics-from-bunch-of-servers/250209/2 "2020-09-28T23:22:13Z")

</div>

Elasticsearch aggregations should do what you want. Have a look at things like the terms aggregation.

You can build all of these in Kibana as graphs too.

---

<div class="post-metadata">

**Author:** ![rmrfchik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmrfchik/32/76332_2.png) [@rmrfchik](https://discuss.elastic.co/u/rmrfchik)\
**Post date:** [September 29, 2020, 8:36am UTC](https://discuss.elastic.co/t/custom-metrics-from-bunch-of-servers/250209/3 "2020-09-29T08:36:09Z")

</div>

Seems like I didn't express all difficulties I see.  
Each server has different cookies to track user (PHPSESSID, JSESSSIONID, metrics and such). Many of users don't like to be tracked and has no cookies enabled.  
So I have access.log line with all common fields and number of cookies I intent to use to track.

To distinguish visitor, I check "main" cookie, say \_ym\_id (from yandex metrica), if it not set then BX\_USER\_ID, if it not set, then PHPSESSID and lastly ip with user-agent.

I'm afraid this aggregation may be difficult for ES under heavy load. Or this is normal way?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 27, 2020, 8:36am UTC](https://discuss.elastic.co/t/custom-metrics-from-bunch-of-servers/250209/4 "2020-10-27T08:36:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
