# Recommendations to Create Entity-Specific Models with Nested Hits/Documents

**URL:** https://discuss.elastic.co/t/recommendations-to-create-entity-specific-models-with-nested-hits-documents/261705
**Category:** Elasticsearch
**Tags:** transforms
**Created:** [January 20, 2021, 8:50pm UTC](https://discuss.elastic.co/t/recommendations-to-create-entity-specific-models-with-nested-hits-documents/261705 "2021-01-20T20:50:46Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![jasonzissman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasonzissman/32/82629_2.png) [@jasonzissman](https://discuss.elastic.co/u/jasonzissman)
#### Post date: [January 20, 2021, 8:50pm UTC](https://discuss.elastic.co/t/recommendations-to-create-entity-specific-models-with-nested-hits-documents/261705/1 "2021-01-20T20:50:46Z")

</div>

Hi all!

I have event logs from my system that describe the actions that users take. I am trying to devise the best method for storing those events in a user entity model.

For example, my event data looks like this:

```
{time: 2021-01-03, customerId: 1234-5678, action: "logged-in"}
{time: 2021-01-03, customerId: 1234-5678, action: "viewed-item", item-id: 222}
{time: 2021-01-07, customerId: 1234-5678, action: "logged-in"}
{time: 2021-01-07, customerId: 1234-5678, action: "viewed-item", item-id: 444}
{time: 2021-01-07, customerId: 1234-5678, action: "viewed-item", item-id: 555}
{time: 2021-01-11, customerId: 1234-5678, action: "logged-in"}
{time: 2021-01-11, customerId: 1234-5678, action: "viewed-item", item-id: 444}

```

And my desired output is something like this:

```
{
    customerId: 1234-5678,
    productsViewed: [{time: 2021-01-03, item-id: 222},{time: 2021-01-07, item-id: 444},{time: 2021-01-07, item-id: 555},{time: 2021-01-11, item-id: 444}]
	logins: [{time: 2021-01-03}],{time: 2021-01-07},{time: 2021-01-11}]
}

```

Is there a recommended approach using the Elastic stack to transform my event logs into entity-centric models that contain nested (and abbreviated) event data like this?

Elasticsearch transforms do not support aggregations that return hits/documents (e.g. top\_hits), so I do not see a way to leverage transforms to output a nested array of abbreviated events. I do not believe ingest pipelines are the solution, either, as they seem to transform documents more than create entities.

---

<div class="post-metadata">

### Author: ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)
#### Post date: [January 21, 2021, 3:06pm UTC](https://discuss.elastic.co/t/recommendations-to-create-entity-specific-models-with-nested-hits-documents/261705/2 "2021-01-21T15:06:32Z")

</div>

> [@jasonzissman](#):
>
> Elasticsearch transforms do not support aggregations that return hits/documents (e.g. top\_hits), so I do not see a way to leverage transforms to output a nested array of abbreviated events.

For accessing full documents you can to use a `scripted_metric` aggregation, the ootb aggregations work on single fields, not documents.

Top hits/metric won't help you, as you still need to collapse the entries into a document.

My advent calendar post from 2019 does collapsing the way you describe it, it is written in german, but maybe you can use in-built browser translation or a web service to translate it:

> [@Dec 5th, 2019 \[DE\] Weihnachtswünsche zusammenfassen mit Transforms](https://discuss.elastic.co/t/dec-5th-2019-de-weihnachtswunsche-zusammenfassen-mit-transforms/209946):
>
> Weihnachtswünsche zusammenfassen mit Transforms Zu Weihnachten gibt es die schöne Tradition des Wunschzettels, Kinder können hier Ihre Wünsche aufschreiben, aber auch Sorgen und andere Dinge dem Weihnachtsmann mitteilen. In Deutschland gibt es mehrere Weihnachtspostämter, die jährlich hunderttausende Briefe beantworten, noch viel mehr Wunschzettel bleiben wahrscheinlich innerhalb der Familie und landen bei den Eltern oder Großeltern. Stellen wir uns vor, dass auch beim Weihnachtsmann Fachkräfte…

---

<div class="post-metadata">

### Author: ![jasonzissman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasonzissman/32/82629_2.png) [@jasonzissman](https://discuss.elastic.co/u/jasonzissman)
#### Post date: [January 21, 2021, 9:02pm UTC](https://discuss.elastic.co/t/recommendations-to-create-entity-specific-models-with-nested-hits-documents/261705/3 "2021-01-21T21:02:07Z")

</div>

Hi. Thanks so much for the response and the link to your article. You covered my use case perfectly.

When I started exploring transforms, I had written a basic `scripted_metric` aggregation within a transform to accomplish my goal, but it felt like I was fighting against the current. Conceptually I feel like aggregations are designed for summarizing documents, not embedding documents, so I openly wondered if there was a different approach to take. It is encouraging to see others following the same approach that I was contemplating, so I will move forward with `scripted_metric` for now.

Thanks again!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 18, 2021, 9:02pm UTC](https://discuss.elastic.co/t/recommendations-to-create-entity-specific-models-with-nested-hits-documents/261705/4 "2021-02-18T21:02:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
