# Enrichment - is it the best way?

**URL:** <https://discuss.elastic.co/t/enrichment-is-it-the-best-way/294802>\
**Category:** Elasticsearch\
**Created:** [January 19, 2022, 11:21am UTC](https://discuss.elastic.co/t/enrichment-is-it-the-best-way/294802 "2022-01-19T11:21:42Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jkulcsar](https://avatars.discourse-cdn.com/v4/letter/j/9de053/32.png) [@jkulcsar](https://discuss.elastic.co/u/jkulcsar)\
**Post date:** [January 19, 2022, 11:21am UTC](https://discuss.elastic.co/t/enrichment-is-it-the-best-way/294802/1 "2022-01-19T11:21:42Z")

</div>

Hello all,

We've been using Elastic in capturing and consolidating IoT data however - my experience with Elastic is however limited to about 6 months.

Consider the scenario:

- large number of IoT devices with known serial numbers are sending timestamped data
- a separate index contains the `geo_point` for each device (key being the unique serial number)
- data/messages from the devices are enriched with the matching `geo_point` (again, key being the unique serial number)

This is great for creating maps (especially heatmaps).

However is this the best way to do it? Is enriching each and every message with the geo\_point info not wasteful by taking up a lot of space that could be correlated from another index. The expected number of messages from the IoT devices is increasing rapidly (ex: 6 months ago the average was 1M/week, now is at 5M/week).

Any suggestions on how to improve or optimize this would be much appreciated.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 19, 2022, 12:06pm UTC](https://discuss.elastic.co/t/enrichment-is-it-the-best-way/294802/2 "2022-01-19T12:06:15Z")

</div>

As Elasticsearch does not support joins the enrichment approach is the one generally recommended unless you have a custom UI where you can merge data from separate queries (basically perform the join at the application layer).

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [January 19, 2022, 12:24pm UTC](https://discuss.elastic.co/t/enrichment-is-it-the-best-way/294802/3 "2022-01-19T12:24:34Z")

</div>

You could do this in Vega if enrichment is **not** an option. In Vega you can make 2 separate Elastic queries on 2 separate indexes and then join the data using transformations. Then you can render a map using the points. Vega is a steep learning curve.

But I would recommend you enrich the data one time during ingest and then create a visualization off that one index vs the Vega route which would be much less optimized since it would have to do all that querying and joining for each time the map is rendered.

---

<div class="post-metadata">

**Author:** ![jkulcsar](https://avatars.discourse-cdn.com/v4/letter/j/9de053/32.png) [@jkulcsar](https://discuss.elastic.co/u/jkulcsar)\
**Post date:** [January 21, 2022, 8:44am UTC](https://discuss.elastic.co/t/enrichment-is-it-the-best-way/294802/4 "2022-01-21T08:44:27Z")

</div>

Thank you all for your input, much appreciated.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 18, 2022, 8:44am UTC](https://discuss.elastic.co/t/enrichment-is-it-the-best-way/294802/5 "2022-02-18T08:44:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.