# Transforming data WHILE indexing

**URL:** <https://discuss.elastic.co/t/transforming-data-while-indexing/35292>\
**Category:** Logstash\
**Created:** [November 23, 2015, 10:25am UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292 "2015-11-23T10:25:12Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![KavyaS](https://avatars.discourse-cdn.com/v4/letter/k/838e76/32.png) [@KavyaS](https://discuss.elastic.co/u/KavyaS)\
**Post date:** [November 23, 2015, 10:25am UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/1 "2015-11-23T10:25:12Z")

</div>

How can we transform data while it gets indexed? Some examples please.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 23, 2015, 10:37am UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/2 "2015-11-23T10:37:24Z")

</div>

The best thing to do is to transform data before it gets indexed!

So use logstash for example.

---

<div class="post-metadata">

**Author:** ![KavyaS](https://avatars.discourse-cdn.com/v4/letter/k/838e76/32.png) [@KavyaS](https://discuss.elastic.co/u/KavyaS)\
**Post date:** [November 23, 2015, 1:23pm UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/3 "2015-11-23T13:23:03Z")

</div>

Ok thanks! In the elasticsearch docs it's mentioned that "As of now there really isn’t a feature to use in its place other than transforming the document in the client application." Could you elaborate on this sentence please. What exactly do they mean by transforming in client application?

This is the link: [https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-transform.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-transform.html)

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 23, 2015, 1:36pm UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/4 "2015-11-23T13:36:47Z")

</div>

Exactly what I meant. That you need to transform your document before sending it to elasticsearch.

---

<div class="post-metadata">

**Author:** ![KavyaS](https://avatars.discourse-cdn.com/v4/letter/k/838e76/32.png) [@KavyaS](https://discuss.elastic.co/u/KavyaS)\
**Post date:** [November 23, 2015, 2:03pm UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/5 "2015-11-23T14:03:32Z")

</div>

So we use the same method for transforming given in the examples in the above link but we only have to do it in logstash?

---

<div class="post-metadata">

**Author:** ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec)\
**Post date:** [November 23, 2015, 2:10pm UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/6 "2015-11-23T14:10:03Z")

</div>

Not quite - you can use Logstash to achieve the same results as with that method. You can find some typical examples for what transforming data with Logstash looks like here: [https://www.elastic.co/blog/little-logstash-lessons-part-using-grok-mutate-type-data](https://www.elastic.co/blog/little-logstash-lessons-part-using-grok-mutate-type-data)

Out of curiosity: What type of data transformation are you looking for exactly? What type of data are you planning to index?

---

<div class="post-metadata">

**Author:** ![KavyaS](https://avatars.discourse-cdn.com/v4/letter/k/838e76/32.png) [@KavyaS](https://discuss.elastic.co/u/KavyaS)\
**Post date:** [November 23, 2015, 2:19pm UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/7 "2015-11-23T14:19:37Z")

</div>

I'll be using log data. I'll have to perform some operations such as converting data from bits to bytes/ seconds to mins etc.

---

<div class="post-metadata">

**Author:** ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec)\
**Post date:** [November 23, 2015, 2:47pm UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/8 "2015-11-23T14:47:18Z")

</div>

Without having deep Logstash knowledge (might be better to ask in their group in discuss) to me that sounds like something that should be well doable in Logstash.

---

<div class="post-metadata">

**Author:** ![KavyaS](https://avatars.discourse-cdn.com/v4/letter/k/838e76/32.png) [@KavyaS](https://discuss.elastic.co/u/KavyaS)\
**Post date:** [November 23, 2015, 7:10pm UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/9 "2015-11-23T19:10:42Z")

</div>

Thank you anyway.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:21am UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292/10 "2017-07-06T05:21:41Z")

</div>


