How can we transform data while it gets indexed? Some examples please.
The best thing to do is to transform data before it gets indexed!
So use logstash for example.
Ok thanks! In the elasticsearch docs it's mentioned that "As of now there really isn’t a feature to use in its place other than transforming the document in the client application." Could you elaborate on this sentence please. What exactly do they mean by transforming in client application?
This is the link: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-transform.html
Exactly what I meant. That you need to transform your document before sending it to elasticsearch.
So we use the same method for transforming given in the examples in the above link but we only have to do it in logstash?
Not quite - you can use Logstash to achieve the same results as with that method. You can find some typical examples for what transforming data with Logstash looks like here: https://www.elastic.co/blog/little-logstash-lessons-part-using-grok-mutate-type-data
Out of curiosity: What type of data transformation are you looking for exactly? What type of data are you planning to index?
I'll be using log data. I'll have to perform some operations such as converting data from bits to bytes/ seconds to mins etc.
Without having deep Logstash knowledge (might be better to ask in their group in discuss) to me that sounds like something that should be well doable in Logstash.
Thank you anyway.