Data tranformation

Elastic Stack Kibana
#1

Hi,
What is the best way of doing data transformations in the ELK flow.
I want for example transform seconds into minutes. Is there a way of doing that kind of calculations in Kibana? Don't think logstash is the right place. Or should I run updated in ES directly?

What's your take on this?

Br
Mathias

#2

You could do that in LS, or you could do a scripted field in KB, it depends on your pipeline.

#3

Thanks for your input,
I think doing tranformations in LS will make the config to to complex in the long run and also have to much of an performance hit. We have an index rate of 10.000 ind/s so i'm trying to keep LS as streamlined as possible.
How does scripted fields in KB work?

Br
Mathias

#4

For transformation from seconds into minutes, scripted fields are indeed a good option. By default scripted fields use Lucene expressions syntax, but that works really well for numerical transformations.

You can find more info on scripted fields in the Kibana interface itself:

Screen Shot 2015-08-31 at 11.44.26 PM.png2868×1416 316 KB

#5

There's also https://www.elastic.co/guide/en/kibana/current/managing-fields.html#create-scripted-field

#6

Thanks, works like a charm... :smile:

Thanks
Mathias

#7