Data tranformation

[mathiasa (Mathias Adler)]

Hi,
What is the best way of doing data transformations in the ELK flow.
I want for example transform seconds into minutes. Is there a way of doing that kind of calculations in Kibana? Don't think logstash is the right place. Or should I run updated in ES directly?

What's your take on this?

Br
Mathias

[warkolm (Mark Walkom)]

You could do that in LS, or you could do a scripted field in KB, it depends on your pipeline.

[mathiasa (Mathias Adler)]

Thanks for your input,
I think doing tranformations in LS will make the config to to complex in the long run and also have to much of an performance hit. We have an index rate of 10.000 ind/s so i'm trying to keep LS as streamlined as possible.
How does scripted fields in KB work?

Br
Mathias

[tbragin (Tanya Bragin)]

For transformation from seconds into minutes, scripted fields are indeed a good option. By default scripted fields use Lucene expressions syntax, but that works really well for numerical transformations.

You can find more info on scripted fields in the Kibana interface itself:

[warkolm (Mark Walkom)]

There's also https://www.elastic.co/guide/en/kibana/current/managing-fields.html#create-scripted-field

[mathiasa (Mathias Adler)]

Thanks, works like a charm...

Thanks
Mathias