Scripted fields defined in kibana - when executed?

Elastic Stack Kibana
1

Hi,
I couldn't find the details in the documentation, so I hope somebody here will be able to help; I'm curious when the scripted field defined in Kibana is executed? Is it executed on each query, or does Elasticsearch somehow cache the values?

My queries often result with more than 50,000,000 documents, and I have 5 scripted fields defined in the index pattern. The scripted fields consist of bunch of if's on string fields. I'm bit worried that this causes dashboards to be slow at responses.

Is it recommended, in this scenario, to pre-compute fields before inserting data to elastic search, instead of building scripted fields? I do understand that the advantage of scripted field is the ability to update the value for document at any time, although in my case performance is the higher priority.

Many thanks for the response

2

Hi Marcin,

Scripted fields executes when you trigger a search from Kibana, hence the word "on the fly"

Scripted fields compute data on the fly from the data in your Elasticsearch indices

The scripted field values are computed at query time, so they aren’t indexed and cannot be searched using the Kibana default query language.

For your second query, Scripted field do impact performance as it is expensive.

You should observe the performance differences between with / without scripted field and determine if the performance is up to your prioritized expectation.

Computing data on the fly with scripted fields can be very resource intensive and can have a direct impact on Kibana performance.

Above infomration are from below document

Hope this answer your questions!

3

Thanks a lot, I think you confirmed what I suspected. Do you know if there's a way to persist the result of scripted field so it's only executed when the document is added or cached on the first query, and only re-run the script on demand for existing documents if needed?

4

I believe @flash1293 answered your question below

scripted fields, Kibana adds a layer on top of that and remembers to attach them to each request Kibana is making on your behalf. You can't persist them directly as part the Elasticsearch index.

However if you want to persist the values of your script you can do so by defining an ingest pipeline with a script processor: https://www.elastic.co/guide/en/elasticsearch/reference/7.5/script-processor.html

This won't save the script itself, but the computed values in the Elasticsearch index. This also means you have to re-index your data if you ever change your script, but it will be much more efficient while querying data.

The post is here

5

Also have a look into runtime fields.
They are the inbuild Elasticsearch version of scripted fields.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.