Use scripted fields with a big mapping table

Elastic Stack Kibana
1

I would like to use a scripted field with a translation table to add a field from the the CMDB for a given host.

Here is an example Painless script adding a new scripted filed host_service:

HashMap mapping = [
'gnu': 'webserver',
'matrix': 'fileserver'
];
mapping[doc['host.keyword'].value];

The only issue is that we don’t only have 2 servers but thousands so with this approach I hit the size limit of Painless scripts (https://github.com/elastic/elasticsearch/issues/23209). Is there any way to still define the mapping table, maybe load it from some other place? PS: Please, please make the limit configurable, as conquel argued. I know that Elasticsearch can pass parameters to Painless but this is not exposed in Kibana for scripted fields. Hacks are welcome :slight_smile: other than recompiling Elasticsearch, it is still a production environment.

2

If it is a static translation, why not apply this at indexing time, which is likely to perform and scale a lot better than doing this at query time?

3

Thanks for the hint! I should have mentioned that this is such a rare use case that I would rather not have it stored for every log event. Possibly the scripted fields are even "hidden" as separate index pattern in Kibana which can be used when needed. Also, we don’t exactly know yet which fields from the CMDB we might need to "join" for some analysis.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.