Is it possible to replace string on the fly?

Elastic Stack Kibana
1

Hi Experts,

I have a requirement actually my data holds value like
Alert=Excessive Firewall Denies Between Hosts Detected more than 400 firewall deny attempts from a single source to a single destination within 5 minutes. This often indicates a service that was once used is now being blocked by a firewall.

Since 80% of my data holds this value so I want to replace it with lets say "abc" or "001" so that it consumes less disk space , now at the search level in kibana 001 or abc should be converted back to original string . Hope it makes sense .

Please suggest how I can achieve this

Thanks
Vikas

2

Today you could write a custom field formatter: https://www.elastic.co/blog/kibana-custom-field-formatters

Once we add support for Elasticsearch's new Painless scripting language in 5.x you could also probably achieve this with a simple scripted field. GH issue to track if you're interested: https://github.com/elastic/kibana/issues/6529

1 Like
3

Thank you @Bargs I found it very interesting kiaban custome field but this is not what I want. I am sorry if I was not clear with my query , actually purpose here to reduce the disk space (I have 80 million records holding this huge string and they consumes more than 300 GB), so before indexing if i can change the whole big string value to a small word and then on the fly at kibana level I can change it back to original huge string .

As you suggested may be in kiaban 5.x this is achievable with scripted fields.

4

Hi @vikas_gopal. I think you could create a custom field formatter so that when it sees "abc" in the actual document, it displays it as the full string in the Kibana UI. This would allow you to store a smaller string while at the same time displaying the full message in the UI. Does that make sense?

1 Like
5

Ah!! I get it, will give it a shot Thank you so much for making it more clear .