Enriching visualizations (data tables) with human readable data

Hello!

I am still new to ELK and all its powers and thus I might go about things the wrong way. Here is what I would like to do but can't figure out a way that seems reasonable:

I am analyzing firewall logs. One of my data fields is the destination port. For my "Top 10 Blocked Ports" visualization I would like to add the Iana or other description for the ports to appear in a separate column.

I am migrating from Sumo Logic, where I had the following table, pulling the description from a Google Spreadsheet read as CSV. A quite elegant solution as I thought:
22%20PM

I am not fixated on doing anything the same way but I am thinking that adding human readable descriptions to code values must be a common requirement in visualizations and I just am missing the obvious.

I have considered doing it in a logstash filter, but that seems totally unreasonable since it would be adding the same exact strings to each record with the same same code. However, maybe there is something about elasticsearch that makes this not as bad an idea as it is in legacy databases?

I also considered (and am still considering) simply putting the logic into a scripted field and embed the whole key-value map (up to thousands of kv pairs) in the script. Seems like an odd way to do it, but I think it would work quite nicely.

However, I am still thinking that there must be a better way. Can I import my dictionary files (I have more use cases of the same nature but different data) into its own elasticsearch index and perform lookups that way? All my research shows I can't do "joins" that way and that that was the wrong approach.

Any help or hints or even RTFMs (with links :slight_smile:) are greatly appreciated!

Thank y'all!

Cheers.
Volker

The recommended approach is indeed to add a field with the description in Logstash before the data is indexed. Denormalising dara this way is very common when working with Elasticsearch and it handles it quite efficiently.

Thank you, @Christian_Dahlqvist! :+1:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.