Displaying Latest Image with filter from Log Message

Elastic Stack Kibana
1

Hi everyone.
So currently, I'm getting logs from various services. I manage to tag these services as a field when it's ingested into elasticsearch via logstash. I'm currently stump with one part where i'm trying to display latest logs with b64 encoded image as its message on Kibana.
I'm wondering this is possible?
The requirement is like this

  1. i would preferably like to display this on dashboard or canvas along with other visualization.
  2. the image will be encoded directly with message field when ingested "data/image:,..........." . Obviously there are other kind of log messages too.
  3. the image shown should be the latest image from that service (a tag which i added to all logs to differentiate between each service).
    I've seen various topics on trying to show image on kibana but i haven't found one that works for me yet. At least for this requirement. If there's a tutorial please direct me to it. I have been google searching for the past few days now that i'm considering just switching to write python Streamlit (I could code this up within a day but in long term this might be undesirable due to maintenance and such, it might add additional technical debt to the stack for someone who will take over this) directly instead of relying on Kibana.
2

have you tried using a field formatter ?

  • go to dataview management, select your dataview
  • find the field with base64 data, click edit
  • choose url formatter
  • select type: image
  • in the url enter {{value}}
3

Yes i've tried the field formatter. I managed to display in the field formatter. However, I have no knowledge on how to display it on dashboard - i need to filter to gain the latest item. So i basically don't have enough markdown knowledge to do it.

So I try to display the value on canvas right now with markdown since it has the data pull feature. I'm also struggling here since the data pulled show the field as null. I suspect that the b64 encoding is too long which does not allow the field to be put in index and becomes an ignored field. So right now i'm trying to find a way to get ignored_field values onto markdown in canvas

4

to put in on dashboard you can try this steps:

  • go to visalizations
  • create new -> aggregation based -> table
    -> define your fields (one of them should be the one with custom field formater)
  • save visualization and put it on dashboard
5

I dont think this works because aggregation based means that you have to do some kind of metric aggregation right - i was trying just now and i see no options to do something like simply showing the field- maybe discovery?

Anyways, I'm probably going to try the method that will most likely work which is to make a small python rest api on localhost that fetch latest images in the file system and let kibana get the image on markdown via url (localhost should technically work). This way i don't need to increase size of elasticsearch by storing images

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.