First of all, thanks for some really awesome products. We are using EK to centralise our logs from our integration plattform and it has done wonders as to what insights we can now provide to business.
We sometimes log large payloads as part of our log messages. For instance:
"attachment.content": "Big XML"
This works fine and we can saw the raw content when do a search in the discover. We wanted now to transform the field into a clickable link so that when the user clicks the link, a new tab opens and the raw content is disabled. The raw content by itself is not a url or an image, it is just some string data.
I am on Kibana 6.1 and have played with the string formatter for using label and url formatter but can't seem to get this to work. Is this possible or does the data itself need to be an image or some url?
Just to be clear I don't have a http://company.net url to refer to, I simply want Kibana to force a new tab showing the raw value of the field.
The field formatter will allow you to use the value of a field as part of a larger template. You can use this turn urls into images, or values into links, but it doesn't offer any way to process the field value, so you couldn't pull out part of the field value... and that wouldn't help in your case anyway.
You also can't use values from other fields. If you could, you might be able to use the id field to create a link to the single document view in Kibana, so that in the table in Discover you see the link to that view. But even then, that view would also just show the link, which doesn't help you.
I can't think of a way to hide that data in Discover (which you will almost certainly want to do anyway so to stop Kibana from processing that large amount of data) but have a view in Kibana that allows you to see it. At least not without writing some custom code, either a standalone field viewer app, or a Kibana plugin to do it for you. But even with that, I'm not sure how you'd link to that view from that field...
Yeah, I kind of suspected that some custom coding would be needed. It would be a cool feature but I guess since the data itself is not a url or part of a url it would be difficult to do this out of the box. Thanks for the analysis though.
Yeah, it would be kind of a neat feature. Kibana has an advanced setting where it'll exclude certain fields, and the XML data you have is a prime candidate for that setting. You basically never want to send that data across the wire, it's very large and it's not very useful. But you do sometimes what to see that data, on a document by document basis. It would be helpful if you could, when looking at document values, open up a view to look at the entire document value, with those excluded fields included. It sounds like that would give you what you're looking for.
So if I exclude that field, it won't show up in the discovery search, but if I open the document, then I will see the entire content? That sounds sort of what I want. I'll try that tomorrow. Thanks for the hint and a superb product!
So if I exclude that field, it won't show up in the discovery search, but if I open the document, then I will see the entire content?
Actually, I believe that it won't show up either way, unless maybe if you explicitly query on that field. I'm pretty sure the single doc view won't include that field. I could be wrong though, it's worth giving that a shot.
Jus to get back to you on this, I did manage to get it to work similar to what you suggested. In management, I went into source fields, and added the two fields I wanted to exclude.
Then when I search, the data for the two fields do not appear. But when I click on the json doc the fields are there. Awesome! Thanks
Huh, neat, I didn't realize it worked that way. Thanks for following up and letting me know, it's helpful for people searching for answers too. Glad you got what you needed, cheers!
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.