Download Files of an API from Kibana Dashboard

Elastic Stack Kibana
1

Hello,

we have a Kibana dashboard showing some metadata of files.

Is it somehow possible to integrate a link or button to download the files from our API after filtering the dashboard using controls?

The goal should be to filter the filelist down to the files you want, klick something and get the files as download in the browser.

I think it should be possible, if you can somehow send a GET to our API with either the values of the controls, or a list of file IDs as URL parameter, no?

I read about drilldown to URL but i can’t test it without a lisence and as far as i read it can only get dashboard filters and not control selections and also only the id per row (so 1 click 1 file) and not a list of all ids currently in the table, right?

Best regards
Jonas

2

If you have the link to the file in a field, you can add a saved search to your dashboard and format the field as a link, kibana will then render this as a link.

Go into the Data View for your data, and edit the format for the specific field:

image
image573Ă—645 27.1 KB

The documentation is here.

1 Like
3

Thank you very much for your reply, but it should not be 1 link per file, but 1 link for all filtered files.

If the table contains like 200 files after control selection you can’t ask the dashboard users to click 200 links to get all the files. It should be 1 click to get them all (probably zipped).

Or did i missunderstand your answer?

My problem is how do i get either the control selections or the current content (id column) of a lens or saved search into a single url.

Is it maybe possible with something like scripted metric aggregation? Could those show a link and access controls or current contents of other visualizations in the script? Or maybe a custom vega visualization?

And my assumption that even with a lisence the URL Drilldown can not fulfill this requirement is correct?

4

Is there really no solution for this? It seems like such an obvious use case to me: Filter data and call APIs based on the results, no?

Should i embed the dashboard as iFrame and build a button that reads the html of the controls? How stable are the elements with new kibana versions?

I don’t like such hacky solutions, but you can’t tell me there is no way to do simple API calls based on filtered dashboards.

5

Kibana does not make external calls, it is a visualization tool for Elasticsearch data, you can however format the fields in the way I mentioned to make it look like a link so the users can click on the link and download the files.

How many links you will have and how this will work will depend on your data and your API, mostly on your API.

For example, you may have a field with all the file names as an array (this can be done during ingestion, query time, scripted field etc) , and then use this array in the request for your API, your API then will need to deal with this information and return a single file to your user.

1 Like
6

For example, you may have a field with all the file names as an array (this can be done during ingestion, query time, scripted field etc) , and then use this array in the request for your API, your API then will need to deal with this information and return a single file to your user.

Thanks again! I can build and change the API, thats not the problem. The data is created by logstash querying a database with sql under my control, so again i can change that in any way.

Could you expand a bit on your idea? I still am not sure how the array you sugest is filtered down by the user making control selections. Of course i can add an array to every document that contains all file IDs in the system, but how would that help me? The documents will get filtered by the controls, but not the array, right? So i can do a link to get a single file or one to get all overall, but i still don’t know how to get the filtered ones.

But you said the array can be added during “query time”, i’m not quite sure what that refers to. The users don’t query the data by kql if that’s what you mean. They also don’t use the dashboard filters, they use controls.

Imagine the dashboard having a saved search showing like 10k files. Now the user sets some criteria in like 3 out of 10 available controls. Afterwards the saved search shows like 200 files. Now there should be a link or button to get exactly those 200 files (zipped as a single file).

And again, the API is happy to take whatever. I just don’t know how to get the information which filter criteria the user chose in whatever form into the URL. That’s why i said either an ID array, or one with controlfield+value is fine. The API can just take the array and return the files, or can take the filter criteria, and do the filtering by itself and return. No problem here.

It’s just about: control selection/resultset ids → URL

7

Yeah, just made some tests, and I don't think this is possible in Kibana.

Using ESQL you can create a single string with all the values from a field for example and concatenate this with some random string, like an URL, but you cannot make this clickable, the user would need to copy the value and past it in another tab.

For example, if you have a result like this:

user1 - file1
user1 - file2
user2 - file3

You can use ESQL in a visualization to create a string like this:

https://endpoint/file=file1,file2,file3

But as far as I know you cannot make this clickable.

If copying the link is something that works for your users, than this can be done, if you need this to be a link, then I don't think it is possible in Kibana.

The following ES|QL query would create the mentioned string:

FROM index 
| WHERE file_name_field IS NOT NULL
| STATS array_of_files = VALUES(file_name_field)
| EVAL files = MV_CONCAT(array_of_files, ",")
| EVAL sanitized_files = URL_ENCODE(files)
| EVAL link = CONCAT("https://endpoint/files=",sanitized_files)
| KEEP link
| LIMIT 10
1 Like
8

If it not being clickable is the only issue, that’s no problem at all! Copy & paste is fine.

If this works like you described this will help us integrate kibana dashboards a lot more in our processes as we can not only use it for this current use case, but in general to filter data and make requests based on the results.

I never worked with ES|QL and just realized i have to update the stack for it to be available, but thats overdue anyways. So it will take me some time to actually try it out, but it sounds very promising.

Thank you so very much for your invested time!

(I even remember your name, you gave me a solution to another problem like 4 years ago were i was not getting on the right track and we still use that today. So again, thank you so much for your continued support!)