Curl'ing Kibana Dashboards exported into NDJSON

plissken · February 8, 2023, 11:44am

I'm having considerable difficulty in understanding the documentation on the Kibana API's. There's so much out of date information on the web that I could literally spend months doing syntax jenga. I have a set of dashboards and visualisations exported from the kibana dashboard into an ndjson file. I'm trying to curl this file back into production systems via the kibana dashboards/import API. My base command looks like this ...

curl -k -XPOST -H 'kbn-xsrf: true' http://127.0.0.1:5601/s/test/api/kibana/dashboards/import -d @dashboardfile.ndjson

I've tried several syntax changes, which always result in errors. Currently getting a 404
"error":"Bad Request","message":"[request body.objects]: expected value of type [array] but got [undefined]"}

Could someone kindly assist before I'm escorted into a padded cell? Thank you in advance.

stephenb · February 8, 2023, 2:48pm

What version are you on?

I assume you actually have a space named test

I Recommend sticking with our documentation.

Tell me the version and I will test

Assuming you are using a fairly new version you should use the Saved Objects API

I suspect your error is that your ndjson is malformed it needs to be an array so curious how did you create / download the dashboardfile.ndjson

plissken · February 8, 2023, 9:39pm

Hi Stephen. I'm on 8.5.0. The ndjson file was created by Kibana on the export saved objects menu. So I went into the kibana UI, navigated to Kibana->Saved Objects-> and exported the selected dashboard. Included all objects associated with the dashboard. Haven't touched the ndjson file, so trying to curl it as downloaded. I did try some permutations of curl for saved objects API but couldn't get that working either. Thanks.

stephenb · February 8, 2023, 9:56pm

Pretty sure you should then be using the saved object API not the dashboard API to reload.

You are mixing UI and APIs.

I will check a little later if I can

plissken · February 9, 2023, 8:09am

Thanks Stephen. I also missed getting back to you about the space "test". Yes, it's been created and ready for dashboards.

plissken · February 10, 2023, 2:11pm

Hi Stephen. Similar issues using the saved objects API. I've tried the following:

curl -k -XPOST -H 'kbn-xsrf: true' -H 'Content-Type: application/json'
http://127.0.0.1:5601/s/test/api/saved_objects/_bulk_create -d @app-server-system-overview.ndjso

which returns {"statusCode":400,"error":"Bad Request","message":"Invalid request payload JSON format"}

and trying x-ndjson (thinking it may explain the error)

{"statusCode":415,"error":"Unsupported Media Type","message":"Unsupported Media Type"}

And then avoiding the bulk API in favour of saved objects/dashboards

curl -k -XPOST -H 'kbn-xsrf: true' -H 'Content-Type: application/json'
http://127.0.0.1:5601/s/test/api/saved_objects/dashboard -d @app-server-system-overview.ndjson

I get {"statusCode":400,"error":"Bad Request","message":"Invalid request payload JSON format"}

There must be something wrong with the ndjson file. Perhaps I need to pre-format it before importing it. But as it comes directly from the Kibana UI export I didn't think this was necessary. Extract of app-server-system-overview.ndjson as follows:

{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{"query":{"query":"","language":"kuery"},"filter":}"},"title":"CPU Usage [Metricbeat System] ECS","uiStateJSON":"{}","version":1,"visState":"{"title":"CPU Usage [Metricbeat System] ECS","type":"metrics","aggs":,"params":{"time_range_mode":"entire_time_range","axis_formatter":"number","axis_position":"left","id":"80a04950-1b19-11e7-b09e-037021c4f8df","index_pattern":"metricbeat-*","interval":"auto","series":[{"time_range_mode":"entire_time_range","axis_position":"right","chart_type":"line","color":"#68BC00","fill":"1","formatter":"percent","id":"80a04951-1b19-11e7-b09e-037021c4f8df","label":"user","line_width":1,"metrics":[{"field":"system.cpu.user.pct","id":"80a04952-1b19-11e7-b09e-037021c4f8df","type":"avg"}],"point_size":"0","seperate_axis":0,"split_color_mode":"gradient","split_mode":"everything","stacked":"stacked"},{"time_range_mode":"entire_time_range","axis_position":"right","chart_type":"line","color":"rgba(211,49,21,1)","fill":"1","formatter":"percent","id":"993acf30-1b19-11e7-b09e-037021c4f8df","label":"system","line_width":1,"metrics":[{"field":"system.cpu.system.pct","id":"993acf31-1b19-11e7-b09e-037021c4f8df","type":"avg"}],"point_size":"0","seperate_axis":0,"split_color_mode":"gradient","split_mode":"everything","stacked":"stacked"},{"time_range_mode":"entire_time_range","axis_position":"right","chart_type":"line","color":"rgba(123,100,255,1)","fill":"1","formatter":"percent","id":"65ca35e0-1b1a-11e7-b09e-037021c4f8df","label":"nice","line_width":1,"metrics":[{"field":"system.cpu.nice.pct","id":"65ca5cf0-1b1a-11e7-b09e-037021c4f8df","type":"avg"}],"point_size":"0","seperate_axis":0,"split_color_mode":"gradient","split_mode":"everything","stacked":"stacked"},{"time_range_mode":"entire_time_range","axis_position":"right","chart_type":"line","color":"rgba(226,115,0,1)","fill":"1","formatter":"percent","id":"741b5f20-1b1a-11e7-b09e-037021c4f8df","label":"irq","line_width":1,"metrics":[{"field":"system.cpu.irq.pct","id":"741b5f21-1b1a-11e7-b09e-037021c4f8df","type":"avg"}],"point_size":"0","seperate_axis":0,"split_color_mode":"gradient","split_mode":"everything","stacked":"stacked"},{"time_range_mode":"entire_time_range","axis_position":"right","chart_type":"line","color":"rgba(176,188,0,1)","fill":"1","formatter":"percent","id":"2efc5d40-1b1a-11e7-b09e-037021c4f8df","label":"softirq","line_width":1,"metrics":[{"field":"system.cpu.softirq.pct","id":"2efc5d41-1b1a-11e7-b09e-037021c4f8df","type":"avg"}],"point_size":"0","seperate_axis":0,"split_color_mode":"gradient","split_mode":"everything","stacked":"stacked"},{"time_range_mode":"entire_time_range","axis_position":"right","chart_type":"line","color":"rgba(15,20,25,1)","fill":"1","formatter":"percent","id":"ae644a30-1b19-11e7-b09e-037021c4f8df","label":"iowait","line_width":1,"metrics":[{"field":"system.cpu.iowait.pct","id":"ae644a31-1b19-11e7-b09e-037021c4f8df","type":"avg"}],"point_size":"0","seperate_axis":0,"split_color_mode":"gradient","split_mode":"everything","stacked":"stacked"}],"show_grid":1,"show_legend":1,"time_field":"@timestamp","type":"timeseries","use_kibana_indexes":false,"drop_last_bucket":1,"axis_scale":"normal","truncate_legend":1,"max_lines_legend":1,"tooltip_mode":"show_all","isModelInvalid":false,"filter":{"query":"host.name:\"metricbeat-pri\"","language":"kuery"}}}"},"coreMigrationVersion":"8.5.0","id":"79e5b8fd-704b-4e00-a38f-6f35e6914773","migrationVersion":{"visualization":"8.5.0"},"references":,"type":"visualization","updated_at":"2023-02-08T08:24:33.365Z","version":"Wzk1MDEsM10="}
{"attributes"

stephenb · February 10, 2023, 4:34pm

Ok @plissken

Here is what I have for you now... and Wow that seemed a bit harder than I thought ...

So this is for an individual I used _export and _import ... I have not tried _bulk yet

So first I list to find the Object I am after in my initial / default space

# Get Dashboards
curl -X GET -H 'Content-Type: application/json' -H 'kbn-xsrf: true' http://localhost:5601/api/saved_objects/_find?type=dashboard | jq

In this case is is the eCommerce Dashboard

    {
      "type": "dashboard",
      "id": "722b74f0-b882-11e8-a6d9-e546fe2bba5f",

So now I _export it ... it takes both the type and id as far as I can tell in the objects array

# Export the actual Dashboard
curl -X POST -H 'Content-Type: application/json' -H 'kbn-xsrf: true' http://localhost:5601/api/saved_objects/_export -d '{"objects":[{"type":"dashboard","id":"722b74f0-b882-11e8-a6d9-e546fe2bba5f"}]}' > saved-dashboard.ndjson

Now _import to another space look at the syntax careful

# Import to a new space
curl -X POST -H 'kbn-xsrf: true' http://localhost:5601/s/test/api/saved_objects/_import?createNewCopies=true --form file=@saved-dashboard.ndjson
{"successCount":1,"success":true,"warnings":[],"successResults":[{"type":"dashboard","id":"722b74f0-b882-11e8-a6d9-e546fe2bba5f","meta":{"title":"[eCommerce] Revenue Dashboard","icon":"dashboardApp"},"destinationId":"e410188d-e9fd-40cc-bdc9-aa307d0a7165"}]}

And my Pretty new dashboard in my space test

I can probably look at the _bulk at some point if you still need that

plissken · February 10, 2023, 4:53pm

That's great Stephen. Thanks for all the hard work on this. Feels like an individual object export and import might be the best way forward. I'll have a play over the weekend. Have a good one.

stephenb · February 10, 2023, 4:55pm

Its bugging me... I will probably try the _bulk at some point

system · March 10, 2023, 4:56pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Export and import via curl kibana's dashboards and visualizations Kibana	8	13913	November 4, 2022
Kibana API for automatic Export all Kibana	7	3122	October 28, 2019
Exporting Dashboards with script Kibana	6	2197	February 15, 2018
Issue with reusing dashboard on another index Kibana	9	2145	June 12, 2020
Export/Import Dashboards and Associated visualization's Kibana	9	4705	August 9, 2018

Curl'ing Kibana Dashboards exported into NDJSON

Related Topics