Kibana dashboard import API not working from API export

Elastic Stack Kibana
1

I am trying to use the export/import APIs so I can extract my objects and put them in a repo so they can be loaded in an install script that also does all the elasticsearch mappings and whatnot. I am testing and the export works fine but the import keeps blowing chunks.

I use a script to export - redirected to a file - here's the important part:

DASH_LIST=$(sed -e 's/^"//' -e 's/"$//' <<<`curl -s http://localhost:9200/.kibana/_search?pretty |grep '_id" : "dashboard:' |awk -F[:,] {'print $3'}`)

for DASH in $DASH_LIST
do
    curl -XGET -H'Content-Type: application/json' \
         http://$KIBANA_HOST:$PORT/api/kibana/dashboards/export?dashboard=$DASH >>./test.json
done

Works fine (see file output below). Then I try the import:

curl -X POST -H 'kbn-xsrf: true' -H 'Content-Type: application/json' "localhost:5601/api/kibana/dashboards/import" --data-binary @./test.json

That's when I get said chunk blowing:

{"statusCode":400,"error":"Bad Request","message":"Invalid request payload JSON format"}

Checking the kibana logs I see this:

    Debug: internal, implementation, error 
    TypeError: Cannot read property 'type' of undefined
    at 6.5.0 (/usr/share/kibana/src/legacy/core_plugins/kibana/migrations/migrations.js:481:44)
    at tryTransformDoc (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:213:22)
    at migrateProp (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:270:11)
    at applyMigrations (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:172:11)
    at DocumentMigrator.transformAndValidate [as transformDoc] (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:151:43)
    at DocumentMigrator.doc [as migrate] (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:47:19)
    at KibanaMigrator.migrateDocument (/usr/share/kibana/src/core/server/saved_objects/migrations/kibana/kibana_migrator.js:146:34)
    at objects.map.object (/usr/share/kibana/src/core/server/saved_objects/service/lib/repository.js:190:74)
    at Array.map (<anonymous>)
    at SavedObjectsRepository.bulkCreate (/usr/share/kibana/src/core/server/saved_objects/service/lib/repository.js:174:37)
    at SavedObjectsClient.bulkCreate (/usr/share/kibana/src/core/server/saved_objects/service/saved_objects_client.js:45:35)
    at EncryptedSavedObjectsClientWrapper.bulkCreate (/usr/share/kibana/x-pack/legacy/plugins/encrypted_saved_objects/server/lib/encrypted_saved_objects_client_wrapper.js:80:89)
    at process._tickCallback (internal/process/next_tick.js:68:7)
{"type":"response","@timestamp":"2019-11-21T18:23:37Z","tags":["api"],"pid":6,"method":"post","statusCode":400,"req":{"url":"/api/kibana/dashboards/import","method":"post","headers":{"host":"localhost:5601","user-agent":"curl/7.58.0","accept":"*/*","kbn-xsrf":"true","content-type":"application/json","content-length":"77870","expect":"100-continue"},"remoteAddress":"172.18.0.1","userAgent":"172.18.0.1"},"res":{"statusCode":400,"responseTime":18,"contentLength":9},"message":"POST /api/kibana/dashboards/import 400 18ms - 9.0B"}

Which doesn't mean much. I do believe that the stack trace has nothing to do with this, but I can't tell from the formatting of the log.

2

How do I upload a file? I have the test.json file but figured it would be a bit long just to splatter on the screen. Thanks.

3

@sdndude can you upload your test.json file on GitHub as a gist? (gist.github.com)

I'm pretty sure import can only accept a single exported payload. In other words, rather than creating a single test.json, you likely need an individual JSON file for each dashboard you're trying to export/import

What version of Kibana are you using? If you're running at least 7.2, then there is a different set of APIs available which we recommend you use instead: https://www.elastic.co/guide/en/kibana/current/saved-objects-api-export.html

It likely won't solve your underlying problem, but I wanted to make you aware, as the APIs you're using will eventually be removed.

4

Ah cool. I saw those APIs but they seemed to be the individual calls whereas the import/export looked like they would just grab everything.
I will try the newer APIs and see what happens. Do you know off the top of you head where I put the includeReferencesDeep? I can't find an example. I will futz with it till I figure it out.

Thanks dude!

5

Something like this should work for you. It will grab the dashboard, and all saved objects that this dashboard depends on (similar to the API you were using before). Although with this new API, you can specify all your dashboards in a single request, and get the entire payload in one shot. Then you can use that same payload in a single _import request :sunglasses:

POST api/saved_objects/_export
{
  "objects": [
    {
      "type": "dashboard",
      "id": "be3733a0-9efe-11e7-acb3-3dab96693fab"
    }
  ],
  "includeReferencesDeep": true
}
6

Thanks @Larry_Gregory , but I must be missing something here. Running 7.4.2 and using the example above in DevTools (with my own id of course) gives me this:

#! Deprecation: [types removal] Specifying types in document index requests is deprecated, use the typeless endpoints instead (/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id}).
{
  "_index" : "api",
  "_type" : "saved_objects",
  "_id" : "_export",
  "_version" : 6,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 5,
  "_primary_term" : 1
}

I was expecting back the json of the exported dashboard (and all of the other stuff). If I try searching that index I get back the exact object text that I sent.

If I try it on the command line thusly:

curl -X POST -H 'kbn-xsrf: true' http://localhost:5601/api/saved_objects/_export -d '
{ 
   "objects": 
          [
            {
              "type": "dashboard",
              "id": "7298b6d0-0cbb-11ea-9d6a-8958e35fe94b"  
            }
          ],
   "includeReferencesDeep": true
} '

I get the following error:

{"statusCode":400,"error":"Bad Request","message":"\"{   \"objects\": [{\"type\": \"dashboard\",\"id\": \"7298b6d0-0cbb-11ea-9d6a-8958e35fe94b\"  } ],\"includeReferencesDeep\": true}\" is not allowed. \"value\" must contain at least one of [type, objects]","validation":{"source":"payload","keys":["&#x7b;   &quot;objects&quot;: &#x5b;&#x7b;&quot;type&quot;: &quot;dashboard&quot;,&quot;id&quot;: &quot;7298b6d0-0cbb-11ea-9d6a-8958e35fe94b&quot;  &#x7d; &#x5d;,&quot;includeReferencesDeep&quot;: true&#x7d;","value"]}}

So I am still messing with it. Thanks.

7

Ah, you unfortunately won't be able to use DevTools here -- DevTools only works with Elasticsearch APIs, and not Kibana APIs. You'll need to use curl or similar to access the Kibana APIs.

Speaking of curl, try adding -H 'Content-type: application/json' to your curl request and see if that improves things.

8

I could have sworn I tried that before and it didn't work. But alas, it did!!! Thanks for that.
So now I will try the import. So get ready for another round of questions. :smiley:

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.