Automating export from Elastic Cloud

[t3fsx]

Hi,

I read this thread (Kibana API for automatic Export all) which is attempting to do something very similar to what I'm trying and the information in it is valuable.

That being said, I'm not entirely sure how to apply what is discussed there to Elastic Cloud, which is where my cluster is running.

I was wondering if someone had automated the export of searches, views and dashboards from Elastic Cloud through a "backup script" of some sort.

Thank you!

[LizaD (Liza Dayoub)]

Hi @t3fsx,

You can use the Kibana API on Cloud instance the same, just plugin the Kibana Cloud URL and your credentials to authenticate the request.
Hope this helps.

Thanks,
Liza

[t3fsx]

Hi @LizaD

Thanks for the reply. I'm not entirely sure how I would go about getting the credentials. What kind of credentials would I be able to utilize via curl?

Also, the documentation doesn't, as far as I was able to tell, list the possible objects that can be exported: https://www.elastic.co/guide/en/kibana/master/saved-objects-api-export.html

index-pattern, dashboard and visualisation are referenced. I assume search is supported as well?

[LizaD (Liza Dayoub)]

Hi @t3fsx,

For curl you can use -u user:password option and the credentials would be same as you login into UI.

https://www.elastic.co/guide/en/kibana/master/saved-objects-api-get.html, shows search as a valid type.

Thanks,
Liza

[t3fsx]

@LizaD

Get object API | Kibana Guide [master] | Elastic

Hey @LizaD, thank you!

I'm much closer, even though still not there

I'm using:

curl -X POST -H "kbn-xsrf: script" -u "$kCreds" --data '{"type":"search","id":"a02fd230-72bf-11ea-97e5-156620cf9598"}' $kURL/$kPATH

However I keep getting the following error:

{"statusCode":400,"error":"Bad Request","message":"child \"id\" fails because [\"id\" is not allowed to be empty]","validation":{"source":"params","keys":["id"]}}%

I've tried escaping the request (--data) in all ways I knew how.

I was hoping bulk would allow me to export all objects by type (without having to specify the ID) but that doesn't seem to be possible.

[LizaD (Liza Dayoub)]

Nice @t3fsx, it is getting closer :).

I think you are just missing the objects key in your data, see:

-d'
{ 
  "objects": [
   {
    "type": "search", 
    "id": "<yourid>"
   }
 ]
}

Give it a try and let me know, thanks!

[t3fsx]

@t3fsx

Export objects API | Kibana Guide [master] | Elastic

Hi again @LizaD! Thank you so much for your help.

I was actually trying to use the _bulk_get API but have since changed to the one you mentioned to better follow up with the recommendations you are giving.

I'm now passing everything as per the documentation:

Any tips? Thank you for all your help!

[LizaD (Liza Dayoub)]

Hi @t3fsx,

You need to add this to the header of the curl command, it should resolve this error.

-H 'Content-Type: application/json'

Thanks,
Liza

[t3fsx]

Hey again @LizaD,

Thank you for your perseverance in this.

I had that before and eventually moved to what I posted on the post before the error. Basically, if you use what you mentioned, you'll get:

`{"statusCode":400,"error":"Bad Request","message":"Request must contain a kbn-xsrf header."}%``

That's what I'm using -H "kbn-xsrf: script" even though putting anything in front of kbn-xsrf: will get rid of that error. Saw that being recommended here: Request must contain a kbn-xsrf header

[LizaD (Liza Dayoub)]

Thanks @t3fsx let's try both

-H 'kbn-xsrf: true' -H 'Content-Type: application/json'

Let me know.

[t3fsx]

Worthy of ! It works! Thank you very much @LizaD! It's was actually my bad as the "copy as curl" link in the documentation actually creates the curl command identical to what we have here.

To make it easier for someone that might end up here, the full command:

curl -X POST -H "kbn-xsrf: script" -H "Content-Type: application/json" -u elastic:pass https://yourID.eu-central-1.aws.cloud.es.io:9243/api/saved_objects/_export -d '{
  "objects": [
   {
    "type": "search",
    "id": "a02fd230-72bf-11ea-97e5-156620cf9598"
   }
 ]
}' -o test.json

Another question, and this might be something for another threat, is related to the IDs. I always assumed that (and the documentation provides an example of this), similar to the web UX, I could specify a type and download all saved objects of that type.

However, using api/saved_objects/_export specifying an ID id required. Is there an endpoint that would allow exporting all from type?

[t3fsx]

The objects was the part that was interfering with downloading all object types at once. As per the documentation it should not be included.

Thank you @LizaD for all your help!

[LizaD (Liza Dayoub)]

Yay @t3fsx! Glad you got it all working !