How to backup "index pattern", "setting of visualization or dashboard" and et al.?

Hi all,

After I updating license (Extend Shield and Watcher License), all of kibana setting (Web operations) are gone, I don't know why?

I would like to backup kibana setting (Web operations), is it possible or someone have any idea? thanks

Jason

That shouldn't happen, does the index still exist?

You can backup using snapshot and restore.

Also do you have a support subscription? If so I'd suggest you contact the support team.

Hi Mark,

Thanks for replying, the settings of past 30 days are gone, the setting status of kibana seems back to before 1 months

For now, the browser never ask user/password when I visit Kibana after license updating even I clear caches of browser. The extend license (Shield and Watcher) is obtained from elastic.co support team, all of my nodes are updated:

Shall the license plugin cover 'Kibana'?

Jason

I removed the license from your post, please don't post them to the forums as then anyone can use them!

Can you explain what you mean by the settings being not what you expect?

Hi Mark,

All status of Kibana setting goes back 1 month ago, for example
July ~ Non-license plugin (non-shield)
I create A, B and C index pattern, save D, E visualization and F dashboard during July

License plugin installed ~ License expirid (Shield installed)
I delete B, C index pattern and Create B' and C', save D', E' visualization and F' Dashboard during Auguest

License expired ~ License updated (Subscription type: Gold, Shield and Watcher, 2 nodes)
The status of Kibana setting: A, B and C index pattern, D, E visualization and F dashboard, all of operations of auguest are gone

Jason

If you have a gold subscription level then you should really be talking to the Support team directly, assistance for these sorts of issues is included in your subscription :slight_smile:

Hi Mark,

Not really gold subscription, license type is trial.

As I know, all of kibana settings should be saved in ".kibana" index, is it a file or a index of elasticsearch, if .kibana index is a index of elasticsearch, it should not be back.

Jason

All of the KB things like dashboards and index settings are saved in Elasticsearch.
Can you grab the entire Kibana index from Elasticsearch and see what is in there?

Hi Mark,

I don't really know how to query to '.kibana' of elasticsearch, following is my steps,

$curl 'http://localhost:9200/_cat/indices?v'
....
green  open   logstash-miki-2015.09.04    2   0       7277            0      2.1mb          2.1mb
yellow open   .watch_history-2015.09.04   1   1      15802            0      2.4mb          2.4mb
yellow open   logstash-2015.08.14         5   1      42595            0       13mb           13mb
yellow open   .kibana                    1   1         15            3     63.9kb         63.9kb
....

And then,

$curl -XGET 'http://localhost:9200/.kibana?pretty'
{
  ".kibana" : {
    "aliases" : { },
    "mappings" : {
      "config" : {
        "properties" : {
          "buildNum" : {
            "type" : "long"
          },
          "defaultIndex" : {
            "type" : "string"
          }
        }
      },
      "index-pattern" : {
        "properties" : {
          "fieldFormatMap" : {
            "type" : "string"
          },
          "fields" : {
            "type" : "string"
          },
          "intervalName" : {
            "type" : "string"
          },
          "timeFieldName" : {
            "type" : "string"
          },
          "title" : {
            "type" : "string"
          }
        }
      },
      "dashboard" : {
        "properties" : {
          "description" : {
            "type" : "string"
          },
          "hits" : {
            "type" : "integer"
          },
          "kibanaSavedObjectMeta" : {
            "properties" : {
              "searchSourceJSON" : {
                "type" : "string"
              }
            }
          },
          "panelsJSON" : {
            "type" : "string"
          },
          "timeFrom" : {
            "type" : "string"
          },
          "timeRestore" : {
            "type" : "boolean"
          },
          "timeTo" : {
            "type" : "string"
          },
          "title" : {
            "type" : "string"
          },
          "version" : {
            "type" : "integer"
          }
        }
      },
      "visualization" : {
        "properties" : {
          "description" : {
            "type" : "string"
          },
          "kibanaSavedObjectMeta" : {
            "properties" : {
              "searchSourceJSON" : {
                "type" : "string"
              }
            }
          },
          "savedSearchId" : {
            "type" : "string"
          },
          "title" : {
            "type" : "string"
          },
          "version" : {
            "type" : "integer"
          },
          "visState" : {
            "type" : "string"
          }
        }
      },
      "search" : {
        "properties" : {
          "columns" : {
            "type" : "string"
          },
          "description" : {
            "type" : "string"
          },
          "hits" : {
            "type" : "integer"
          },
          "kibanaSavedObjectMeta" : {
            "properties" : {
              "searchSourceJSON" : {
                "type" : "string"
              }
            }
          },
          "sort" : {
            "type" : "string"
          },
          "title" : {
            "type" : "string"
          },
          "version" : {
            "type" : "integer"
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1436151025757",
        "number_of_shards" : "1",
        "uuid" : "g-uznPAnQJyqPwe8vNQbnA",
        "version" : {
          "created" : "1060099"
        },
        "number_of_replicas" : "1"
      }
    },
    "warmers" : { }
  }
}

Try curl -XGET 'http://localhost:9200/.kibana/_search?pretty' and then see if all your dashboards are there.

Shield will not just delete things, it has to have been done by someone.

Hi Mark,

The result is:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "hits" : {
    "total" : 15,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : ".kibana",
      "_type" : "visualization",
      "_id" : "AreaChart_RXTX",
      "_score" : 1.0,
      "_source":{"title":"AreaChart_RXTX","visState":"{\"type\":\"line\",\"params\":{\"addLegend\":true,\"addTimeMarker\":false,\"addTooltip\":true,\"defaultYExtents\":false,\"drawLinesBetweenPoints\":true,\"interpolate\":\"linear\",\"radiusRatio\":9,\"scale\":\"linear\",\"setYExtents\":false,\"shareYAxis\":true,\"showCircles\":true,\"smoothLines\":false,\"times\":[],\"yAxis\":{}},\"aggs\":[{\"id\":\"1\",\"type\":\"max\",\"schema\":\"metric\",\"params\":{\"field\":\"ismb.miki.vnic.rx\"}},{\"id\":\"2\",\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"d\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{}}},{\"id\":\"3\",\"type\":\"max\",\"schema\":\"metric\",\"params\":{\"field\":\"ismb.miki.vnic.tx\"}}],\"listeners\":{}}","description":"","savedSearchId":"TypeJson","version":1,"kibanaSavedObjectMeta":{"searchSourceJSON":"{\"filter\":[]}"}}
    }, {
      "_index" : ".kibana",
      "_type" : "visualization",
      "_id" : "AreaChart",
      "_score" : 1.0,

For example, _id AreaChart has been to deleted by myself on Auguest, but its recover by itself?

The status of dashboards or visualizations setting is go back to 1 month ago (July), I create something and delete something on Aug, but now all of my operations are missing just like I recover '.kibana' index to July, but I didn't do any snapshot or recover for '.kibana', or satisfy something to make '.kibana' to recover by itself?

Jason

Who else has access to the cluster? Have you asked them if they have restored anything?

Hi Mark,

All of my cluster were built on my VM, and it provision under firewall of my office, only me access to the cluster, nobody else, so it so strange, the status of '.kibana' index goes back to 1 month ago after updating license (Shield and Watcher), BTW, the kibana server shutdown accidentally when I updating license ( does it matter?)

Jason

No it should not matter.