Importing dashboard and index pattern on ECK

Hey there,

I am currently playing around on ECK on minikube on macOS and I was wondering how to load previously saved Saved Objects.

The idea I had was to create a pod, mount the .ndjson and push it with curl to the kibana node, as explained in the docs.

During an earlier try, it was complaining about the actual .ndjson I was using, so after googling around I found this article where the person is splitting the dashboard and index pattern before pushing them separately, which is fine for me.

The problem is that I still can't get it to work and I have no idea where to go from here.

Here are the 2 files:

dashboard-cm.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: kibana-data
data:

  curl-script.sh: |
    #!/bin/sh

    function import_index {
        echo "Importing index..."
        until curl -k --user ${ES_USER}:${ES_PASSWORD} -X POST ${KB_HOST}/api/saved_objects/_import -H "kbn-xsrf: true" --form file=@/kibana/index-pattern.ndjson -H 'kbn-xsrf: true'; do sleep 5; done
        if [[ $(echo "$?") == "0" ]]; then
          printf "\n########## Imported index-pattern successfully! #############################\n"
        else
          printf "\n########## Failure while importing index-pattern #############\n"
        fi
    }
    function import_dashboards {
        echo "Importing dashboards..."
        until curl -k --user ${ES_USER}:${ES_PASSWORD} -X POST ${KB_HOST}/api/kibana/dashboards/import?exclude=index-pattern -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d @/kibana/dashboards.json; do sleep 5; done
        if [[ $(echo "$?") == "0" ]]; then
          printf "\n########## Imported dashboards successfully! #############################\n"
        else
          printf "\n########## Failure while importing dashboards #############\n"
        fi
    }
    import_index
    import_dashboards
  index_pattern.ndjson: |
    {"attributes":{"fieldAttrs":"{}","fields":"[]","runtimeFieldMap":"{}","title":"index1"},"coreMigrationVersion":"7.13.2","id":"58141060-d808-11eb-a87c-b5dcc59a4142","migrationVersion":{"index-pattern":"7.11.0"},"references":[],"type":"index-pattern","updated_at":"2021-06-28T21:44:22.007Z","version":"WzQ5LDFd"}
    {"exportedCount":1,"missingRefCount":0,"missingReferences":[]}
  dashboards.json: |
    "version": "7.13.2",
    "objects": [
      {
        "id": "58b7b120-d80d-11eb-a87c-b5dcc59a4142",
        "type": "dashboard",
        "namespaces": [
          "default"
        ],
        "updated_at": "2021-06-28T21:50:21.854Z",
        "version": "Wzc4LDFd",
        "attributes": {
          "title": "Dashboard Test",
          "hits": 0,
          "description": "",
          "panelsJSON": "[{\"version\":\"7.13.2\",\"type\":\"visualization\",\[...],
          "optionsJSON": "{\"hidePanelTitles\":false,\"syncColors\":true,\"useMargins\":false}",
          "version": 1,
          "timeRestore": false,
          "kibanaSavedObjectMeta": {
            "searchSourceJSON": "{\"query\":{\"language\":\"kuery\",\"query\":\"\"},\"filter\":[]}"
          }
        },
        "references": [
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "ac99e2a1-79c5-478b-a10c-8349d8ff4584:indexpattern-datasource-current-indexpattern"
          },
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "ac99e2a1-79c5-478b-a10c-8349d8ff4584:indexpattern-datasource-layer-245dc3b6-65d8-4515-8596-24b8c1a0de1d"
          },
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "35017a5f-9388-42f9-88ac-9eed6402784e:indexpattern-datasource-current-indexpattern"
          },
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "35017a5f-9388-42f9-88ac-9eed6402784e:indexpattern-datasource-layer-8a9d322b-947e-45b5-a6c7-6252082ed26b"
          },
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "2a119d55-4de8-4e23-aa80-96439f068a2c:indexpattern-datasource-current-indexpattern"
          },
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "2a119d55-4de8-4e23-aa80-96439f068a2c:indexpattern-datasource-layer-c625be08-63b8-454b-ac43-abc628dd3ff9"
          },
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "d908c5ff-51de-42e1-b7b5-431f624241ff:indexpattern-datasource-current-indexpattern"
          },
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "d908c5ff-51de-42e1-b7b5-431f624241ff:indexpattern-datasource-layer-d2fecd84-d751-402d-9156-c01b247fb8bc"
          },
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "3256b0f0-6aa9-41d8-835c-1782de7445a5:indexpattern-datasource-current-indexpattern"
          },
          {
            "type": "index-pattern",
            "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
            "name": "3256b0f0-6aa9-41d8-835c-1782de7445a5:indexpattern-datasource-layer-9da5dd88-2bed-461d-b7d0-906f051eeab9"
          }
        ],
        "migrationVersion": {
          "dashboard": "7.13.1"
        },
        "coreMigrationVersion": "7.13.2"
      },
      {
        "id": "58141060-d808-11eb-a87c-b5dcc59a4142",
        "type": "index-pattern",
        "namespaces": [
          "default"
        ],
        "updated_at": "2021-06-28T21:44:22.007Z",
        "version": "WzQ5LDFd",
        "attributes": {
          "fieldAttrs": "{}",
          "fields": "[]",
          "runtimeFieldMap": "{}",
          "title": "people"
        },
        "references": [],
        "migrationVersion": {
          "index-pattern": "7.11.0"
        },
        "coreMigrationVersion": "7.13.2"
      }

and people-dashboard-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: dashboard-pod
spec:
  containers:
  - image: centos:centos7.9.2009
    name: dashboard-pod
    env:
    - name: KB_HOST
      value: "https://cluster1-kb-http:5601"
    - name: ES_USER
      value: "elastic"
    - name: ES_PASSWORD
      valueFrom:
        secretKeyRef:
          name: cluster1-es-elastic-user
          key: elastic
    command: ["/bin/sh","/kibana/curl-script.sh"]
    resources: {}
    volumeMounts:
    - name: kibana-data
      mountPath: /kibana
  restartPolicy: Never
  volumes:
  - name: kibana-data
    configMap:
      name: kibana-data
      defaultMode: 0777

The pod launches fine but I have the following error in the logs:

curl: (26) couldn't open file "/kibana/index-pattern.ndjson"

Do you have any idea what's happening? Or is there a better solution to import Saved Objects into Kibana?

Thanks in advance for your help!

Cheers,

Julien

You referenced index-pattern.ndjson in the curl-script.sh but used index_pattern.ndjson as key in the kibana-data ConfigMap.

1 Like

Oh my...
Thanks a lot! This completely fixed this part.

So it uploaded the index pattern just fine, but I still have an error with the dashboard:
{"statusCode":400,"error":"Bad Request","message":"Invalid request payload JSON format"}

Any idea why? It does seem like valid json.

edit: If it helps, when I remove the -H 'Content-Type: application/json' from the curl command, I have the following error {"statusCode":400,"error":"Bad Request","message":"[request body.objects]: expected value of type [array] but got [undefined]"}

final EDIT: as now the first part was working, I tried to import everything at once using a single .ndjson file and it's working. So all good, the only issue was indeed the typo you mentionned... Thanks again!

The dashboards.json secret entry is not a valid JSON. It is missing at the beginning { and at the end ]} assuming that you've shortened the value of panelsJSON to make the secret content readable.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.