Kibana dashboard import: failed to parse field visualization.kibanaSavedObjectMeta.searchSourceJSON

Elastic Stack Beats
1

Hi, just got filebeat installed with modules and pregenerated dashboards via make fields and make kibana. When trying to run for the first time with setup.dashboards.enabled: true, all dashboards error out with the following message:

Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/filebeat/kibana: Failed to import dashboard: Failed to load directory /usr/share/filebeat/kibana/6/dashboard:
  error loading /usr/share/filebeat/kibana/6/dashboard/Filebeat-Kafka-overview.json: failed to parse field [visualization.kibanaSavedObjectMeta.searchSourceJSON] of type [text]. Response: {"objects":[{"id":"Number of Kafka stracktraces by class","type":"visualization","error":{"message":"failed to parse field [visualization.kibanaSavedObjectMeta.searchSourceJSON] of type [text]"}},{"id":"Kafka stacktraces","type":"search","error":{"me... (truncated)
  error loading /usr/share/filebeat/kibana/6/dashboard/Filebeat-Mongodb-overview.json: failed to parse field [visualization.kibanaSavedObjectMeta.searchSourceJSON] of type [text]. Response: {"objects":[{"id":"0fef5710-0a82-11e8-bffe-ff7d4f68cf94","type":"visualization","error":{"message":"failed to parse field [visualization.kibanaSavedObjectMeta.searchSourceJSON] of type [text]"}},{"id":"e49fe000-0a7e-11e8-bffe-ff7d4f68cf94","type":"se... (truncated)
  error loading /usr/share/filebeat/kibana/6/dashboard/Filebeat-Postgresql-overview.json: failed to parse field [visualization.kibanaSavedObjectMeta.searchSourceJSON] of type [text]. Response: {"objects":[{"id":"PostgreSQL Log Level Count","type":"visualization","error":{"message":"failed to parse field [visualization.kibanaSavedObjectMeta.searchSourceJSON] of type [text]"}},{"id":"PostgreSQL All Logs","type":"search","error":{"message":"f... (truncated)

and so forth for every json file in the dashboard directory. Note that the (truncated) is done by filebeat, not me. As best I can tell, the dashboards are perfectly valid json, but I have uploaded a sample here: https://gist.github.com/matoro/cfc01d6e8f8fc8024ea7769be6384c08

Is there any way to at least see the full contents of the error message? What can I do to help debug this problem? Thank you.

2

How did you create the dashboard JSON files?

Which filebeat/kibana version are you using?

3

@steffens Thank you for replying. The files were created from a make kibana in the tagged checkout of the repo at v6.5.4. Versions are consistent across the stack (ES, LS, Kibana, FB).

4

Can you check kibana logs?

Have you modified any files? What does git status and git diff say?

5

@steffens The kibana logs contain hits looking like this (one for each json file):

{"type":"response","@timestamp":"2019-01-02T14:08:14Z","tags":[],"pid":5546,"method":"post","statusCode":200,"req":{"url":"/api/kibana/dashboards/import?exclude=index-pattern&force=true","method":"post","headers":{"host":"10.0.0.2:5601","user-agent":"Go-http-client/1.1","content-length":"7446","accept":"application/json","content-type":"application/json","kbn-version":"6.5.4","accept-encoding":"gzip"},"remoteAddress":"10.0.0.2","userAgent":"10.0.0.2"},"res":{"statusCode":200,"responseTime":119,"contentLength":9},"message":"POST /api/kibana/dashboards/import?exclude=index-pattern&force=true 200 119ms - 9.0B"}
{"type":"response","@timestamp":"2019-01-02T14:08:15Z","tags":[],"pid":5546,"method":"post","statusCode":200,"req":{"url":"/api/kibana/dashboards/import?exclude=index-pattern&force=true","method":"post","headers":{"host":"10.0.0.2:5601","user-agent":"Go-http-client/1.1","content-length":"9283","accept":"application/json","content-type":"application/json","kbn-version":"6.5.4","accept-encoding":"gzip"},"remoteAddress":"10.0.0.2","userAgent":"10.0.0.2"},"res":{"statusCode":200,"responseTime":94,"contentLength":9},"message":"POST /api/kibana/dashboards/import?exclude=index-pattern&force=true 200 94ms - 9.0B"}
{"type":"response","@timestamp":"2019-01-02T14:08:15Z","tags":[],"pid":5546,"method":"post","statusCode":200,"req":{"url":"/api/kibana/dashboards/import?exclude=index-pattern&force=true","method":"post","headers":{"host":"10.0.0.2:5601","user-agent":"Go-http-client/1.1","content-length":"5563","accept":"application/json","content-type":"application/json","kbn-version":"6.5.4","accept-encoding":"gzip"},"remoteAddress":"10.0.0.2","userAgent":"10.0.0.2"},"res":{"statusCode":200,"responseTime":77,"contentLength":9},"message":"POST /api/kibana/dashboards/import?exclude=index-pattern&force=true 200 77ms - 9.0B"}

I should have been more careful with phrasing: the sources are not technically from a tagged checkout but from a .tar.gz download of the github repo at the tagged commit for the 6.5.4 release. There are minimal changes to the archive made, namely removing x-pack and replacing the git commit id with the release version in the makefile. The build script is here: https://gitweb.gentoo.org/repo/gentoo.git/tree/app-admin/filebeat/filebeat-6.5.4.ebuild . If necessary, I could convert this script to one that does actually use a tagged checkout if the changes are not clear enough. Please let me know if this helps. Thank you!

6

Any other error logs in kibana?

Using tcpdump to capture traffic (assuming no TLS) or curl might help us getting the complete response from kibana.

Install Filebeat, start kibana and run filebeat setup --dashboards manually so to install dashboards only. Starting tcpdump -w setup.pcap port 5601 before running filebeat setup should get us a trace we can inspect in detail using wireshark.

7

I looked into your issue and found that make kibana does not produce dashboards which can be loaded. There is a Python script which unpacks dashboard data: https://github.com/elastic/beats/blob/master/libbeat/scripts/unpack_dashboards.py
You need to run this script in order to get loadable dashboards.

See how dashboards are packaged: https://github.com/elastic/beats/blob/master/dev-tools/mage/kibana.go#L31

8

Thank you so much @kvch ! That script was just what I needed. Very unsure why it is not invoked automatically by make kibana - unless there is another make target that I needed to run? I was basing my work on the build script used by Arch here which does not seem to run that script, but everything is working great now. And thank you @steffens for your help.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.