# Vega visualization in Kibana - unable to load data json

**URL:** <https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062>\
**Category:** Kibana\
**Created:** [September 19, 2018, 6:36am UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062 "2018-09-19T06:36:21Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![deuskars](https://avatars.discourse-cdn.com/v4/letter/d/b19c9b/32.png) [@deuskars](https://discuss.elastic.co/u/deuskars)\
**Post date:** [September 19, 2018, 6:36am UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/1 "2018-09-19T06:36:22Z")

</div>

Hi,

I am a beginner at Vega - I successfully got a "line chart" working with a inline dataset (specified with "Values") and wanted to take it to next step of getting the dataset from a json served via a URL.

When I try that, Kibana visualization area shows me:  
**Loading failed**  
**[http://localhost/jsons/ihs\_vega.json](http://localhost/jsons/ihs_vega.json)**

I have enabled  
`vega.enableExternalUrls: true`

My json is being served by nginx on my PC. Both http and https URLs do successfully serve the json when accessed through browser.

In Vega, I tried the url to be http as well as https - both give the same problem. Note: my https server is running with a self-signed certificate (don't know if that needs to be added to some truststore to make it work).

I turned on Kibana verbose logging (logging.verbose: true), but it didn't offer any clue.

I could not find out any other parameters to turn on vega logging/tracing.

**Environment Details:**  
Operating System: Windows 10  
Kibana and Elastic version: 6.4

**Complete vega code:**

```
{
  "$schema": "https://vega.github.io/schema/vega/v3.json",
  "width": 500,
  "height": 200,
  "padding": 5,
  "autosize": "pad",

  "data": [
    {
      "name": "ihsdata",
      "url": "https://localhost/jsons/ihs_vega.json",
      "format": {
        "type": "json",
        "parse": {"RequestTime": "date"}
      },
    }
  ],

  "scales": [
    { "name": "ixscale", "type": "time",
      "range": "width",
      "domain": {"data": "ihsdata", "field": "RequestTime"} },
    { "name": "iyscale", "type": "linear",
      "range": "height", "nice": true, "zero": false,
      "domain": {"data": "ihsdata", "field": "ResponseTimeMS"} }
  ],

  "axes": [
    {
      "orient": "left",
      "scale": "iyscale",
      "grid": true,
      "_tickCount": 5
    },
    {
      "orient": "bottom",
      "scale": "ixscale",
      "_tickCount": 5
    }
  ],

  "marks": [
    {
      "type": "line",
      "from": { "data": "ihsdata" },
      "encode": {
        "enter": {
          "stroke": {"value": "#F00"}
        },
        "update": {
          "x": {"scale": "ixscale", "field": "RequestTime"},
          "y": {"scale": "iyscale", "field": "ResponseTimeMS"},
          "strokeWidth": {"value": 2},
          "strokeDash": {"value": [15,3]},
          "opacity": {"value": 1},
          "stroke": {"value": "#F00"}
        },
        "hover" : {
          "strokeDash": {"value":[1,0]},
          "stroke": {"value": "#00F"}
        }
      }
    }
  ]
}

```

**ihs\_vega.json looks like:**

```
[
  {
    "RequestSize":52,
    "RequestType":"POST",
    "AppServer":"Server4",
    "ResponseTimeMS":29.6,
    "RequestDate":"2018-09-11",
    "RequestTime":"2018-09-11T05:00:00.000+0530",
    "IHS":"IHS3"},
  {
    "RequestSize":51,
    "RequestType":"POST",
    "AppServer":"Server5",
    "ResponseTimeMS":36.0,
    "RequestDate":"2018-09-11",
    "RequestTime":"2018-09-11T05:00:02.000+0530",
    "IHS":"IHS1"},
  {
    "RequestSize":54,
    "RequestType":"GET",
    "AppServer":"Server3",
    "ResponseTimeMS":37.4,
    "RequestDate":"2018-09-11",
    "RequestTime":"2018-09-11T05:00:07.000+0530",
    "IHS":"IHS2"}
]

```

Can someone point me to the right direction?

Thanks so much.

**24/Sep ... adding one more (related) question here**  
My kibana-vega visualization doesn't work for elasticsearch index as well. It doesn't give any errors, but it doesn't draw the desired picture. When my dataset points to "`values`", it works properly, but when I use the dataset as

```
{
    "name": "ihsdata",
    "url": { "index": "ihs-proxy-server*"
           }
}

```

it doesn't draw the desired picture. I don't know how to troubleshoot this.

I have one index called `ihs-proxy-server-2018-09-11` and its structure is as follows:

`http://localhost:9200/ihs-proxy-server-2018-09-11/?pretty=true`

```
{
  "ihs-proxy-server-2018-09-11" : {
    "aliases" : { },
    "mappings" : {
      "doc" : {
        "properties" : {
          "@timestamp" : { "type" : "date" },
          "@version" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } },
          "AppServer" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } },
          "IHS" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } },
          "RequestDate" : { "type" : "date" },
          "RequestSize" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } },
          "RequestTime" : { "type" : "date" },
          "RequestType" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } },
          "ResponseTimeMS" : { "type" : "float" },
          "host" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } },
          "message" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } },
          "path" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1537265561043",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "0ohmIIhASJqxRgFVRXrvgA",
        "version" : { "created" : "6040099" },
        "provided_name" : "ihs-proxy-server-2018-09-11"
      }
    }
  }
}

```

Thanks

-- Parag

---

<div class="post-metadata">

**Author:** ![rashmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rashmi/32/16391_2.png) [@rashmi](https://discuss.elastic.co/u/rashmi)\
**Post date:** [September 19, 2018, 1:29pm UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/2 "2018-09-19T13:29:08Z")

</div>

cc @nyuriks - any help here ?

Thanks  
Rashmi

---

<div class="post-metadata">

**Author:** ![nyuriks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nyuriks/32/26171_2.png) [@nyuriks](https://discuss.elastic.co/u/nyuriks)\
**Post date:** [September 25, 2018, 5:17am UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/3 "2018-09-25T05:17:52Z")

</div>

Hi @deuskars! The best way to troubleshoot something like this is to use browser's built-in debugging tools. There could be a number of things going wrong. Try these things:

- Open browser developer tools (in Chrome - menu icon, tools, dev tools). Open networking tab. Do some minor vega modification (e.g. add some top-level key like `title: graph title` and click the "play" button. Observe the networking requests your browser makes, and see if there are any issues - if it gives you the expected result, no http errors, etc.
- Try uploading your data to some well know well host, e.g. [https://gist.github.com/](https://gist.github.com/) and use the link to the RAW version of that file. This should work with both http and https.

P.S. this will also help with the Elasticsearch request

---

<div class="post-metadata">

**Author:** ![deuskars](https://avatars.discourse-cdn.com/v4/letter/d/b19c9b/32.png) [@deuskars](https://discuss.elastic.co/u/deuskars)\
**Post date:** [September 26, 2018, 11:57am UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/4 "2018-09-26T11:57:00Z")

</div>

Thanks @nyuriks and @rashmi ...

Regarding getting data from json - I was able to move forward a bit - as expected, it was an invalid cert error (since I was using self signed cert). Thanks for the tip about [gist.github.com](http://gist.github.com), it was indeed helpful. I am facing some different problems there (cors need to be enabled and facing some problems there).

Regarding getting data from elastic index (which is my real aim) - I am still facing some issues. The tip for F12 debug and then looking at VEGADEBUG.view.data was very useful. I still haven't solved my problem, but at least got a better view into it. The data returned by elasticsearch is much more nested json that my plain json. I tried to add `"format": { "property": "hits.hits" }` to the dataset, that helped to reduce the elasticsearch output significantly, but still didn't work fully. I need to do some more due diligence on this and then if needed I will add details here (or maybe in a separate thread).

Thanks.

---

<div class="post-metadata">

**Author:** ![deuskars](https://avatars.discourse-cdn.com/v4/letter/d/b19c9b/32.png) [@deuskars](https://discuss.elastic.co/u/deuskars)\
**Post date:** [September 27, 2018, 9:25am UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/5 "2018-09-27T09:25:35Z")

</div>

Hi @nyuriks and @rashmi,

Looks like I have been able to reach beyond these two problems now.

For json - I had to save it in raw format on [gist.github.com](http://gist.github.com), and then it worked. I haven't actually figured out how it returns the data differently if saved in raw or otherwise, but I decided to move on.

For index - apart from the `"format": { "property": "hits.hits" }`, I had to refer to my fields as "`_source.<field_name>`". The `VEGADEBUG.view.data()` has been an invaluable help.

Thanks so much.

---

<div class="post-metadata">

**Author:** ![tonyskulk](https://avatars.discourse-cdn.com/v4/letter/t/3be4f8/32.png) [@tonyskulk](https://discuss.elastic.co/u/tonyskulk)\
**Post date:** [September 28, 2018, 5:18pm UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/6 "2018-09-28T17:18:45Z")

</div>

Hi,

I´m starting to build my own vega-lite visualizations in kibana but I´m a little stuck. I think my problem is very much related to yours so I hope it´s ok to reply on your thread.

I´ve just built an index and filled it with over 5M entries. But I can´t manage to get any data to be shown on my plot. Here is my vega-lite spec:

```
{
  $schema: https://vega.github.io/schema/vega-lite/v2.json
  data: {
    name: healthData
    url: {
      %context%: true
      %timefield%: @timestamp
      index: gaptrader.heartbeat
      body: {
        size: 1000
      }
    }
    format: { property: "hits.hits" }
  }
  layer: [
    {
      mark: {type: "line", interpolate: "step-after"}
      encoding: {
        x: {
          field: timestamp
          type: temporal
          timeUnit: hoursminutesseconds
          axis: {grid: false, ticks: false, title: null}
        }
        y: {
          field: lastSignalSeconds
          type: quantitative
          scale: {
            domain: [0, 600]
            type: sqrt
          }
          axis: {grid: false, ticks: false, title: null}
        }
      }
    }
    {
      mark: line
      encoding: {
        x: {
          field: timestamp
          type: temporal
          timeUnit: hoursminutesseconds
          axis: {grid: false, ticks: false, title: null}
        }
        y: {field: "warning", type: "quantitative"}
        color: {value: "#f2c037"}
        opacity: {value: "0.5"}
      }
    }
    {
      mark: line
      encoding: {
        x: {
          field: timestamp
          type: temporal
          timeUnit: hoursminutesseconds
          axis: {grid: false, ticks: false, title: null}
        }
        y: {field: "error", type: "quantitative"}
        color: {value: "#db2828"}
        opacity: {value: "0.3"}
      }
    }
  ]
}

```

When I am looking at the network request Kibana sends I see that Kibana does not return any hits. What am I doing wrong?

Request-Body:

```
{"size":1000,"query":{"bool":{"must":[{"match_all":{}},{"range":{"@timestamp":{"gte":1380387787229,"lte":1538154187229,"format":"epoch_millis"}}}],"must_not":[]}}}

```

Response:

```
{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}

```

Maybe there is a basic mistake I made, writing the "data"-spec in Kibana. But I can not find out why I get empty hits.

---

<div class="post-metadata">

**Author:** ![deuskars](https://avatars.discourse-cdn.com/v4/letter/d/b19c9b/32.png) [@deuskars](https://discuss.elastic.co/u/deuskars)\
**Post date:** [September 29, 2018, 4:22pm UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/7 "2018-09-29T16:22:18Z")

</div>

> [@tonyskulk](#):
>
> %context%: true

@tonyskulk, I am not sure off hand, but one thing might be worth looking at is to remove the `%context%: true` specification. Your query seems to be looking for data where timestamp is between "Saturday, September 28, 2013 5:03:07.229 PM" (GMT) and "Friday, September 28, 2018 5:03:07.229 PM" (GMT) [Ref: [https://www.epochconverter.com/](https://www.epochconverter.com/)]. Is it possible that your index doesn't have a matching data for that range?

-- Parag

---

<div class="post-metadata">

**Author:** ![tonyskulk](https://avatars.discourse-cdn.com/v4/letter/t/3be4f8/32.png) [@tonyskulk](https://discuss.elastic.co/u/tonyskulk)\
**Post date:** [October 1, 2018, 5:18pm UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/8 "2018-10-01T17:18:28Z")

</div>

Thank you @deuskars for your answer.  
I tried different filter specs like last 5 years with or without **%context%: true** but could not get any data shown in my visualization.

I at least found something: When I use

```
index: *

```

then in the corresponding http request´s ( **.../elasticsearch/\*/\_search** ) response, there are over 1 million hits. Unfortunalety none of my **gaptrader.heartbeat** data, but at least all hits regarding to some **filebeat** \*-indexes.

So I think it has something to do with my vega-lite specification on the **data.url.index** attribute.

Currently I am using

```
index: gaptrader.heartbeat

```

and nothing has been returned. But when I run the same query in Kibana Dev Tools

```
GET gaptrader.streamgap/_search

```

I got over 5 million data points returned. So there must be a difference between dev tools and the **/\_search** endpoint...  
Does anyone has also faced that problem?

---

<div class="post-metadata">

**Author:** ![tonyskulk](https://avatars.discourse-cdn.com/v4/letter/t/3be4f8/32.png) [@tonyskulk](https://discuss.elastic.co/u/tonyskulk)\
**Post date:** [October 2, 2018, 10:29am UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/9 "2018-10-02T10:29:28Z")

</div>

Ok, I found the mistake. I used

```
%timefield%: @timestamp

```

Instead using

```
%timefield%: timestamp

```

results in the desired hits.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 30, 2018, 10:29am UTC](https://discuss.elastic.co/t/vega-visualization-in-kibana-unable-to-load-data-json/149062/10 "2018-10-30T10:29:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.