# Raw data visualization takes too much time

**URL:** <https://discuss.elastic.co/t/raw-data-visualization-takes-too-much-time/262479>\
**Category:** Kibana\
**Created:** [January 28, 2021, 10:40am UTC](https://discuss.elastic.co/t/raw-data-visualization-takes-too-much-time/262479 "2021-01-28T10:40:25Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shalhevet](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Shalhevet](https://discuss.elastic.co/u/Shalhevet)\
**Post date:** [January 28, 2021, 10:40am UTC](https://discuss.elastic.co/t/raw-data-visualization-takes-too-much-time/262479/1 "2021-01-28T10:40:25Z")

</div>

Hi,  
I am trying to create line chart visualization of field Y vs. PatchNo field.  
I have more then 10K documents, but even if I limit the buckets size to 1000, it takes few minutes for the vis. to get updated.  
why is it so slow?  
and i can i visualize all the documents? (more then 10k..)  
the chart script:

```auto
{
  "aggs": {
    "2": {
      "terms": {
        "field": "Strip.Patches.PatchNo",
        "order": {
          "2-orderAgg": "asc"
        },
        "size": 1000
      },
      "aggs": {
        "6": {
          "avg": {
            "field": "Y"
          }
        },
        "2-orderAgg": {
          "min": {
            "field": "Strip.Patches.PatchNo"
          }
        }
      }
    }
  },
  "size": 0,
  "_source": {
    "excludes": []
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {
    "PatchNo/SampleNo": {
      "script": {
        "source": "if (doc['Strip.IsRaw'].size() != 0) {\nif (doc['Strip.IsRaw'].value == true) { \nreturn doc['Strip.Patches.SampleNo'].value\n} else {\nreturn doc['Strip.Patches.PatchNo'].value\n}\n}",
        "lang": "painless"
      }
    }
  },
  "docvalue_fields": [
    {
      "field": "StartTime",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "match_phrase": {
            "Strip.IsRaw": {
              "query": true
            }
          }
        },
        {
          "range": {
            "StartTime": {
              "format": "strict_date_optional_time",
              "gte": "2021-01-19T19:39:56.870Z",
              "lte": "2021-01-21T02:27:09.253Z"
            }
          }
        }
      ],
      "filter": [
        {
          "match_all": {}
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

```

 ![Capture](https://us1.discourse-cdn.com/elastic/original/3X/c/3/c3bbb7d4fab7695dec1cbbd16a2720308692e427.png)

---

<div class="post-metadata">

**Author:** ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)\
**Post date:** [January 28, 2021, 11:01am UTC](https://discuss.elastic.co/t/raw-data-visualization-takes-too-much-time/262479/2 "2021-01-28T11:01:28Z")

</div>

If `Strip.Patches.PatchNo` is a number field, consider using a histogram aggregation instead of terms to reduce the number of buckets Elasticsearch has to calculate.

Another option would be to try using a vega visualization ([https://www.timroes.de/kibana-vega-scatterplot](https://www.timroes.de/kibana-vega-scatterplot))

---

<div class="post-metadata">

**Author:** ![Shalhevet](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Shalhevet](https://discuss.elastic.co/u/Shalhevet)\
**Post date:** [January 28, 2021, 1:39pm UTC](https://discuss.elastic.co/t/raw-data-visualization-takes-too-much-time/262479/3 "2021-01-28T13:39:33Z")

</div>

I tried with Vega,  
however an empty chart received. Maybe because Strip.Patches.PatcNo is a nested field?  
this is its format:

```auto
    "Strip": {
      "Id": 1,
      "Status": "OK",
      "Patches": [
        {
          "PatchNo": 1,
          "SampleNo": 2,
          "data": []
        }
      ]
}

```

and this is the viz. script:

```auto
{
  $schema: https://vega.github.io/schema/vega-lite/v2.json

  data: {
    data: table
    url: {
      %context%: true
      index: my-index
      body: {
        size: 1000
        _source: ["Y", "Strip.Patches.PatcNo"]
      }
    }
    format: {property: "hits.hits"}
  }
  mark: {type: "point", filled: true, size: 200 }
encoding: {
    x: {
      field: "_source.Strip.Patches.PatcNo"
      type: quantitative
      axis: {title: "PatchNo"}
    }
    y: {
    aggregate: "mean"
      field: "_source.Y"
      type: quantitative
      axis: {title: "Y"}
    }  
}
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 25, 2021, 1:39pm UTC](https://discuss.elastic.co/t/raw-data-visualization-takes-too-much-time/262479/4 "2021-02-25T13:39:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.