Raw data visualization takes too much time

Hi,
I am trying to create line chart visualization of field Y vs. PatchNo field.
I have more then 10K documents, but even if I limit the buckets size to 1000, it takes few minutes for the vis. to get updated.
why is it so slow?
and i can i visualize all the documents? (more then 10k..)
the chart script:

{
  "aggs": {
    "2": {
      "terms": {
        "field": "Strip.Patches.PatchNo",
        "order": {
          "2-orderAgg": "asc"
        },
        "size": 1000
      },
      "aggs": {
        "6": {
          "avg": {
            "field": "Y"
          }
        },
        "2-orderAgg": {
          "min": {
            "field": "Strip.Patches.PatchNo"
          }
        }
      }
    }
  },
  "size": 0,
  "_source": {
    "excludes": []
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {
    "PatchNo/SampleNo": {
      "script": {
        "source": "if (doc['Strip.IsRaw'].size() != 0) {\nif (doc['Strip.IsRaw'].value == true) { \nreturn doc['Strip.Patches.SampleNo'].value\n} else {\nreturn doc['Strip.Patches.PatchNo'].value\n}\n}",
        "lang": "painless"
      }
    }
  },
  "docvalue_fields": [
    {
      "field": "StartTime",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "match_phrase": {
            "Strip.IsRaw": {
              "query": true
            }
          }
        },
        {
          "range": {
            "StartTime": {
              "format": "strict_date_optional_time",
              "gte": "2021-01-19T19:39:56.870Z",
              "lte": "2021-01-21T02:27:09.253Z"
            }
          }
        }
      ],
      "filter": [
        {
          "match_all": {}
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

If Strip.Patches.PatchNo is a number field, consider using a histogram aggregation instead of terms to reduce the number of buckets Elasticsearch has to calculate.

Another option would be to try using a vega visualization (https://www.timroes.de/kibana-vega-scatterplot)

I tried with Vega,
however an empty chart received. Maybe because Strip.Patches.PatcNo is a nested field?
this is its format:

    "Strip": {
      "Id": 1,
      "Status": "OK",
      "Patches": [
        {
          "PatchNo": 1,
          "SampleNo": 2,
          "data": []
        }
      ]
}

and this is the viz. script:

{
  $schema: https://vega.github.io/schema/vega-lite/v2.json

  data: {
    data: table
    url: {
      %context%: true
      index: my-index
      body: {
        size: 1000
        _source: ["Y", "Strip.Patches.PatcNo"]
      }
    }
    format: {property: "hits.hits"}
  }
  mark: {type: "point", filled: true, size: 200 }
encoding: {
    x: {
      field: "_source.Strip.Patches.PatcNo"
      type: quantitative
      axis: {title: "PatchNo"}
    }
    y: {
    aggregate: "mean"
      field: "_source.Y"
      type: quantitative
      axis: {title: "Y"}
    }  
}
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.