How to transform the 2 separate items into one object to plot a scatter chart in vega

hi experts,
my data mapping is something like below:

LotNumber:keyword
Operation:keyword
Type:keyword
Pin:keyword
context:keyword
UnitId:keyword
value:double

below is my query of 2 different TestName measured value term by UnitId/TestName.
what I want to do is plot a scatter chart, X/Y are just the value of the 2 TestNames in the same UnitId.

GET ft-test-2019-01-01/_search?size=0
    {
      "query": {
        "bool": {
          "must": [
            {
              "term": {
                "LotNumber": {
                  "value": "HG50212"
                }
              }
            },
            {
              "term": {
                "Operation": {
                  "value": "FT"
                }
              }
            },
            {
              "term": {
                "Type": {
                  "value": "PinMeasure"
                }
              }
            },
            {
              "term": {
                "Pin": {
                  "value": "S_VDDCR_SOC"
                }
              }
            }
          ],
          "filter": {
            "terms": {
              "context": [
                "Sidd_VddCrSoc_V1",
                "Sidd_VddCrSoc_V3"
              ]
            }
          }
        }
      },
      
      "aggs": {
        "UnitId": {
          "terms": {
            "field": "UnitId"
          },
          "aggs": {
            "TestName": {
              "terms": {
                "field": "context"
              },
              "aggs": {
                "Avg": {
                  "avg": {
                    "field": "value"
                  }
                }
              }
            }
          }
        }
      }
    }

the response is something like:

  "aggregations": {
    "UnitId": {
      "doc_count_error_upper_bound": 5,
      "sum_other_doc_count": 455,
      "buckets": [
        {
          "key": "H801T0006080804",
          "doc_count": 6,
          "TestName": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Sidd_VddCrSoc_V1",
                "doc_count": 3,
                "Avg": {
                  "value": 2.884828964869181
                }
              },
              {
                "key": "Sidd_VddCrSoc_V3",
                "doc_count": 3,
                "Avg": {
                  "value": 4.808348655700684
                }
              }
            ]
          }
        },
        {
          "key": "H801T0006121104",
          "doc_count": 5,
          "TestName": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Sidd_VddCrSoc_V1",
                "doc_count": 3,
                "Avg": {
                  "value": 2.7391002972920737
                }
              },
              {
                "key": "Sidd_VddCrSoc_V3",
                "doc_count": 2,
                "Avg": {
                  "value": 4.6945860385894775
                }
              }
            ]
          }
        }
    ...

so now the problem is how to access the x: [TestName.Sidd_VddCrSoc_V1.Avg.value]
and y[TestName.Sidd_VddCrSoc_V3.Avg.value] from the aggregations result in vega?

I also try use the composite agg and the result is something lie below, and dont know if can I use the vega fitler transform

     "aggregations": {
        "table": {
          "after_key": {
            "UnitID": "H801T0006051204",
            "TestName": "Sidd_VddCrSoc_V3"
          },
          "buckets": [
            {
              "key": {
                "UnitID": "H801T0006021304",
                "TestName": "Sidd_VddCrSoc_V1"
              },
              "doc_count": 1,
              "NAME": {
                "value": 2.412668466567993
              }
            },
            {
              "key": {
                "UnitID": "H801T0006021304",
                "TestName": "Sidd_VddCrSoc_V3"
              },
              "doc_count": 1,
              "NAME": {
                "value": 4.2574052810668945
              }
            },
            {
              "key": {
                "UnitID": "H801T0006031004",
                "TestName": "Sidd_VddCrSoc_V1"
              },
              "doc_count": 2,
              "NAME": {
                "value": 2.6428310871124268
              }
            },
            {
              "key": {
                "UnitID": "H801T0006031004",
                "TestName": "Sidd_VddCrSoc_V3"
              },
              "doc_count": 2,
              "NAME": {
                "value": 4.514222621917725
              }
            },
...

let me know if you have any idea.
thanks!

@nyuriks thanks :slight_smile:

Hi, could you post your "semi-working" graph following these steps? This way it would be much easier to try things out and point out any possible solutions. Thanks!

thanks @nyuriks

we have already fixed it and please close this ticket please.

thank you again for your amazing idea on the Vega vis.