How to read Data from ES response

Elastic Stack Elasticsearch
1

Hi, so I see the response of ES is tied to the kind of Query I write. I know Kibana is able to map it into charts so there must be a formal way of extracting information of BUCKETS property. Can I get info about it or is it a open-source lib?

Example Request

{
    "size": 0,
    "aggs": {
        "agg_project": {
            "terms": {
                "field": "project.code"
            },
            "aggs": {
                "agg_custom_field": {
                    "nested": {
                        "path": "traceability"
                    },
                    "aggs": {
                        "agg_value": {
                            "terms": {
                                "field": "traceability.status"
                            }
                        }
                    }
                }
            }
        }
    }
}

Response

{
  "agg_project": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 3607,
    "buckets": [
      {
        "key": "NXTMigrationTest",
        "doc_count": 1546,
        "agg_custom_field": {
          "doc_count": 2674,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 156,
            "buckets": [
              {
                "key": "New",
                "doc_count": 635
              },
              {
                "key": "Ready for Release",
                "doc_count": 521
              },
              {
                "key": "Done",
                "doc_count": 400
              },
              {
                "key": "SIT",
                "doc_count": 275
              },
              {
                "key": "Closed",
                "doc_count": 224
              },
              {
                "key": "Internal Review Done",
                "doc_count": 136
              },
              {
                "key": "Ready for UAT",
                "doc_count": 91
              },
              {
                "key": "Build in progress",
                "doc_count": 85
              },
              {
                "key": "To Do",
                "doc_count": 85
              },
              {
                "key": "Rejected",
                "doc_count": 66
              }
            ]
          }
        }
      },
      {
        "key": "copyofInvocare120223_1",
        "doc_count": 1232,
        "agg_custom_field": {
          "doc_count": 280,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Done",
                "doc_count": 216
              },
              {
                "key": "Rejected",
                "doc_count": 22
              },
              {
                "key": "Closed",
                "doc_count": 13
              },
              {
                "key": "Ready for Release",
                "doc_count": 13
              },
              {
                "key": "To Do",
                "doc_count": 6
              },
              {
                "key": "Open",
                "doc_count": 5
              },
              {
                "key": "READY TO TEST",
                "doc_count": 2
              },
              {
                "key": "FIX IN PROGRESS",
                "doc_count": 1
              },
              {
                "key": "New",
                "doc_count": 1
              },
              {
                "key": "Prioritised",
                "doc_count": 1
              }
            ]
          }
        }
      },
      {
        "key": "AutomationDemo",
        "doc_count": 1219,
        "agg_custom_field": {
          "doc_count": 280,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Done",
                "doc_count": 216
              },
              {
                "key": "Rejected",
                "doc_count": 22
              },
              {
                "key": "Closed",
                "doc_count": 13
              },
              {
                "key": "Ready for Release",
                "doc_count": 13
              },
              {
                "key": "To Do",
                "doc_count": 6
              },
              {
                "key": "Open",
                "doc_count": 5
              },
              {
                "key": "READY TO TEST",
                "doc_count": 2
              },
              {
                "key": "FIX IN PROGRESS",
                "doc_count": 1
              },
              {
                "key": "New",
                "doc_count": 1
              },
              {
                "key": "Prioritised",
                "doc_count": 1
              }
            ]
          }
        }
      },
      {
        "key": "copyofInvocare030123",
        "doc_count": 1214,
        "agg_custom_field": {
          "doc_count": 280,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Done",
                "doc_count": 216
              },
              {
                "key": "Rejected",
                "doc_count": 22
              },
              {
                "key": "Closed",
                "doc_count": 13
              },
              {
                "key": "Ready for Release",
                "doc_count": 13
              },
              {
                "key": "To Do",
                "doc_count": 6
              },
              {
                "key": "Open",
                "doc_count": 5
              },
              {
                "key": "READY TO TEST",
                "doc_count": 2
              },
              {
                "key": "FIX IN PROGRESS",
                "doc_count": 1
              },
              {
                "key": "New",
                "doc_count": 1
              },
              {
                "key": "Prioritised",
                "doc_count": 1
              }
            ]
          }
        }
      },
      {
        "key": "copyofInvocare02122022",
        "doc_count": 1205,
        "agg_custom_field": {
          "doc_count": 265,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Done",
                "doc_count": 206
              },
              {
                "key": "Rejected",
                "doc_count": 22
              },
              {
                "key": "Closed",
                "doc_count": 13
              },
              {
                "key": "Ready for Release",
                "doc_count": 13
              },
              {
                "key": "Open",
                "doc_count": 5
              },
              {
                "key": "READY TO TEST",
                "doc_count": 2
              },
              {
                "key": "To Do",
                "doc_count": 2
              },
              {
                "key": "FIX IN PROGRESS",
                "doc_count": 1
              },
              {
                "key": "Prioritised",
                "doc_count": 1
              }
            ]
          }
        }
      },
      {
        "key": "copyofInvocare21112022",
        "doc_count": 1169,
        "agg_custom_field": {
          "doc_count": 262,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Done",
                "doc_count": 203
              },
              {
                "key": "Rejected",
                "doc_count": 22
              },
              {
                "key": "Closed",
                "doc_count": 13
              },
              {
                "key": "Ready for Release",
                "doc_count": 13
              },
              {
                "key": "Open",
                "doc_count": 5
              },
              {
                "key": "READY TO TEST",
                "doc_count": 2
              },
              {
                "key": "To Do",
                "doc_count": 2
              },
              {
                "key": "FIX IN PROGRESS",
                "doc_count": 1
              },
              {
                "key": "Prioritised",
                "doc_count": 1
              }
            ]
          }
        }
      },
      {
        "key": "copyofInvocare311022",
        "doc_count": 1147,
        "agg_custom_field": {
          "doc_count": 0,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": []
          }
        }
      },
      {
        "key": "copyofInvocare251022",
        "doc_count": 1137,
        "agg_custom_field": {
          "doc_count": 0,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": []
          }
        }
      },
      {
        "key": "copyofInvocare26092022",
        "doc_count": 1119,
        "agg_custom_field": {
          "doc_count": 0,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": []
          }
        }
      },
      {
        "key": "eCommerce",
        "doc_count": 839,
        "agg_custom_field": {
          "doc_count": 1586,
          "agg_value": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Done",
                "doc_count": 1201
              },
              {
                "key": "To Do",
                "doc_count": 134
              },
              {
                "key": "In Progress",
                "doc_count": 129
              },
              {
                "key": "In QA",
                "doc_count": 68
              },
              {
                "key": "PO Approval",
                "doc_count": 23
              },
              {
                "key": "Code review",
                "doc_count": 17
              },
              {
                "key": "Ready for QA",
                "doc_count": 8
              },
              {
                "key": "On Hold",
                "doc_count": 6
              }
            ]
          }
        }
      }
    ]
  }
}
2

You can use something like jq to extract this from the json, but that may not help your situation. Can you provide more info on what you are looking to do so we can provide contextually specific help?

3

So we want to render charts on the result of ES queries, but we need to convert ES response to Charts input type. And since queries could of any type, the response comes in same structure.

We just want to code the middleware to support all such transformations

4

Are you basically saying you want to build your own Kibana?

5

Not like kibana. Just the charts rendering static. We already have our dashboards, its an addition to our exisiting codebase.

6

Maybe look at GitHub - elastic/eui: Elastic UI Framework 🙌?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.