Elastic Search - How to make stats bucket over calculated bucket using bucket script and bucket path

Elastic Stack Elasticsearch
1

Lets start with a simple query in an Elastic Search netflow database. I need to get the sum of network bytes aggregated by network.direction, so I do:

{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {
              "gte": "now-30m",
              "lte": "now"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "table": {
      "terms": {
        "field": "network.direction",
        "order": {
          "agg_query_order": "desc"
        },
        "size": 10
      },
      "aggs": {
        "sum_network_bytes": {
          "sum": {
            "field": "network.bytes"
          }
        },
				
        "agg_query_order": {
          "sum": {
            "field": "network.bytes"
          }
        }				
				
      }
    }
  },
  "size": 0
}

Here I have an aggregation called sum_network_bytes to get sum "network.bytes" per "network.direction" and another aggregation called agg_query_order just to order the results.

Now i need to add a stats for "network.bytes" by "network.direction", and I do this adding the code below:

		"stats_network_bytes": {
          "stats": {
            "field": "network.bytes"
          }
        }

resulting in:

{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {
              "gte": "now-30m",
              "lte": "now"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "table": {
      "terms": {
        "field": "network.direction",
        "order": {
          "agg_query_order": "desc"
        },
        "size": 10
      },
      "aggs": {
        "sum_network_bytes": {
          "sum": {
            "field": "network.bytes"
          }
        },
				
        "agg_query_order": {
          "sum": {
            "field": "network.bytes"
          }
        },
		
		"stats_network_bytes": {
          "stats": {
            "field": "network.bytes"
          }
        }					
      }
    }
  },
  "size": 0
}

This added another aggregation called stats_network_bytes with "network.bytes" stats.

My sample rate is 1:1000 (one netflow of each 1000) and I need to multiply the results by 1000 to get accurate results. I need also to multiply by 8 to get results in bits instead of bytes, so I will add the code below

        "sum_network_bits": {
          "bucket_script": {
            "buckets_path": {
              "total_value": "sum_network_bytes"
            },
            "script": "params.total_value * 1000 * 8"
          }
        }

And by whole code becomes:

{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {
              "gte": "now-30m",
              "lte": "now"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "table": {
      "terms": {
        "field": "network.direction",
        "order": {
          "agg_query_order": "desc"
        },
        "size": 10
      },
      "aggs": {
        "sum_network_bytes": {
          "sum": {
            "field": "network.bytes"
          }
        },
				
        "agg_query_order": {
          "sum": {
            "field": "network.bytes"
          }
        },
		
				"stats_network_bytes": {
          "stats": {
            "field": "network.bytes"
          }
        },
				
        "sum_network_bits": {
          "bucket_script": {
            "buckets_path": {
              "total_value": "sum_network_bytes"
            },
            "script": "params.total_value * 1000 * 8"
          }
        }				

      }
    }
  },
  "size": 0
}

Now I have a new bucket called sum_network_bits with the total bits per "network.direction".

How can I add a stats bucket for sum_network_bits metric ?

I need also to get the speed (in bits per second) but without get date_histogram aggregation.

2

From Elastic Search to Elasticsearch

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.