# Calculating count and percentage

**URL:** https://discuss.elastic.co/t/calculating-count-and-percentage/229029
**Category:** Logstash
**Created:** [April 21, 2020, 12:12pm UTC](https://discuss.elastic.co/t/calculating-count-and-percentage/229029 "2020-04-21T12:12:10Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![rcganesh](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rcganesh](https://discuss.elastic.co/u/rcganesh)
#### Post date: [April 21, 2020, 12:12pm UTC](https://discuss.elastic.co/t/calculating-count-and-percentage/229029/1 "2020-04-21T12:12:10Z")

</div>

Hello Experts,

I am couple of weeks old to Logstash and I need some help in trasforming data through logstash and send it to either some file or Elasticsearch. Below is the data-flow we are implementing.

1. Input data into logstash from Elasticsearch
2. Transform data received from Elasticsearch
3. Output the transformed data into some file or Elasticsearch

**What data to be transformed?**  
We need to calculate the percentage of `"ServiceCommonAttributes.ExternalStatus.StatusCode"` _**(refer Elasticsearch input below)**_ based some values in this field. For instance, if `"ServiceCommonAttributes.ExternalStatus.StatusCode"` in ('0','41100') is considered success. So, we need to evaluate this data for, say, last 5 minutes and calculate percentage of success for last 5 minutes.

**What challenge we are facing?**  
The biggest challenge we are facing here is to get the total count **(not sum)** of `"ServiceCommonAttributes.ExternalStatus.StatusCode"` filed values for last 5 minutes. I believe, once we figure out how to get the count for last 5 minutes, we can use RUBY filter to calculate the percentage.

**Is this the right approach?**  
We have achieved calculating percentage by _"Metaquery"_ plugin in Grafana. However, this plugin doesnot support alert notification. So, the other option we thought of is to use logstash.

After lots of reserach and googling, we found that [Metrics](https://www.elastic.co/guide/en/logstash/current/plugins-filters-metrics.html) filter can be used for calculating count. However, there is no much documentation on how to use this filter for Elasticsearch input data. If not [Metrics](https://www.elastic.co/guide/en/logstash/current/plugins-filters-metrics.html) filter, then any other filter which can fulfill the requirement will do.

Any help, suggestions or inputs are really appreciated.

Thanks in advance.

Ravi Chandran

**Elasticserach Input:**

```
{
    "_index": "MON",
    "_type": "_doc",
    "_id": "jSpSlHExC4JXqSgNty-7Of",
    "_version": 1,
    "_score": null,
    "_source":
    {
        "gte": "http://www.somelink.com",
        "ServiceCommonAttributes":
        {
            "MessageIdentifier": "Monitoring\\TXN",
            "ProcessName": "My Query",
            "ResponseStatus": "Technical Failure",
            "InternalStatus":
            {
                "StatusCode": "191",
                "StatusDescription": "ERROR:191:Account is not active."
            },
            "ServiceStartTime": "2020-04-20T00:25:22.063+03:00",
            "ServiceEndTime": "2020-04-20T00:25:22.113+03:00",
            "ExternalStatus":
            {
                "StatusCode": "41028",
                "StatusDescription": "Account is not active."
            },
            "ResponseTimeE2E": 50
        },
        "fields":
		{
            "ServiceCommonAttributes.ServiceEndTime": ["2020-04-19T21:25:22.113Z"],
            "ServiceCommonAttributes.ServiceStartTime": ["2020-04-19T21:25:22.063Z"]
        },
        "sort": [1587331522063]
    }
}

```

**Logstash config:**

```
input
{
    elasticsearch
    {
        hosts => "es-server"
        index => "MON"
        query => '{ "query": { "match_all": {} } }'
        scroll => "5m"
        schedule => "*/2 * * * *"
    }
}

filter
{
    if [StatusCode] >= 0
    {
        metrics
        {
            meter => "[ServiceCommonAttributes][ExternalStatus][StatusCode]"
            add_tag => "process"
        }
    }
}

output
{
    # only emit events with the 'metric' tag
    elasticsearch
    {
        hosts => ["es-server:9200"]
        index => "MON_logstash"
    }

    if "process" in [tags]
    {
        stdout
        {
            codec => line
            {
                format => "count: %{[ServiceCommonAttributes][ExternalStatus][StatusCode][count]}"
            }
        }
    }
}
```

---

<div class="post-metadata">

### Author: ![rcganesh](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rcganesh](https://discuss.elastic.co/u/rcganesh)
#### Post date: [April 28, 2020, 2:03pm UTC](https://discuss.elastic.co/t/calculating-count-and-percentage/229029/2 "2020-04-28T14:03:36Z")

</div>

Could someone please help me out with this?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 26, 2020, 2:03pm UTC](https://discuss.elastic.co/t/calculating-count-and-percentage/229029/3 "2020-05-26T14:03:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.