# Count of not unique term's

**URL:** <https://discuss.elastic.co/t/count-of-not-unique-terms/265246>\
**Category:** Kibana\
**Created:** [February 23, 2021, 7:14pm UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246 "2021-02-23T19:14:44Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![artobstrel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/artobstrel/32/16874_2.png) [@artobstrel](https://discuss.elastic.co/u/artobstrel)\
**Post date:** [February 23, 2021, 7:14pm UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/1 "2021-02-23T19:14:44Z")

</div>

There is my sample dataset:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/5/c536015b46dc49f5de7c0ae899e66c50749d2bac.png)

I can get a list of not unique text's in the table:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/b/1bdb3bb6f49146936b1dcba81067a755c109ce96.png)

> **source**
>
> ```
> {
> "aggs": {
> "2": {
> "terms": {
> "field": "text.keyword",
> "order": {
> "_count": "desc"
> },
> "size": 5,
> "min_doc_count": 2
> }
> }
> },
> "size": 0,
> "fields": [
> {
> "field": "@timestamp",
> "format": "date_time"
> }
> ],
> "script_fields": {},
> "stored_fields": [
> "*"
> ],
> "_source": {
> "excludes": []
> },
> "query": {
> "bool": {
> "must": [],
> "filter": [
> {
> "match_all": {}
> },
> {
> "range": {
> "@timestamp": {
> "gte": "2021-02-22T22:00:00.000Z",
> "lte": "2021-02-23T21:59:59.999Z",
> "format": "strict_date_optional_time"
> }
> }
> }
> ],
> "should": [],
> "must_not": []
> }
> }
> }
> 
> ```

**How to get the count of not unique text's in the metric visualisation?**

In my case it should be like this:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/2/22d9bd94a103792e37029881a75517b11d9e79af.png)

---

<div class="post-metadata">

**Author:** ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)\
**Post date:** [February 23, 2021, 7:50pm UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/2 "2021-02-23T19:50:32Z")

</div>

This requires something more than the standard Kibana visualizations. You basically have two options:

1. Use [Vega](https://www.elastic.co/guide/en/kibana/current/vega.html) because it supports math
2. Change the data that you store in Elasticsearch by using [Data transforms](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) and scripting

---

<div class="post-metadata">

**Author:** ![artobstrel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/artobstrel/32/16874_2.png) [@artobstrel](https://discuss.elastic.co/u/artobstrel)\
**Post date:** [February 24, 2021, 12:33pm UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/3 "2021-02-24T12:33:30Z")

</div>

Thank you @wylie

1. Vega does not contain visualization of type metric
2. I'll try but transforms too cumbersome for such task ☹

---

<div class="post-metadata">

**Author:** ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)\
**Post date:** [February 24, 2021, 1:51pm UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/4 "2021-02-24T13:51:15Z")

</div>

Vega is absolutely capable of displaying a single text field. Use `mark: text`.

---

<div class="post-metadata">

**Author:** ![artobstrel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/artobstrel/32/16874_2.png) [@artobstrel](https://discuss.elastic.co/u/artobstrel)\
**Post date:** [February 24, 2021, 3:08pm UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/5 "2021-02-24T15:08:41Z")

</div>

I need to display metric — not text.

---

<div class="post-metadata">

**Author:** ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)\
**Post date:** [February 24, 2021, 3:29pm UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/6 "2021-02-24T15:29:16Z")

</div>

Yes, I'm trying to tell you that those are the same thing in Vega

---

<div class="post-metadata">

**Author:** ![artobstrel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/artobstrel/32/16874_2.png) [@artobstrel](https://discuss.elastic.co/u/artobstrel)\
**Post date:** [February 26, 2021, 5:10pm UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/7 "2021-02-26T17:10:03Z")

</div>

To get what I need, I can make such a request:

> **Request**
>
> ```
> GET /<my_index>/_search
> {  
> "size":0,
> "aggs":{  
> "duplicates":{  
> "terms":{  
> "field":"text.keyword",
> "min_doc_count":2
> }
> },
> "duplicates_count":{  
> "stats_bucket":{  
> "buckets_path":"duplicates._count"
> }
> },
> }
> }
> 
> ```

Sample response:

> **Response**
>
> ```
> {
> "took" : 0,
> "timed_out" : false,
> "_shards" : {
> "total" : 1,
> "successful" : 1,
> "skipped" : 0,
> "failed" : 0
> },
> "hits" : {
> "total" : {
> "value" : 10,
> "relation" : "eq"
> },
> "max_score" : null,
> "hits" : []
> },
> "aggregations" : {
> "duplicates" : {
> "doc_count_error_upper_bound" : 0,
> "sum_other_doc_count" : 0,
> "buckets" : [
> {
> "key" : "bar",
> "doc_count" : 3
> },
> {
> "key" : "foo",
> "doc_count" : 2
> }
> ]
> },
> "duplicates_count" : {
> "count" : 2,
> "min" : 2.0,
> "max" : 3.0,
> "avg" : 2.5,
> "sum" : 5.0
> }
> }
> }
> 
> ```

But I can't figure out how to visualize **duplicates\_count** metric  
Can you give me an example?

---

<div class="post-metadata">

**Author:** ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)\
**Post date:** [February 27, 2021, 7:45pm UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/8 "2021-02-27T19:45:24Z")

</div>

I have written several tutorials explaining how to connect Elasticsearch queries to Vega output:

> **[Vega | Kibana Guide \[7.11\] | Elastic](https://www.elastic.co/guide/en/kibana/current/vega.html)**

```auto
{
  $schema: https://vega.github.io/schema/vega-lite/v4.json
  data: {
    url: {
      %context%: true
      %timefield%: order_date
      index: kibana_sample_data_ecommerce
      body: {
        aggs: // add your stuff here
        size: 0
      }
    }
    format: {property: "aggregations.path.to.bucket" }
  }
  mark: text
  encoding: {
    text: {
      field: path.to.count
    }
  }

```

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [February 28, 2021, 2:41am UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/9 "2021-02-28T02:41:51Z")

</div>

Here is an example of the transformations of what I think you are looking for if you wanted to do it all in Vega. Output is `2`.

```auto
{
  "$schema": "https://vega.github.io/schema/vega/v5.json",
  "data": [
    {
      "name": "table",
      "values": [
        {"id": 1, "text": "foo"},
        {"id": 2, "text": "bar"},
        {"id": 3, "text": "abc"},
        {"id": 4, "text": "def"},
        {"id": 5, "text": "foo"},
        {"id": 6, "text": "ghi"},
        {"id": 7, "text": "bar"},
        {"id": 8, "text": "jkl"}
      ],
      "transform": [
        {
          "type": "aggregate",
          "groupby": ["text"],
          "fields": ["text"],
          "ops": ["valid"],
          "as": ["matched"]
        },
        {
          "type": "formula",
          "as": "matched-count",
          "expr": "datum.matched == 2 ? 1 : 0"
        },
        {
          "type": "aggregate",
          "fields": ["matched-count"],
          "ops": ["sum"],
          "as": ["sum_match_count"]
        }
      ]
    }
  ],
  "marks": [
    {
      "type": "text",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "fill": {"value": "#333"},
          "x": {"value": 10},
          "y": {"value": 10},
          "text": {"field": "sum_match_count"}
        }
      }
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 28, 2021, 2:42am UTC](https://discuss.elastic.co/t/count-of-not-unique-terms/265246/10 "2021-03-28T02:42:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.