# Get all docs with the occurrence of field values only once

**URL:** <https://discuss.elastic.co/t/get-all-docs-with-the-occurrence-of-field-values-only-once/166466>\
**Category:** Elasticsearch\
**Created:** [January 31, 2019, 4:37am UTC](https://discuss.elastic.co/t/get-all-docs-with-the-occurrence-of-field-values-only-once/166466 "2019-01-31T04:37:59Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kaushik.vankayala](https://avatars.discourse-cdn.com/v4/letter/k/aca169/32.png) [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\
**Post date:** [January 31, 2019, 4:37am UTC](https://discuss.elastic.co/t/get-all-docs-with-the-occurrence-of-field-values-only-once/166466/1 "2019-01-31T04:37:59Z")

</div>

Hi There,

I have the documents in my index in the following format;

```
{
  "_index": "mt_uat-app",
  "_type": "doc",
  "_source": {
    "Environment": "UAT",
    "loglevel": "INFO",
    "APITransactionId": "4d95e9b12cf0488084a22a8760c92c20",
    "audittype": "Audit-IN",
    "@timestamp": "2018-08-31T01:03:10.917Z",
    "Request": "POST /api/experience/member/login",
  },
  "fields": {
    "@timestamp": [
      "2018-08-31T01:03:10.917Z"
    ]
  }
}

```

another one say like below;

```
{
  "_index": "mt_uat-app",
  "_type": "doc",
  "_source": {
    "Environment": "UAT",
    "APITransactionId": "4d95e9b12cf0488084a22a8760c92c20",
    "audittype": "Audit-OUT",
    "@timestamp": "2018-08-31T01:03:12.917Z",
    "ResponseTime": "2 sec",
  },
  "fields": {
    "@timestamp": [
      "2018-08-31T01:03:10.917Z"
    ]
  }
}

```

Could anybody suggest how its possible to filter out the docs which have the count of APITransactionId exactly 1.

So in brief for every hit of an API, an Audit-IN and and an Audit-OUT will be generated which will have same APITransactionId. But if the API does not respond then we will **NOT** get an Audit-OUT.

I am looking for a DSL query probably with aggregation or an alternate approach to get all the docs which have the count of APITransactionId only 1, that way i get to filter out all docs which have only Audit-IN and NO Audit-OUT for a specific APITransactionId

_Can anybody help at the earliest!_

Regards

Kaushik

---

<div class="post-metadata">

**Author:** ![kaushik.vankayala](https://avatars.discourse-cdn.com/v4/letter/k/aca169/32.png) [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\
**Post date:** [February 26, 2019, 12:26pm UTC](https://discuss.elastic.co/t/get-all-docs-with-the-occurrence-of-field-values-only-once/166466/2 "2019-02-26T12:26:35Z")

</div>

Hello Team, Any help or update here? 🙄

---

<div class="post-metadata">

**Author:** ![cbuescher](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cbuescher/32/60402_2.png) [@cbuescher](https://discuss.elastic.co/u/cbuescher)\
**Post date:** [February 27, 2019, 9:19am UTC](https://discuss.elastic.co/t/get-all-docs-with-the-occurrence-of-field-values-only-once/166466/3 "2019-02-27T09:19:29Z")

</div>

I'd say you can try to use a `terms` aggregation on the `APITransactionId` field, the only problem with that is probably that the `size` would have to be high in order to catch all ids with count 1. A possible alternative could be the [Composite Aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html) introduced in I believe Elasticsearch 6.1. There you could use the `APITransactionId` as a terms value source and paginate over the result in ascending order, stopping when you reach the buckets with count 2. I haven't given it a try yet but I think it could work.

Cheers

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 27, 2019, 9:19am UTC](https://discuss.elastic.co/t/get-all-docs-with-the-occurrence-of-field-values-only-once/166466/4 "2019-03-27T09:19:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.