# Elasticsearch terms aggregation and querrying

**URL:** https://discuss.elastic.co/t/elasticsearch-terms-aggregation-and-querrying/192748
**Category:** Elasticsearch
**Created:** [July 29, 2019, 5:06pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-and-querrying/192748 "2019-07-29T17:06:25Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Vladpov](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@Vladpov](https://discuss.elastic.co/u/Vladpov)
#### Post date: [July 29, 2019, 5:06pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-and-querrying/192748/1 "2019-07-29T17:06:25Z")

</div>

Hi I have two types of log messages:

`Jul 23 09:24:16 rrr mrr-core[222]: Aweg3AOMTs_1563866656871111.mt processMTMessage() #12798 realtime: 5.684 ms`

`Jul 23 09:24:18 rrr mrr-core[2222]: Aweg3AOMTs_1563866656871111.0.dn processDN() #7750 realtime: 1.382 ms`

First message is kind of sent message and second is message which confirm that message was delivered.

The difference between them is the suffix which I have separated from "id" and can query it.

These messages are parsed and stored in elasticsearch in following format:

```auto
messageId: Aweg3AOMTs_1563866656871111.0.dn
text: Aweg3AOMTs
num1: 1563866656871111
num2: 0
suffix: mt/dn

```

I would like to find out which messages were succesfully delivered and which weren't. I am very begginer in elasticsearch so I'm really struggling.

I'm trying terms aggregations at the moment but all I could've achived is this code:

```auto
GET /my_index3/_search
{
  "size": 0,
  "aggs": {
    "num1": {
      "terms": {
        "field": "messageId.keyword",
        "include": ".*mt*."
      }
    }
  } 
}

```

Which shows me the sent messages. I don't know how to add some filter there or clause that could show me only messages having both mt and dn suffix.

If anyone has an idea I'd be really thankfull :))

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 29, 2019, 5:18pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-and-querrying/192748/2 "2019-07-29T17:18:15Z")

</div>

Assuming there's a large number of unique messageIDs this is one of those tricky problems to do for any distributed data store.  
You'll likely need to maintain an entity-centric index keyed on the message ID rather than attempting this analysis on a purely log-centric index.

Here's a link to [why entity centric indexes are sometimes required](https://twitter.com/elasticmark/status/1009380268409610240). It includes some example scripts to build an entity-centric index but we also now have the [dataframes](https://www.elastic.co/guide/en/elasticsearch/reference/master/preview-data-frame-transform.html) feature in 7.2 which can also fuse related data around an ID.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 26, 2019, 5:26pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-and-querrying/192748/4 "2019-08-26T17:26:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
