# Could I query all docs where there are multiple occurrences of my term?

**URL:** https://discuss.elastic.co/t/could-i-query-all-docs-where-there-are-multiple-occurrences-of-my-term/163276
**Category:** Elasticsearch
**Created:** [January 7, 2019, 10:22pm UTC](https://discuss.elastic.co/t/could-i-query-all-docs-where-there-are-multiple-occurrences-of-my-term/163276 "2019-01-07T22:22:01Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![tchung](https://avatars.discourse-cdn.com/v4/letter/t/ce73a5/32.png) [@tchung](https://discuss.elastic.co/u/tchung)
#### Post date: [January 7, 2019, 10:22pm UTC](https://discuss.elastic.co/t/could-i-query-all-docs-where-there-are-multiple-occurrences-of-my-term/163276/1 "2019-01-07T22:22:01Z")

</div>

I want to use Elasticsearch to index event data and as events come in, I want to append to an "events" array in the document. Say it ends up looking like this:

```
{
  "user":"abc",
  "events": [
    {
      "action": "foo",
      "date": "2015-01-01"
    },
    {
      "action": "foo",
      "date": "2016-01-01"
    },
    {
      "action": "bar",
      "date": "2017-01-01"
    }
  ]
}

```

And another doc looks like:

```
{
  "user":"def",
  "events": [
    {
      "action": "foo",
      "date": "2017-01-01"
    },
    {
      "action": "baz",
      "date": "2018-01-01"
    }
  ]
}

```

I get that I can query/filter for all users/docs with action=foo but could I:

- find all the documents where action=foo has occurred, say, exactly 2 times?
- find all the documents where action=foo has occurred 2 times between some start and end date.

Thanks in advance.

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [January 11, 2019, 11:08am UTC](https://discuss.elastic.co/t/could-i-query-all-docs-where-there-are-multiple-occurrences-of-my-term/163276/2 "2019-01-11T11:08:40Z")

</div>

To avoid "cross matching" of the dates for `foo` and `bar` you'll need to use the nested data type for the events.

I've not yet figured out how to trim results to those with 2 _different_ events but this should at least avoid confusing matches on the dates of your "foo" events with those of "bar" events::

```
DELETE test
PUT test
{
  "mappings": {
	"_doc": {
	  "properties": {
		"user":{
		  "type":"keyword"
		},
		"events": {
		  "type": "nested",
		  "properties": {
			"action": {
			  "type": "keyword"
			},
			"date":{
			  "type":"date"
			}
		  }
		}
	  }
	}
  }
}
PUT test/_doc/1
{
  "user":"abc",
  "events": [
	{
	  "action": "foo",
	  "date": "2015-01-01"
	},
	{
	  "action": "foo",
	  "date": "2016-01-01"
	},
	{
	  "action": "bar",
	  "date": "2017-01-01"
	}
  ]
}
PUT test/_doc/2
{
  "user":"abc",
  "events": [
	{
	  "action": "foo",
	  "date": "2015-01-01"
	},
	{
	  "action": "bar",
	  "date": "2016-01-01"
	},
	{
	  "action": "foo",
	  "date": "2017-01-01"
	}
  ]
}
GET test/_search
{
  "query": {
	"bool": {
	  "must": [
		{
		  "nested": {
			"path": "events",
			"query": {
			  "bool": {
				"must": [
				  {
					"match":{
					  "events.action":"foo"
					}
				  },
				  {
					"range": {
					  "events.date": {
						"gte": "2013-01-01",
						"lte": "2016-12-31"
					  }
					}
				  }
				]
			  }
			},
			"inner_hits": {}
		  }
		}
	  ]
	}
  },
  "stored_fields": [
	"user"
  ]
}

```

The use of `inner_hits` prevents you from seeing non-matching events. Your client would need to trim results that only matched one event,

Having said that, what can be hard to match at query time with searches/aggregations can be trivial at index time using an update script - see [entity centric indexing](https://twitter.com/elasticmark/status/1009380268409610240)  
Useful attributes like an "outage" range field can be derived by a script recognising the 2 event types of interest and storing a derived value when those events come in as an update.

---

<div class="post-metadata">

### Author: ![tchung](https://avatars.discourse-cdn.com/v4/letter/t/ce73a5/32.png) [@tchung](https://discuss.elastic.co/u/tchung)
#### Post date: [January 14, 2019, 7:39pm UTC](https://discuss.elastic.co/t/could-i-query-all-docs-where-there-are-multiple-occurrences-of-my-term/163276/3 "2019-01-14T19:39:32Z")

</div>

Thanks @Mark_Harwood. I watched the video about entity centric indexing and it looks good and I will try that approach.

I think I can index individual events as they occur (the "doc" will be the event) into an "events" index. To find users that have done "foo exactly twice" (like in my original post), I can periodically update a secondary "user" index and a user doc, that has a "didFooTwice" boolean. Then a simple bool query can get me what I want.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 11, 2019, 7:39pm UTC](https://discuss.elastic.co/t/could-i-query-all-docs-where-there-are-multiple-occurrences-of-my-term/163276/4 "2019-02-11T19:39:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
