# How to search for a first occurrence of a term **URL:** \ **Category:** Elasticsearch\ **Created:** [September 21, 2018, 9:50am UTC](https://discuss.elastic.co/t/how-to-search-for-a-first-occurrence-of-a-term/149426 "2018-09-21T09:50:06Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![caub](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/caub/32/33783_2.png) [@caub](https://discuss.elastic.co/u/caub)\ **Post date:** [September 21, 2018, 9:50am UTC](https://discuss.elastic.co/t/how-to-search-for-a-first-occurrence-of-a-term/149426/1 "2018-09-21T09:50:06Z")
I'd like to search in the last 5 minutes, for values in the `err_msg` field that occured for the first time ever. And repeat this search every 5 minutes, so it should be as efficient as possible I wonder how to shape this in one query So far I've been doing: ```auto GET /filebeat*/_search?size=0 { "query": { "bool": { "filter": [ { "match": { "stream": "stderr" } }, { "range": { "@timestamp": { "gte": "now-5m" } } } ] } }, "aggs": { "errors": { "terms": { "field": "err_msg", "size": 10 } } } } ``` followed by multiple queries for each `err_msg` in the response, then keeping only the `err_msg` with no hits ```auto GET /filebeat*/_search?size=0 { "query": { "bool": { "filter": [ { "match": { "stream": "stderr" } }, { "match": { "err_msg": err_msg } }, { "range": { "@timestamp": { "lt": "now-1d" } } } ] } } } ``` It feels like it could be in one query, that's why I'm asking for a bit of help I don't think it has to be an aggregation, a search could work, but I don't know how, maybe as a scripted search? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [September 21, 2018, 11:44am UTC](https://discuss.elastic.co/t/how-to-search-for-a-first-occurrence-of-a-term/149426/2 "2018-09-21T11:44:37Z")
> [@caub](#): > > that occured for the first time ever. In a cluster with time based indices and lots of potential error types this will be hard. A “new” index will not have visibility of the content in old indices and vice versa ---
**Author:** ![caub](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/caub/32/33783_2.png) [@caub](https://discuss.elastic.co/u/caub)\ **Post date:** [September 21, 2018, 11:49am UTC](https://discuss.elastic.co/t/how-to-search-for-a-first-occurrence-of-a-term/149426/3 "2018-09-21T11:49:19Z")
err\_msg is a keyword, and it is only the first 160 chars of the original error message (`.slice(0, 160)`) . After having ran a stack for more than a month, I got less than 20 different err\_msg with that query: ```auto GET /filebeat*/_search?size=0 { "query": { "bool": { "filter": [ { "match": { "stream": "stderr" } }, { "range": { "@timestamp": { "gte": "now-300d" } } } ] } }, "aggs": { "errors": { "terms": { "field": "err_msg" } } } } ``` ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [September 21, 2018, 12:43pm UTC](https://discuss.elastic.co/t/how-to-search-for-a-first-occurrence-of-a-term/149426/4 "2018-09-21T12:43:05Z")
> [@caub](#): > > I got less than 20 different err\_msg In which case something like this might work. This is finding the first uses of tags on StackOverflow (note there are thousands of tags so I limit them in this example using the `include` param) ``` GET so/_search { "size": 0, "aggs": { "tag": { "terms": { "field": "tag", "include": [ "logstash", "java", "kibana" ], "order": { "firstSeen": "asc" } }, "aggs": { "firstSeen": { "min": { "field": "creationDate" } } } } } } ``` Your client would have to do the work to filter out the dates \> 5 minutes ago but the bulk of the heavy lifting is done in this request. ---
**Author:** ![caub](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/caub/32/33783_2.png) [@caub](https://discuss.elastic.co/u/caub)\ **Post date:** [September 21, 2018, 6:19pm UTC](https://discuss.elastic.co/t/how-to-search-for-a-first-occurrence-of-a-term/149426/5 "2018-09-21T18:19:41Z")
Thanks your suggestion works ```auto GET /filebeat*/_search?size=0 { "query": { "bool": { "filter": [ { "match": { "stream": "stderr" } } ] } }, "aggs": { "errors": { "terms": { "field": "err_msg", "order": { "firstSeen": "asc" } }, "aggs": { "firstSeen": { "min": { "field": "@timestamp" } } } } } } ``` I was still wondering if we could rather have a "2-level" query, like what I posted originally, but written in one query. Where the first level queries very recent errors in the last 5m, then the second level, will query for a possible second match for these error, before `now-5m`. Because this way seems more scalable, I think, since most of the time, there are no errors in the last 5m, and even the second level search can be efficient ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 19, 2018, 6:19pm UTC](https://discuss.elastic.co/t/how-to-search-for-a-first-occurrence-of-a-term/149426/6 "2018-10-19T18:19:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.