# Best practice to search for a regular expression next to a list of terms **URL:** \ **Category:** Elasticsearch\ **Created:** [May 17, 2018, 6:53pm UTC](https://discuss.elastic.co/t/best-practice-to-search-for-a-regular-expression-next-to-a-list-of-terms/132383 "2018-05-17T18:53:43Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![kevitra](https://avatars.discourse-cdn.com/v4/letter/k/b9e5f3/32.png) [@kevitra](https://discuss.elastic.co/u/kevitra)\ **Post date:** [May 17, 2018, 6:53pm UTC](https://discuss.elastic.co/t/best-practice-to-search-for-a-regular-expression-next-to-a-list-of-terms/132383/1 "2018-05-17T18:53:43Z")
I am trying to write a query with the following criteria: 1. The query has a regular expression and the document must contain a hit 2. The query has a list of words and at least one of the words must appear within N words of the regular expression. I want to use slop between the regular expression and the list of terms. Here is what I have come up with so far. It finds hits on the regexp and the list of terms but not within N words of each other. ``` { "from": 0, "size": 100, "explain": true, "_source": { "includes": [ "*" ], "excludes": [ "FileText" ] }, "query": { "bool": { "must": { "regexp": { "FileText": { "value": "[0-9]{3}" } } }, "should": { "match": { "FileText": { "query": "list words find please", "minimum_should_match": "1%" } } } } } } ``` I also tried ``` { "from": 0, "size": 100, "explain": false, "_source": { "includes": [ "*" ], "excludes": [ "FileText" ] }, "query": { "bool": { "must": [ { "regexp": { "FileText": { "value": "[0-9]{3}" } } }, { "match": { "FileText": { "query": "list words find please" } } } ] } } } ``` This finds documents with the regex and terms but they are not in proximity to each other. Thanks in advance for any guidance you can provide. ---
**Author:** ![rjernst](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rjernst/32/6363_2.png) [@rjernst](https://discuss.elastic.co/u/rjernst)\ **Post date:** [May 17, 2018, 7:58pm UTC](https://discuss.elastic.co/t/best-practice-to-search-for-a-regular-expression-next-to-a-list-of-terms/132383/2 "2018-05-17T19:58:47Z")
Have you looked at [span queries](https://www.elastic.co/guide/en/elasticsearch/reference/current/span-queries.html)? Note that span queries themselves are already io heavy (since they must gather and compute position data at multiple levels within the query execution). In combination with regex queries, this is likely to be very slow. ---
**Author:** ![loren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/loren/32/44942_2.png) [@loren](https://discuss.elastic.co/u/loren)\ **Post date:** [May 17, 2018, 8:36pm UTC](https://discuss.elastic.co/t/best-practice-to-search-for-a-regular-expression-next-to-a-list-of-terms/132383/3 "2018-05-17T20:36:28Z")
I don't think you can use `regexp` with span queries, even though the [documentation says you can](https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl-span-multi-term-query.html). Someone else mentioned this [over here on the Lucene list](http://search-lucene.com/m/ElasticSearch/TnrDOBYBdS1NFF471?subj=+Elasticsearch+Regex+phrase+search). ---
**Author:** ![kevitra](https://avatars.discourse-cdn.com/v4/letter/k/b9e5f3/32.png) [@kevitra](https://discuss.elastic.co/u/kevitra)\ **Post date:** [May 17, 2018, 8:44pm UTC](https://discuss.elastic.co/t/best-practice-to-search-for-a-regular-expression-next-to-a-list-of-terms/132383/4 "2018-05-17T20:44:10Z")
I have been trying to use span and you are correct, regexp is not supported: ``` { "error": { "root_cause": [ { "type": "parsing_exception", "reason": "[span_multi] query does not support [regexp]", "line": 1, "col": 59 } ], "type": "parsing_exception", "reason": "[span_multi] query does not support [regexp]", "line": 1, "col": 59 }, "status": 400 } ``` That was looking really promising too. Without proximity searching we get thousands of false positives. ---
**Author:** ![loren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/loren/32/44942_2.png) [@loren](https://discuss.elastic.co/u/loren)\ **Post date:** [May 17, 2018, 8:47pm UTC](https://discuss.elastic.co/t/best-practice-to-search-for-a-regular-expression-next-to-a-list-of-terms/132383/5 "2018-05-17T20:47:11Z")
Actually, the docs are right but it's confusing. Here's a working example for you: ```auto DELETE my_index PUT my_index/doc/2 { "content": "I got 99 problems but this query ain't one" } GET my_index/doc/_search { "query": { "span_near": { "clauses": [ { "span_multi": { "match": { "regexp": { "content": "[0-9]{2}" } } } }, { "span_term": { "content": "query" } } ], "slop": 3, "in_order": true } } } ``` Change slop to 2 and there's no match. ---
**Author:** ![kevitra](https://avatars.discourse-cdn.com/v4/letter/k/b9e5f3/32.png) [@kevitra](https://discuss.elastic.co/u/kevitra)\ **Post date:** [May 17, 2018, 9:16pm UTC](https://discuss.elastic.co/t/best-practice-to-search-for-a-regular-expression-next-to-a-list-of-terms/132383/6 "2018-05-17T21:16:51Z")
That works great and with our actual regexp and a search term at a customer installation the performance wasn't horrible. These queries will be run by a backend service nightly so I don't need sub second performance. Thank you both for the assistance. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 14, 2018, 9:17pm UTC](https://discuss.elastic.co/t/best-practice-to-search-for-a-regular-expression-next-to-a-list-of-terms/132383/7 "2018-06-14T21:17:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.