# Regular expession queries in elasticsearch **URL:** \ **Category:** Elasticsearch\ **Created:** [May 9, 2020, 11:31am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836 "2020-05-09T11:31:12Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Attila816](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/attila816/32/68007_2.png) [@Attila816](https://discuss.elastic.co/u/Attila816)\ **Post date:** [May 9, 2020, 11:31am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/1 "2020-05-09T11:31:12Z")
I know that the `.*` operators match any characters of any length, including no characters and elastic has a [RegexpQuery syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/regexp-syntax.html). So from that page I see that there are standard operators: . , ? , + , \*, {…}, |, (…), […]. Optional Operators: complement: a~bc, interval: foo\<1-100\>, intersection: aaa.+&.+bbb, anystring: @&~(abc.+) So my question is how can I solve `^` (beginning of line) or `$` (end of line) character searching? What can I use for '' character to treat next character literally. Example: in "\$100", the \ indicates that the pattern is "100", not end-of-line () followed by "100"? Other question is how are these managed to search in elastic: - A _negative assert_ of the form "(! _subexpression_ )". Matches any sequence of characters in the target sequence that does not match the pattern between the delimiters, and does not change the match position in the target sequence. - A _hexadecimal escape sequence_ of the form "\x `hh` ". Matches a character in the target sequence that is represented by the two hexadecimal digits `hh` . - A _unicode escape sequence_ of the form "\u `hhhh` ". Matches a character in the target sequence that is represented by the four hexadecimal digits `hhhh` . - A _control escape sequence_ of the form "\c `k` ". Matches the control character that is named by the character `k` . - A _word boundary assert_ of the form "\b". Matches when the current position in the target sequence is immediately after a _word boundary_ . - A _negative word boundary assert_ of the form "\B". Matches when the current position in the target sequence is not immediately after a _word boundary_ . - A _dsw character escape_ of the form "\d", "\D", "\s", "\S", "\w", "\W". Provides a short name for a character class. Thanks. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 11, 2020, 6:28am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/2 "2020-05-11T06:28:03Z")
What is the high level problem you are trying to solve? Regexp queries can be quite slow and often do not scale well, so I would recommend only going down that path as a last resort. ---
**Author:** ![Attila816](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/attila816/32/68007_2.png) [@Attila816](https://discuss.elastic.co/u/Attila816)\ **Post date:** [May 11, 2020, 7:23am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/3 "2020-05-11T07:23:02Z")
Hi @Christian_Dahlqvist, I need to query with a dtSearch query syntax on ElasticSearch. Therefore, I think I need to support dtSearch RegExp syntax so for example I need to manage the following charaters in a regexp query: `\b, \d, \s, \w, ^, $` So for example I need to manage these RegExp queries: `(\b\d+\b)` `(\b[a-z]+[0-9]+\b) ` Thanks, Attila ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [May 11, 2020, 8:40am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/4 "2020-05-11T08:40:42Z")
Probably worth checking a few assumptions here first - are you searching `text` or `keyword` field types here? With text fields the regular expressions are typically testing individual words in your content so notions of "line start", "word start" or whitespace are redundant - the content you are testing has already had been sliced up on the boundaries defined by your choice of analyzer. ---
**Author:** ![Attila816](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/attila816/32/68007_2.png) [@Attila816](https://discuss.elastic.co/u/Attila816)\ **Post date:** [May 11, 2020, 10:22am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/5 "2020-05-11T10:22:53Z")
@Mark_Harwood Regular expression queries should be managed in both text and keyword fields. In my understanding if I have a property of a document for example I have an email document: - EmailFrom, EmailTo and Subject properties are keyword porperties while - Body has a text field type I brought this example from [this](https://kb.objectrocket.com/elasticsearch/when-to-use-the-keyword-type-vs-text-datatype-in-elasticsearch) article. I don't know if I am correct. How would You determine which property should be text or keyword type? So in my case there would be queries from both keyword and text fields. How would You manage Regexp Queries in both of these types? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [May 11, 2020, 10:40am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/6 "2020-05-11T10:40:10Z")
> [@Attila816](#): > > How would You determine which property should be text or keyword type? A single JSON field [can be mapped as both](https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html). Generally speaking it would be most likely to consider from/to = keyword and subject/body = text but it depends on your search and aggregation needs. - If you want an aggregation to know `top participants` on a topic then I'd use a `keyword` field on an "email\_addresses" field which contained copies of both `from` and `to` fields (see "copy to" feature in mappings) - If you want to search for email addresses by domain name only I'd either - index email addresses as `text` as well as keyword to tokenize OR - parse out domain names into a separate fields in your client or using an "ingest pipeline" - If you want to find the most active email threads I'd consider having the subject field mapped as `keyword` but minus any "Re:" prefix added by replies. I generally would only be looking to run any regex queries as a last resort. They're hard for users to write and read. In elasticsearch matching words near each other in the body of text is handled using phrase queries. Text analyzers typically perform cleansing like case normalisation, stemming and punctuation removal to make searches simpler. Structured queries like matching domain names can be made simpler if the regexes to extract domain names are applied at index time rather than query time. It does depend on your specific content and queries though. ---
**Author:** ![Attila816](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/attila816/32/68007_2.png) [@Attila816](https://discuss.elastic.co/u/Attila816)\ **Post date:** [May 11, 2020, 11:25am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/7 "2020-05-11T11:25:46Z")
Thank @Mark_Harwood, I need to do further investigation as You mentioned with using analyzer, phrase queries and text fields almost eveywhere. ---
**Author:** ![Attila816](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/attila816/32/68007_2.png) [@Attila816](https://discuss.elastic.co/u/Attila816)\ **Post date:** [May 12, 2020, 6:48am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/8 "2020-05-12T06:48:20Z")
@Mark_Harwood or others could You please help me with these Regexp characters, thanks: - `\d` : Whole Number 0 - 9, - `\w` : Alphanumeric Character, - `\W` : Symbols ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [May 12, 2020, 7:45am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/9 "2020-05-12T07:45:16Z")
That’s [being worked on](https://github.com/apache/lucene-solr/pull/1489) but won’t be available for some time. For now you could write code in your client that expanded those shorthand expressions in search strings to their full form. ---
**Author:** ![Attila816](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/attila816/32/68007_2.png) [@Attila816](https://discuss.elastic.co/u/Attila816)\ **Post date:** [May 12, 2020, 8:35am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/10 "2020-05-12T08:35:13Z")
Thanks @Mark_Harwood, this is again a really useful answer, because from this we could see which are the equivalent for the regexp meta characters [here](https://github.com/apache/lucene-solr/pull/1489/files). ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [May 12, 2020, 8:37am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/11 "2020-05-12T08:37:51Z")
You may actually want to beef up the expansion list - as my colleague Alan said: ``` "Defining a 'word character' as [a-zA-Z_0-9] doesn't feel very 2020..." ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 9, 2020, 8:37am UTC](https://discuss.elastic.co/t/regular-expession-queries-in-elasticsearch/231836/12 "2020-06-09T08:37:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.