# Can't find unit tests for reserved characters

**URL:** <https://discuss.elastic.co/t/cant-find-unit-tests-for-reserved-characters/19398>\
**Category:** Elasticsearch\
**Created:** [August 21, 2014, 5:56pm UTC](https://discuss.elastic.co/t/cant-find-unit-tests-for-reserved-characters/19398 "2014-08-21T17:56:24Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ben1](https://avatars.discourse-cdn.com/v4/letter/b/9de053/32.png) [@ben1](https://discuss.elastic.co/u/ben1)\
**Post date:** [August 21, 2014, 5:56pm UTC](https://discuss.elastic.co/t/cant-find-unit-tests-for-reserved-characters/19398/1 "2014-08-21T17:56:24Z")

</div>

In the ES documentation there is a list of reserved characters.

I'm looking for the unit tests that test reserved characters inside a  
query\_string query.

Could someone kindly point me in the right direction?

Thanks!

If you need to use any of the characters which function as operators in  
your query itself (and not as operators), then you should escape them with  
a leading backslash. For instance, to search for (1+1)=2, you would need to  
write your query as (1+1)=2.

The reserved characters are: + - && || ! ( ) { } [] ^ " ~ \* ? : \ /

Failing to escape these special characters correctly could lead to a syntax  
error which prevents your query from running.

_Watch this space_

A space may also be a reserved character. For instance, if you have a  
synonym list which converts "wi fi" to "wifi", a query\_string search for "wi  
fi" would fail. The query string parser would interpret your query as a  
search for "wi OR fi", while the token stored in your index is actually  
"wifi". Escaping the space will protect it from being touched by the query  
string parser: "wi\ fi".

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e4fa9e63-e3d4-41cd-ba9a-bff96361758d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e4fa9e63-e3d4-41cd-ba9a-bff96361758d%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![ben1](https://avatars.discourse-cdn.com/v4/letter/b/9de053/32.png) [@ben1](https://discuss.elastic.co/u/ben1)\
**Post date:** [August 21, 2014, 6:11pm UTC](https://discuss.elastic.co/t/cant-find-unit-tests-for-reserved-characters/19398/2 "2014-08-21T18:11:58Z")

</div>

More info...trying to figure out why this query throws exception.

org.elasticsearch.search.SearchParseException: [example][1]:  
from[-1],size[-1]: Parse Failure [Failed to parse source [  
{  
"query": {  
"query\_string": {  
"query": "name:exampleof\ bug"  
}  
}  
}  
]]  
at  
org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)  
at  
org.elasticsearch.search.SearchService.createContext(SearchService.java:515)  
at  
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)  
at  
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:256)  
at  
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:206)  
at  
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:203)  
at  
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.elasticsearch.index.query.QueryParsingException: [example]  
Failed to parse  
at  
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:270)  
at  
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:263)  
at  
org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)  
at  
org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)  
... 9 more  
Caused by: org.elasticsearch.common.jackson.core.JsonParseException:  
Unrecognized character escape ' ' (code 32)  
at [Source: [B@685c9be8; line: 5, column: 33]  
at  
org.elasticsearch.common.jackson.core.JsonParser.\_constructError(JsonParser.java:1419)  
at  
org.elasticsearch.common.jackson.core.base.ParserMinimalBase.\_reportError(ParserMinimalBase.java:508)  
at  
org.elasticsearch.common.jackson.core.base.ParserMinimalBase.\_handleUnrecognizedCharacterEscape(ParserMinimalBase.java:485)  
at  
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.\_decodeEscaped(UTF8StreamJsonParser.java:2924)  
at  
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.\_finishString2(UTF8StreamJsonParser.java:2209)  
at  
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.\_finishString(UTF8StreamJsonParser.java:2165)  
at  
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:279)  
at  
org.elasticsearch.common.xcontent.json.JsonXContentParser.text(JsonXContentParser.java:85)  
at  
org.elasticsearch.index.query.QueryStringQueryParser.parse(QueryStringQueryParser.java:131)  
at  
org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:234)  
at  
org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:342)  
at  
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:268)  
... 12 more

On Thursday, August 21, 2014 10:56:24 AM UTC-7, ben wrote:

> In the ES documentation there is a list of reserved characters.
> 
> I'm looking for the unit tests that test reserved characters inside a  
> query\_string query.
> 
> Could someone kindly point me in the right direction?
> 
> Thanks!
> 
> If you need to use any of the characters which function as operators in  
> your query itself (and not as operators), then you should escape them with  
> a leading backslash. For instance, to search for (1+1)=2, you would need  
> to write your query as (1+1)=2.
> 
> The reserved characters are: + - && || ! ( ) { } ^ " ~ \* ? : \ /
> 
> Failing to escape these special characters correctly could lead to a  
> syntax error which prevents your query from running.
> 
> _Watch this space_
> 
> A space may also be a reserved character. For instance, if you have a  
> synonym list which converts "wi fi" to "wifi", a query\_string search for "wi  
> fi" would fail. The query string parser would interpret your query as a  
> search for "wi OR fi", while the token stored in your index is actually  
> "wifi". Escaping the space will protect it from being touched by the  
> query string parser: "wi\ fi".

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e4d302c4-5f9c-40a5-a54c-48d9b671220a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e4d302c4-5f9c-40a5-a54c-48d9b671220a%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![ben1](https://avatars.discourse-cdn.com/v4/letter/b/9de053/32.png) [@ben1](https://discuss.elastic.co/u/ben1)\
**Post date:** [August 21, 2014, 6:54pm UTC](https://discuss.elastic.co/t/cant-find-unit-tests-for-reserved-characters/19398/3 "2014-08-21T18:54:28Z")

</div>

In the unit tests for the jackson json library not all characters noted in  
the ES documentation are tested. Shouldn't ES test the character escaping?

> <https://github.com/FasterXML/jackson-core/blob/c587db79d6853fe21d56e070a343d018f681575e/src/test/java/com/fasterxml/jackson/core/json/TestParserNonStandard.java>

On Thursday, August 21, 2014 10:56:24 AM UTC-7, ben wrote:

> In the ES documentation there is a list of reserved characters.
> 
> I'm looking for the unit tests that test reserved characters inside a  
> query\_string query.
> 
> Could someone kindly point me in the right direction?
> 
> Thanks!
> 
> If you need to use any of the characters which function as operators in  
> your query itself (and not as operators), then you should escape them with  
> a leading backslash. For instance, to search for (1+1)=2, you would need  
> to write your query as (1+1)=2.
> 
> The reserved characters are: + - && || ! ( ) { } ^ " ~ \* ? : \ /
> 
> Failing to escape these special characters correctly could lead to a  
> syntax error which prevents your query from running.
> 
> _Watch this space_
> 
> A space may also be a reserved character. For instance, if you have a  
> synonym list which converts "wi fi" to "wifi", a query\_string search for "wi  
> fi" would fail. The query string parser would interpret your query as a  
> search for "wi OR fi", while the token stored in your index is actually  
> "wifi". Escaping the space will protect it from being touched by the  
> query string parser: "wi\ fi".

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5c0d5fe8-d3de-4be8-bb90-f65cc223084b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5c0d5fe8-d3de-4be8-bb90-f65cc223084b%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [August 21, 2014, 7:31pm UTC](https://discuss.elastic.co/t/cant-find-unit-tests-for-reserved-characters/19398/4 "2014-08-21T19:31:16Z")

</div>

That's the JSON parsing, not the query\_string parsing.

You need to use a double slash in JSON in order to pass a single slash, ie:

{  
"query": {  
"query\_string": {  
"query": "name:exampleof\ bug"  
}  
}  
}

Also, re the reserved characters in the query string - that is all handled  
by Lucene, not Elasticsearch

clint

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQ%3DFLOZwab577DUgGc4wZA9VinxsUoA%2BTwZMqNubVBiqg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQ%3DFLOZwab577DUgGc4wZA9VinxsUoA%2BTwZMqNubVBiqg%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![ben1](https://avatars.discourse-cdn.com/v4/letter/b/9de053/32.png) [@ben1](https://discuss.elastic.co/u/ben1)\
**Post date:** [August 21, 2014, 9:38pm UTC](https://discuss.elastic.co/t/cant-find-unit-tests-for-reserved-characters/19398/5 "2014-08-21T21:38:40Z")

</div>

I was trying to demonstrate the escaping of a space not a slash.

According to the ES documentation (copied in my original post) that says a  
space must be escaped.

Thanks!

On Thursday, August 21, 2014 12:31:29 PM UTC-7, Clinton Gormley wrote:

> That's the JSON parsing, not the query\_string parsing.
> 
> You need to use a double slash in JSON in order to pass a single slash, ie:
> 
> {  
> "query": {  
> "query\_string": {  
> "query": "name:exampleof\ bug"  
> }  
> }  
> }
> 
> Also, re the reserved characters in the query string - that is all handled  
> by Lucene, not Elasticsearch
> 
> clint

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d33ad037-cf5e-40dd-8db0-4dab613c8672%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d33ad037-cf5e-40dd-8db0-4dab613c8672%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [August 23, 2014, 12:05pm UTC](https://discuss.elastic.co/t/cant-find-unit-tests-for-reserved-characters/19398/6 "2014-08-23T12:05:23Z")

</div>

You have two levels of escaping going on: query string escaping, and JSON  
escaping.

So you want the query string to be escaped like this: "foo\ bar"

but you're passing that string in via JSON, so to get a literal \ followed  
by a space, you need to escape the \ as \, resulting in: "foo\ bar"

On 21 August 2014 23:38, ben [billumina2@gmail.com](mailto:billumina2@gmail.com) wrote:

> I was trying to demonstrate the escaping of a space not a slash.
> 
> According to the ES documentation (copied in my original post) that says a  
> space must be escaped.
> 
> Thanks!
> 
> On Thursday, August 21, 2014 12:31:29 PM UTC-7, Clinton Gormley wrote:
> 
> > That's the JSON parsing, not the query\_string parsing.
> > 
> > You need to use a double slash in JSON in order to pass a single slash,  
> > ie:
> > 
> > {  
> > "query": {  
> > "query\_string": {  
> > "query": "name:exampleof\ bug"  
> > }  
> > }  
> > }
> > 
> > Also, re the reserved characters in the query string - that is all  
> > handled by Lucene, not Elasticsearch
> > 
> > clint
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/d33ad037-cf5e-40dd-8db0-4dab613c8672%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d33ad037-cf5e-40dd-8db0-4dab613c8672%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/d33ad037-cf5e-40dd-8db0-4dab613c8672%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d33ad037-cf5e-40dd-8db0-4dab613c8672%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPt3XKS8fEFWSH319joETa6DCJ9u0%3D%2BQpdpE4NhhgLry2%3D-Ekg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPt3XKS8fEFWSH319joETa6DCJ9u0%3D%2BQpdpE4NhhgLry2%3D-Ekg%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:06am UTC](https://discuss.elastic.co/t/cant-find-unit-tests-for-reserved-characters/19398/7 "2017-07-06T01:06:50Z")

</div>