Can't find unit tests for reserved characters

Elastic Stack Elasticsearch
1

In the ES documentation there is a list of reserved characters.

I'm looking for the unit tests that test reserved characters inside a
query_string query.

Could someone kindly point me in the right direction?

Thanks!

If you need to use any of the characters which function as operators in
your query itself (and not as operators), then you should escape them with
a leading backslash. For instance, to search for (1+1)=2, you would need to
write your query as (1+1)=2.

The reserved characters are: + - && || ! ( ) { } [ ] ^ " ~ * ? : \ /

Failing to escape these special characters correctly could lead to a syntax
error which prevents your query from running.

Watch this space

A space may also be a reserved character. For instance, if you have a
synonym list which converts "wi fi" to "wifi", a query_string search for "wi
fi" would fail. The query string parser would interpret your query as a
search for "wi OR fi", while the token stored in your index is actually
"wifi". Escaping the space will protect it from being touched by the query
string parser: "wi\ fi".

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e4fa9e63-e3d4-41cd-ba9a-bff96361758d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

More info...trying to figure out why this query throws exception.

org.elasticsearch.search.SearchParseException: [example][1]:
from[-1],size[-1]: Parse Failure [Failed to parse source [
{
"query": {
"query_string": {
"query": "name:exampleof\ bug"
}
}
}
]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:515)
at
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:256)
at
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:206)
at
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:203)
at
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.index.query.QueryParsingException: [example]
Failed to parse
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:270)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:263)
at
org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)
... 9 more
Caused by: org.elasticsearch.common.jackson.core.JsonParseException:
Unrecognized character escape ' ' (code 32)
at [Source: [B@685c9be8; line: 5, column: 33]
at
org.elasticsearch.common.jackson.core.JsonParser._constructError(JsonParser.java:1419)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:485)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._decodeEscaped(UTF8StreamJsonParser.java:2924)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2209)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:2165)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:279)
at
org.elasticsearch.common.xcontent.json.JsonXContentParser.text(JsonXContentParser.java:85)
at
org.elasticsearch.index.query.QueryStringQueryParser.parse(QueryStringQueryParser.java:131)
at
org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:234)
at
org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:342)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:268)
... 12 more

On Thursday, August 21, 2014 10:56:24 AM UTC-7, ben wrote:

In the ES documentation there is a list of reserved characters.

I'm looking for the unit tests that test reserved characters inside a
query_string query.

Could someone kindly point me in the right direction?

Thanks!

If you need to use any of the characters which function as operators in
your query itself (and not as operators), then you should escape them with
a leading backslash. For instance, to search for (1+1)=2, you would need
to write your query as (1+1)=2.

The reserved characters are: + - && || ! ( ) { } ^ " ~ * ? : \ /

Failing to escape these special characters correctly could lead to a
syntax error which prevents your query from running.

Watch this space

A space may also be a reserved character. For instance, if you have a
synonym list which converts "wi fi" to "wifi", a query_string search for "wi
fi" would fail. The query string parser would interpret your query as a
search for "wi OR fi", while the token stored in your index is actually
"wifi". Escaping the space will protect it from being touched by the
query string parser: "wi\ fi".

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e4d302c4-5f9c-40a5-a54c-48d9b671220a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

In the unit tests for the jackson json library not all characters noted in
the ES documentation are tested. Shouldn't ES test the character escaping?

On Thursday, August 21, 2014 10:56:24 AM UTC-7, ben wrote:

In the ES documentation there is a list of reserved characters.

I'm looking for the unit tests that test reserved characters inside a
query_string query.

Could someone kindly point me in the right direction?

Thanks!

If you need to use any of the characters which function as operators in
your query itself (and not as operators), then you should escape them with
a leading backslash. For instance, to search for (1+1)=2, you would need
to write your query as (1+1)=2.

The reserved characters are: + - && || ! ( ) { } ^ " ~ * ? : \ /

Failing to escape these special characters correctly could lead to a
syntax error which prevents your query from running.

Watch this space

A space may also be a reserved character. For instance, if you have a
synonym list which converts "wi fi" to "wifi", a query_string search for "wi
fi" would fail. The query string parser would interpret your query as a
search for "wi OR fi", while the token stored in your index is actually
"wifi". Escaping the space will protect it from being touched by the
query string parser: "wi\ fi".

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5c0d5fe8-d3de-4be8-bb90-f65cc223084b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

That's the JSON parsing, not the query_string parsing.

You need to use a double slash in JSON in order to pass a single slash, ie:

{
"query": {
"query_string": {
"query": "name:exampleof\ bug"
}
}
}

Also, re the reserved characters in the query string - that is all handled
by Lucene, not Elasticsearch

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQ%3DFLOZwab577DUgGc4wZA9VinxsUoA%2BTwZMqNubVBiqg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

I was trying to demonstrate the escaping of a space not a slash.

According to the ES documentation (copied in my original post) that says a
space must be escaped.

Thanks!

On Thursday, August 21, 2014 12:31:29 PM UTC-7, Clinton Gormley wrote:

That's the JSON parsing, not the query_string parsing.

You need to use a double slash in JSON in order to pass a single slash, ie:

{
"query": {
"query_string": {
"query": "name:exampleof\ bug"
}
}
}

Also, re the reserved characters in the query string - that is all handled
by Lucene, not Elasticsearch

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d33ad037-cf5e-40dd-8db0-4dab613c8672%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

You have two levels of escaping going on: query string escaping, and JSON
escaping.

So you want the query string to be escaped like this: "foo\ bar"

but you're passing that string in via JSON, so to get a literal \ followed
by a space, you need to escape the \ as \, resulting in: "foo\ bar"

On 21 August 2014 23:38, ben billumina2@gmail.com wrote:

I was trying to demonstrate the escaping of a space not a slash.

According to the ES documentation (copied in my original post) that says a
space must be escaped.

Thanks!

On Thursday, August 21, 2014 12:31:29 PM UTC-7, Clinton Gormley wrote:

That's the JSON parsing, not the query_string parsing.

You need to use a double slash in JSON in order to pass a single slash,
ie:

{
"query": {
"query_string": {
"query": "name:exampleof\ bug"
}
}
}

Also, re the reserved characters in the query string - that is all
handled by Lucene, not Elasticsearch

clint

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d33ad037-cf5e-40dd-8db0-4dab613c8672%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d33ad037-cf5e-40dd-8db0-4dab613c8672%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKS8fEFWSH319joETa6DCJ9u0%3D%2BQpdpE4NhhgLry2%3D-Ekg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.