URL query parameters limitation

Elastic Stack Elasticsearch
1

I'm ramping up on Query DSL and have heard that URL parameter queries have
a limitation compared to JSON queries in the Request body.
Could someone please elaborate (or point to a doc) on those limitations?

Thanks,
Nick

--

2

In HTTP, it is up to the implementation to limit the request URI length, see

http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.2.1

ES uses Netty for HTTP implementation, and current Netty HTTP request
limits are documented here

http://static.netty.io/3.5/api/org/jboss/netty/handler/codec/http/HttpRequestDecoder.html

Best regards,

Jörg

On Thursday, September 20, 2012 11:37:22 PM UTC+2, TheOutlander wrote:

I'm ramping up on Query DSL and have heard that URL parameter queries have
a limitation compared to JSON queries in the Request body.
Could someone please elaborate (or point to a doc) on those limitations?

Thanks,
Nick

--

3

Thanks for the quick response! I didn't even think about the URI length as
the limitation!!
So does that mean there are no syntactical limitations?

Thanks,
Nick

On Thursday, September 20, 2012 3:32:19 PM UTC-7, Jörg Prante wrote:

In HTTP, it is up to the implementation to limit the request URI length,
see

HTTP/1.1: Protocol Parameters

ES uses Netty for HTTP implementation, and current Netty HTTP request
limits are documented here

http://static.netty.io/3.5/api/org/jboss/netty/handler/codec/http/HttpRequestDecoder.html

Best regards,

Jörg

On Thursday, September 20, 2012 11:37:22 PM UTC+2, TheOutlander wrote:

I'm ramping up on Query DSL and have heard that URL parameter queries
have a limitation compared to JSON queries in the Request body.
Could someone please elaborate (or point to a doc) on those limitations?

Thanks,
Nick

--

4

On Thu, 2012-09-20 at 15:53 -0700, TheOutlander wrote:

Thanks for the quick response! I didn't even think about the URI
length as the limitation!!
So does that mean there are no syntactical limitations?

That kinda depends :slight_smile:

You can pass the a full query using the query DSL as a parameter in the
query string (as opposed to the POST body), in which case the only
limitation is the URI length, but I find that to be quite unreadable.

However, most people, when referring to a query in the query string, are
referring to: ?q=my+query+phrase

And that is more limiting than using the query DSL. The 'q=' maps to
the query string query:

But using the query DSL directly gives you access to a much richer query
language:

clint

--

5

Sorry, I misunderstood. I considered "limitation" being just a question
about a length limit. Not sure what you mean with "syntactical limitation"
though.

Maybe you assume only a subset of possible queries can be passed via an URL
parameter in HTTP GET? The answer is no, see the 'source' parameter.

Or you assume the query syntax is limited to JSON? I never saw ES queries
in YAML or SMILE, but it should be possible to use another syntax beside
JSON, the code is there in org.elasticearch.common.xcontent

ES has a powerful domain-specific language (DSL) for queries, which is kind
of agnostic to the protocol the search request is transported with. You can
pass a search request

  • with REST by the following methods:

-- HTTP POST in the request body
-- HTTP GET in the request body (some think a request body in GET requests
is not RESTful, but it is not forbidden)
-- HTTP GET in the 'source' URL parameter

  • with the ES Transport Protocol (Java API)

  • with other transport protocols (e.g. provided by plugins)

So, by using REST, you can put a query into a HTTP GET URL parameter
'source' as you would put the query into a HTTP POST body. That should
work.

See also the java code in
org.elasticsearch.rest.action.search.RestSearchAction

Clinton pointed out that URL parameters need to be URI-escaped which makes
them not well readable, and the GET parameter 'q' maps to a 'query_string'
query. But that is just for convenience if you do not want to enter JSON
syntax in favor of pure Lucene syntax. That is a feature, not a limitation.

Best regards,

Jörg

On Friday, September 21, 2012 12:53:06 AM UTC+2, TheOutlander wrote:

Thanks for the quick response! I didn't even think about the URI length as
the limitation!!
So does that mean there are no syntactical limitations?

Thanks,
Nick

On Thursday, September 20, 2012 3:32:19 PM UTC-7, Jörg Prante wrote:

In HTTP, it is up to the implementation to limit the request URI length,
see

HTTP/1.1: Protocol Parameters

ES uses Netty for HTTP implementation, and current Netty HTTP request
limits are documented here

http://static.netty.io/3.5/api/org/jboss/netty/handler/codec/http/HttpRequestDecoder.html

Best regards,

Jörg

On Thursday, September 20, 2012 11:37:22 PM UTC+2, TheOutlander wrote:

I'm ramping up on Query DSL and have heard that URL parameter queries
have a limitation compared to JSON queries in the Request body.
Could someone please elaborate (or point to a doc) on those limitations?

Thanks,
Nick

--

6

That makes sense. I agree that URL queries can get unreadable and usually
we're thinking of simpler queries when referring to query strings.

-Nick

On Friday, September 21, 2012 2:08:37 AM UTC-7, Clinton Gormley wrote:

On Thu, 2012-09-20 at 15:53 -0700, TheOutlander wrote:

Thanks for the quick response! I didn't even think about the URI
length as the limitation!!
So does that mean there are no syntactical limitations?

That kinda depends :slight_smile:

You can pass the a full query using the query DSL as a parameter in the
query string (as opposed to the POST body), in which case the only
limitation is the URI length, but I find that to be quite unreadable.

However, most people, when referring to a query in the query string, are
referring to: ?q=my+query+phrase

And that is more limiting than using the query DSL. The 'q=' maps to
the query string query:

Elasticsearch Platform — Find real-time answers at scale | Elastic

But using the query DSL directly gives you access to a much richer query
language:
Elasticsearch Platform — Find real-time answers at scale | Elastic

clint

--

7

Thanks, that clears it up!
I asked about a "limitation" specifically because I think I saw it in the
documentation somewhere, but it never explained what it was.

Thanks,
Nick

On Friday, September 21, 2012 8:27:52 AM UTC-7, Jörg Prante wrote:

Sorry, I misunderstood. I considered "limitation" being just a question
about a length limit. Not sure what you mean with "syntactical limitation"
though.

Maybe you assume only a subset of possible queries can be passed via an
URL parameter in HTTP GET? The answer is no, see the 'source' parameter.

Or you assume the query syntax is limited to JSON? I never saw ES queries
in YAML or SMILE, but it should be possible to use another syntax beside
JSON, the code is there in org.elasticearch.common.xcontent

ES has a powerful domain-specific language (DSL) for queries, which is
kind of agnostic to the protocol the search request is transported
with. You can pass a search request

  • with REST by the following methods:

-- HTTP POST in the request body
-- HTTP GET in the request body (some think a request body in GET requests
is not RESTful, but it is not forbidden)
-- HTTP GET in the 'source' URL parameter

  • with the ES Transport Protocol (Java API)

  • with other transport protocols (e.g. provided by plugins)

So, by using REST, you can put a query into a HTTP GET URL parameter
'source' as you would put the query into a HTTP POST body. That should
work.

See also the java code in
org.elasticsearch.rest.action.search.RestSearchAction

Clinton pointed out that URL parameters need to be URI-escaped which makes
them not well readable, and the GET parameter 'q' maps to a 'query_string'
query. But that is just for convenience if you do not want to enter JSON
syntax in favor of pure Lucene syntax. That is a feature, not a limitation.

Best regards,

Jörg

On Friday, September 21, 2012 12:53:06 AM UTC+2, TheOutlander wrote:

Thanks for the quick response! I didn't even think about the URI length
as the limitation!!
So does that mean there are no syntactical limitations?

Thanks,
Nick

On Thursday, September 20, 2012 3:32:19 PM UTC-7, Jörg Prante wrote:

In HTTP, it is up to the implementation to limit the request URI length,
see

HTTP/1.1: Protocol Parameters

ES uses Netty for HTTP implementation, and current Netty HTTP request
limits are documented here

http://static.netty.io/3.5/api/org/jboss/netty/handler/codec/http/HttpRequestDecoder.html

Best regards,

Jörg

On Thursday, September 20, 2012 11:37:22 PM UTC+2, TheOutlander wrote:

I'm ramping up on Query DSL and have heard that URL parameter queries
have a limitation compared to JSON queries in the Request body.
Could someone please elaborate (or point to a doc) on those limitations?

Thanks,
Nick

--