Any chance of facets in URI?


(ian mayo) #1

Hi all,
I'm aspiring to a nice & tidy REST API for my app. ES forms part of my
app, but unfortunately the complexity of search is dirtying my lovely API.
Across the rest of the system I've managed to included all data in the URI

  • which both has a functional beauty, but it also offers opportunities for
    cache-ing, either at the server or the client.

In my ES searches, I'm able to encode the search in the URI, using "
_search?q=user:kimchy". But, to get the facets that relate to that search
I'll have to switch to storing the data in a JSON payload.

I imagine it's possible to use a similar encoding to list which facets
should be present in the results, particularly if the field-name is both
used to specify which field to select, plus what to call the results object
in the return payload.

I'd like to find out if there's any chance that facets could be included in
the URI construct. Is it technically possible?

cheers,
Ian


(Benjamin Devèze) #2

Hey Ian,

I have begun working on something that would do exactly that and more.

This is the concept of search templates. Basically you can define a
template using the full query DSL with variables inside of it. When
you search you precise which template you want to use and the
variables in the templates are replaced using variables from the uri.

This approach has a lot of advantages:

  • less network transfer for each query which can be quite important if
    you have a lot of complex queries
  • it is easier to cache results in cache layers like varnish and you
    can even change your templates without invalidating the cache
  • you generally use only a few classes of queries in a common
    application so it is very clear if you define a search template for
    each class of query what queries you are actually using
  • you can abstract query complexity from search clients and expose
    precisely the kind of queries and parameters you want. Even if the
    templates are quite complex the exposed search uri are very simple if
    you want to.
  • you could decide to restrict queries to only search templates which
    would prevent client to do very costly queries

...

On Thu, Apr 26, 2012 at 6:37 PM, ian mayo ian@planetmayo.com wrote:

Hi all,
I'm aspiring to a nice & tidy REST API for my app. ES forms part of my app,
but unfortunately the complexity of search is dirtying my lovely API.
Across the rest of the system I've managed to included all data in the URI

  • which both has a functional beauty, but it also offers opportunities for
    cache-ing, either at the server or the client.

In my ES searches, I'm able to encode the search in the URI, using
"_search?q=user:kimchy". But, to get the facets that relate to that search
I'll have to switch to storing the data in a JSON payload.

I imagine it's possible to use a similar encoding to list which facets
should be present in the results, particularly if the field-name is both
used to specify which field to select, plus what to call the results object
in the return payload.

I'd like to find out if there's any chance that facets could be included in
the URI construct. Is it technically possible?

cheers,
Ian

--
Benjamin DEVEZE


(Benjamin Devèze) #3

I have forgotten to add that ES provides index templates, mapping
templates so for the sake of consistency we need search templates :slight_smile:

On Thu, Apr 26, 2012 at 8:10 PM, Benjamin Devèze
benjamin.deveze@gmail.com wrote:

Hey Ian,

I have begun working on something that would do exactly that and more.

This is the concept of search templates. Basically you can define a
template using the full query DSL with variables inside of it. When
you search you precise which template you want to use and the
variables in the templates are replaced using variables from the uri.

This approach has a lot of advantages:

  • less network transfer for each query which can be quite important if
    you have a lot of complex queries
  • it is easier to cache results in cache layers like varnish and you
    can even change your templates without invalidating the cache
  • you generally use only a few classes of queries in a common
    application so it is very clear if you define a search template for
    each class of query what queries you are actually using
  • you can abstract query complexity from search clients and expose
    precisely the kind of queries and parameters you want. Even if the
    templates are quite complex the exposed search uri are very simple if
    you want to.
  • you could decide to restrict queries to only search templates which
    would prevent client to do very costly queries

...

On Thu, Apr 26, 2012 at 6:37 PM, ian mayo ian@planetmayo.com wrote:

Hi all,
I'm aspiring to a nice & tidy REST API for my app. ES forms part of my app,
but unfortunately the complexity of search is dirtying my lovely API.
Across the rest of the system I've managed to included all data in the URI

  • which both has a functional beauty, but it also offers opportunities for
    cache-ing, either at the server or the client.

In my ES searches, I'm able to encode the search in the URI, using
"_search?q=user:kimchy". But, to get the facets that relate to that search
I'll have to switch to storing the data in a JSON payload.

I imagine it's possible to use a similar encoding to list which facets
should be present in the results, particularly if the field-name is both
used to specify which field to select, plus what to call the results object
in the return payload.

I'd like to find out if there's any chance that facets could be included in
the URI construct. Is it technically possible?

cheers,
Ian

--
Benjamin DEVEZE

--
Benjamin DEVEZE


(Shay Banon) #4

There isn't a specific format for it, but, you can pass the actual json
body in the URI under the source URI parameter.

On Thu, Apr 26, 2012 at 7:37 PM, ian mayo ian@planetmayo.com wrote:

Hi all,
I'm aspiring to a nice & tidy REST API for my app. ES forms part of my
app, but unfortunately the complexity of search is dirtying my lovely API.
Across the rest of the system I've managed to included all data in the URI

  • which both has a functional beauty, but it also offers opportunities for
    cache-ing, either at the server or the client.

In my ES searches, I'm able to encode the search in the URI, using "
_search?q=user:kimchy". But, to get the facets that relate to that
search I'll have to switch to storing the data in a JSON payload.

I imagine it's possible to use a similar encoding to list which facets
should be present in the results, particularly if the field-name is both
used to specify which field to select, plus what to call the results object
in the return payload.

I'd like to find out if there's any chance that facets could be included
in the URI construct. Is it technically possible?

cheers,
Ian


(tfreitas) #5

Hi kimcky

can you sample from that?

--


(Mark Waddle-2) #6

curl
'http://localhost:9200/_search?pretty=true&fields=&source={"query":{"query_string":{"query":""}}}'

Although I am not doing it here, I recommend you URL encode the JSON.

On Sunday, August 12, 2012 9:03:10 PM UTC-7, tfreitas wrote:

Hi kimcky

can you sample from that?

--


(system) #7