Quick question (as I have not seen any opinions on this topic) - using
part of ES as a public application's API would be a good idea?
Our app relies heavily on search (livesearch, in general) - limiting
every layer (ruby app, in this case) would be a huge performance boost
and since ES uses JSON - this was an obvious thing that came to my
mind: use it as the livesearch underlying backend API directly. Bad
idea, good idea?
Hi,
I would rather not do it. Couple of reasons why I think this is not a good idea:
I would personaly recommend the opposite: write your own proxy to
expose only the minimal function set.
Regards,
Lukas
Dne neděle, 26. června 2011, Bartosz Pietrzak pietrzak@bartosz.me napsal(a):
Quick question (as I have not seen any opinions on this topic) - using
part of ES as a public application's API would be a good idea?Our app relies heavily on search (livesearch, in general) - limiting
every layer (ruby app, in this case) would be a huge performance boost
and since ES uses JSON - this was an obvious thing that came to my
mind: use it as the livesearch underlying backend API directly. Bad
idea, good idea?
I've been thinking about this too - is there any way within ES to limit
public access to the read-only endpoints, or is that something that has to
be configured within the hosting web server (not sure how the web server
world works in Javaland)?
-- Eric
On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček lukas.vlcek@gmail.com wrote:
Hi,
I would rather not do it. Couple of reasons why I think this is not a good
idea:
- API can change. Although ES API has been very stable for major part
the release 1.0 is still not here.- Depending on what part of API you want to expose you should be
careful. Even if you expose only search related API it would allow
detailed inspection of your index structure and then anybody could
create queries that can put unnecessary load on our servers.- If you want to log your use activity then you should do it before
the request hits first ES node.I would personaly recommend the opposite: write your own proxy to
expose only the minimal function set.Regards,
LukasDne neděle, 26. června 2011, Bartosz Pietrzak pietrzak@bartosz.me
napsal(a):Quick question (as I have not seen any opinions on this topic) - using
part of ES as a public application's API would be a good idea?Our app relies heavily on search (livesearch, in general) - limiting
every layer (ruby app, in this case) would be a huge performance boost
and since ES uses JSON - this was an obvious thing that came to my
mind: use it as the livesearch underlying backend API directly. Bad
idea, good idea?
Guys, this is seriously not a good idea, ES is designed as a systems component, It is designed to operate inside a closed network as part of a back-end system. It does not have the security and intrusion hardening exposure needed to operate as a direct connected internet service.
As stated before, use a service between ES and the web to buffer, rate limit, log and allow blocking of dangerous options.
Also remember that even with a readonly query system, if you allow arbitrary queries from hosts that have little or not involvement in the design and construction of the data sets provided, then you WILL get major performance issues, again a intervening service would limit queries to the operations that your datasets can safely provide.
--
Tim Hawkins
Sent with Sparrow (http://bit.ly/sigsprw)
On Monday, June 27, 2011 at 9:33 AM, Eric Mill wrote:
I've been thinking about this too - is there any way within ES to limit public access to the read-only endpoints, or is that something that has to be configured within the hosting web server (not sure how the web server world works in Javaland)?
-- Eric
On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček <lukas.vlcek@gmail.com (mailto:lukas.vlcek@gmail.com)> wrote:
Hi,
I would rather not do it. Couple of reasons why I think this is not a good idea:
- API can change. Although ES API has been very stable for major part
the release 1.0 is still not here.- Depending on what part of API you want to expose you should be
careful. Even if you expose only search related API it would allow
detailed inspection of your index structure and then anybody could
create queries that can put unnecessary load on our servers.- If you want to log your use activity then you should do it before
the request hits first ES node.I would personaly recommend the opposite: write your own proxy to
expose only the minimal function set.Regards,
LukasDne neděle, 26. června 2011, Bartosz Pietrzak <pietrzak@bartosz.me (mailto:pietrzak@bartosz.me)> napsal(a):
Quick question (as I have not seen any opinions on this topic) - using
part of ES as a public application's API would be a good idea?Our app relies heavily on search (livesearch, in general) - limiting
every layer (ruby app, in this case) would be a huge performance boost
and since ES uses JSON - this was an obvious thing that came to my
mind: use it as the livesearch underlying backend API directly. Bad
idea, good idea?
Elasticsearch also supports JSONP and powers the search on elasticsearch.org. I
see your points, and even agree that it is likely a bad idea, but you didn't
answer my question. Is there any way within ES to limit destructive
endpoints to credentialed users, or should this be done at the web server
level?
-- Eric
On Mon, Jun 27, 2011 at 2:46 AM, Tim Hawkins tim.thawkins@gmail.com wrote:
Guys, this is seriously not a good idea, ES is designed as a systems
component, It is designed to operate inside a closed network as part of a
back-end system. It does not have the security and intrusion hardening
exposure needed to operate as a direct connected internet service.As stated before, use a service between ES and the web to buffer, rate
limit, log and allow blocking of dangerous options.Also remember that even with a readonly query system, if you allow
arbitrary queries from hosts that have little or not involvement in the
design and construction of the data sets provided, then you WILL get major
performance issues, again a intervening service would limit queries to the
operations that your datasets can safely provide.--
Tim Hawkins
Sent with Sparrow (http://bit.ly/sigsprw)On Monday, June 27, 2011 at 9:33 AM, Eric Mill wrote:
I've been thinking about this too - is there any way within ES to limit
public access to the read-only endpoints, or is that something that has to
be configured within the hosting web server (not sure how the web server
world works in Javaland)?-- Eric
On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček <lukas.vlcek@gmail.com(mailto:
lukas.vlcek@gmail.com)> wrote:Hi,
I would rather not do it. Couple of reasons why I think this is not a
good idea:
- API can change. Although ES API has been very stable for major part
the release 1.0 is still not here.- Depending on what part of API you want to expose you should be
careful. Even if you expose only search related API it would allow
detailed inspection of your index structure and then anybody could
create queries that can put unnecessary load on our servers.- If you want to log your use activity then you should do it before
the request hits first ES node.I would personaly recommend the opposite: write your own proxy to
expose only the minimal function set.Regards,
LukasDne neděle, 26. června 2011, Bartosz Pietrzak <pietrzak@bartosz.me(mailto:
pietrzak@bartosz.me)> napsal(a):Quick question (as I have not seen any opinions on this topic) -
using
part of ES as a public application's API would be a good idea?Our app relies heavily on search (livesearch, in general) - limiting
every layer (ruby app, in this case) would be a huge performance
boost
and since ES uses JSON - this was an obvious thing that came to my
mind: use it as the livesearch underlying backend API directly. Bad
idea, good idea?
Hi,
there is nothing in ES out of the box to support user roles, authorization,
authentication and secure access. You need to handle that yourself.
Regards,
Lukas
On Mon, Jun 27, 2011 at 4:37 PM, Eric Mill kprojection@gmail.com wrote:
Elasticsearch also supports JSONP and powers the search on
elasticsearch.org. I see your points, and even agree that it is likely a
bad idea, but you didn't answer my question. Is there any way within ES to
limit destructive endpoints to credentialed users, or should this be done at
the web server level?-- Eric
On Mon, Jun 27, 2011 at 2:46 AM, Tim Hawkins tim.thawkins@gmail.comwrote:
Guys, this is seriously not a good idea, ES is designed as a systems
component, It is designed to operate inside a closed network as part of a
back-end system. It does not have the security and intrusion hardening
exposure needed to operate as a direct connected internet service.As stated before, use a service between ES and the web to buffer, rate
limit, log and allow blocking of dangerous options.Also remember that even with a readonly query system, if you allow
arbitrary queries from hosts that have little or not involvement in the
design and construction of the data sets provided, then you WILL get major
performance issues, again a intervening service would limit queries to the
operations that your datasets can safely provide.--
Tim Hawkins
Sent with Sparrow (http://bit.ly/sigsprw)On Monday, June 27, 2011 at 9:33 AM, Eric Mill wrote:
I've been thinking about this too - is there any way within ES to limit
public access to the read-only endpoints, or is that something that has to
be configured within the hosting web server (not sure how the web server
world works in Javaland)?-- Eric
On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček <lukas.vlcek@gmail.com(mailto:
lukas.vlcek@gmail.com)> wrote:Hi,
I would rather not do it. Couple of reasons why I think this is not a
good idea:
- API can change. Although ES API has been very stable for major
part
the release 1.0 is still not here.- Depending on what part of API you want to expose you should be
careful. Even if you expose only search related API it would allow
detailed inspection of your index structure and then anybody could
create queries that can put unnecessary load on our servers.- If you want to log your use activity then you should do it before
the request hits first ES node.I would personaly recommend the opposite: write your own proxy to
expose only the minimal function set.Regards,
LukasDne neděle, 26. června 2011, Bartosz Pietrzak <pietrzak@bartosz.me(mailto:
pietrzak@bartosz.me)> napsal(a):Quick question (as I have not seen any opinions on this topic) -
using
part of ES as a public application's API would be a good idea?Our app relies heavily on search (livesearch, in general) - limiting
every layer (ruby app, in this case) would be a huge performance
boost
and since ES uses JSON - this was an obvious thing that came to my
mind: use it as the livesearch underlying backend API directly. Bad
idea, good idea?
a) It is very easy to support authorization, authentication, secure
access and much more on the HTTP level. See this gist for
inspiration: Route requests to ElasticSearch to authenticated user's own index with an Nginx reverse-proxy · GitHub
b) It is a not convenient to directly expose the ES API to your
users. I don't believe a thin Ruby wrapper around ES adds any
significant overhead. And you'll need things like authentication,
throttling, storing analytics anyway...
Karel
On Jun 27, 4:46 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:
Hi,
there is nothing in ES out of the box to support user roles, authorization,
authentication and secure access. You need to handle that yourself.Regards,
LukasOn Mon, Jun 27, 2011 at 4:37 PM, Eric Mill kproject...@gmail.com wrote:
Elasticsearch also supports JSONP and powers the search on
elasticsearch.org. I see your points, and even agree that it is likely a
bad idea, but you didn't answer my question. Is there any way within ES to
limit destructive endpoints to credentialed users, or should this be done at
the web server level?-- Eric
On Mon, Jun 27, 2011 at 2:46 AM, Tim Hawkins tim.thawk...@gmail.comwrote:
Guys, this is seriously not a good idea, ES is designed as a systems
component, It is designed to operate inside a closed network as part of a
back-end system. It does not have the security and intrusion hardening
exposure needed to operate as a direct connected internet service.As stated before, use a service between ES and the web to buffer, rate
limit, log and allow blocking of dangerous options.Also remember that even with a readonly query system, if you allow
arbitrary queries from hosts that have little or not involvement in the
design and construction of the data sets provided, then you WILL get major
performance issues, again a intervening service would limit queries to the
operations that your datasets can safely provide.--
Tim Hawkins
Sent with Sparrow (http://bit.ly/sigsprw)On Monday, June 27, 2011 at 9:33 AM, Eric Mill wrote:
I've been thinking about this too - is there any way within ES to limit
public access to the read-only endpoints, or is that something that has to
be configured within the hosting web server (not sure how the web server
world works in Javaland)?-- Eric
On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček <lukas.vl...@gmail.com(mailto:
lukas.vl...@gmail.com)> wrote:Hi,
I would rather not do it. Couple of reasons why I think this is not a
good idea:
- API can change. Although ES API has been very stable for major
part
the release 1.0 is still not here.- Depending on what part of API you want to expose you should be
careful. Even if you expose only search related API it would allow
detailed inspection of your index structure and then anybody could
create queries that can put unnecessary load on our servers.- If you want to log your use activity then you should do it before
the request hits first ES node.I would personaly recommend the opposite: write your own proxy to
expose only the minimal function set.Regards,
LukasDne neděle, 26. června 2011, Bartosz Pietrzak <pietr...@bartosz.me(mailto:
pietr...@bartosz.me)> napsal(a):Quick question (as I have not seen any opinions on this topic) -
using
part of ES as a public application's API would be a good idea?Our app relies heavily on search (livesearch, in general) - limiting
every layer (ruby app, in this case) would be a huge performance
boost
and since ES uses JSON - this was an obvious thing that came to my
mind: use it as the livesearch underlying backend API directly. Bad
idea, good idea?
Hi,
On Sun, Jun 26, 2011 at 9:43 PM, Bartosz Pietrzak pietrzak@bartosz.me wrote:
Quick question (as I have not seen any opinions on this topic) - using
part of ES as a public application's API would be a good idea?Our app relies heavily on search (livesearch, in general) - limiting
every layer (ruby app, in this case) would be a huge performance boost
and since ES uses JSON - this was an obvious thing that came to my
mind: use it as the livesearch underlying backend API directly. Bad
idea, good idea?
In my case, I have started off by exposing CouchDB APIs to external clients, but
over time I have exposed Elasticsearch APIs to our data.
My setup is something like this:
The plan is to slowly remove the Django setup in between.
Also planning to use the "aliases" feature to restrict what queries
can be run on ES.
( Aliases: add an ability to specify filters on aliases · Issue #971 · elastic/elasticsearch · GitHub ). Have
not tried the aliases feature myself. Just my idea.
Regards,
Mahendra
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.