# Using part of ES as a public, frontend API? **URL:** \ **Category:** Elasticsearch\ **Created:** [June 26, 2011, 4:13pm UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710 "2011-06-26T16:13:25Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Bartosz\_Pietrzak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bartosz_pietrzak/32/3195_2.png) [@Bartosz\_Pietrzak](https://discuss.elastic.co/u/Bartosz_Pietrzak)\ **Post date:** [June 26, 2011, 4:13pm UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710/1 "2011-06-26T16:13:25Z")
Quick question (as I have not seen any opinions on this topic) - using part of ES as a public application's API would be a good idea? Our app relies heavily on search (livesearch, in general) - limiting every layer (ruby app, in this case) would be a huge performance boost and since ES uses JSON - this was an obvious thing that came to my mind: use it as the livesearch underlying backend API directly. Bad idea, good idea? ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [June 26, 2011, 8:56pm UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710/2 "2011-06-26T20:56:26Z")
Hi, I would rather not do it. Couple of reasons why I think this is not a good idea: 1. API can change. Although ES API has been very stable for major part the release 1.0 is still not here. 2. Depending on what part of API you want to expose you should be careful. Even if you expose only search related API it would allow detailed inspection of your index structure and then anybody could create queries that can put unnecessary load on our servers. 3. If you want to log your use activity then you should do it before the request hits first ES node. I would personaly recommend the opposite: write your own proxy to expose only the minimal function set. Regards, Lukas Dne neděle, 26. června 2011, Bartosz Pietrzak [pietrzak@bartosz.me](mailto:pietrzak@bartosz.me) napsal(a): > Quick question (as I have not seen any opinions on this topic) - using > part of ES as a public application's API would be a good idea? > > Our app relies heavily on search (livesearch, in general) - limiting > every layer (ruby app, in this case) would be a huge performance boost > and since ES uses JSON - this was an obvious thing that came to my > mind: use it as the livesearch underlying backend API directly. Bad > idea, good idea? ---
**Author:** ![Eric\_Mill](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_mill/32/3124_2.png) [@Eric\_Mill](https://discuss.elastic.co/u/Eric_Mill)\ **Post date:** [June 27, 2011, 1:33am UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710/3 "2011-06-27T01:33:52Z")
I've been thinking about this too - is there any way within ES to limit public access to the read-only endpoints, or is that something that has to be configured within the hosting web server (not sure how the web server world works in Javaland)? -- Eric On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: > Hi, > I would rather not do it. Couple of reasons why I think this is not a good > idea: > > 1. API can change. Although ES API has been very stable for major part > the release 1.0 is still not here. > 2. Depending on what part of API you want to expose you should be > careful. Even if you expose only search related API it would allow > detailed inspection of your index structure and then anybody could > create queries that can put unnecessary load on our servers. > 3. If you want to log your use activity then you should do it before > the request hits first ES node. > > I would personaly recommend the opposite: write your own proxy to > expose only the minimal function set. > > Regards, > Lukas > > Dne neděle, 26. června 2011, Bartosz Pietrzak [pietrzak@bartosz.me](mailto:pietrzak@bartosz.me) > napsal(a): > > > Quick question (as I have not seen any opinions on this topic) - using > > part of ES as a public application's API would be a good idea? > > > > Our app relies heavily on search (livesearch, in general) - limiting > > every layer (ruby app, in this case) would be a huge performance boost > > and since ES uses JSON - this was an obvious thing that came to my > > mind: use it as the livesearch underlying backend API directly. Bad > > idea, good idea? ---
**Author:** ![Tim\_Hawkins](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_hawkins/32/3163_2.png) [@Tim\_Hawkins](https://discuss.elastic.co/u/Tim_Hawkins)\ **Post date:** [June 27, 2011, 6:46am UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710/4 "2011-06-27T06:46:55Z")
Guys, this is seriously not a good idea, ES is designed as a systems component, It is designed to operate inside a closed network as part of a back-end system. It does not have the security and intrusion hardening exposure needed to operate as a direct connected internet service. As stated before, use a service between ES and the web to buffer, rate limit, log and allow blocking of dangerous options. Also remember that even with a readonly query system, if you allow arbitrary queries from hosts that have little or not involvement in the design and construction of the data sets provided, then you WILL get major performance issues, again a intervening service would limit queries to the operations that your datasets can safely provide. -- Tim Hawkins Sent with Sparrow ([http://bit.ly/sigsprw](http://bit.ly/sigsprw)) On Monday, June 27, 2011 at 9:33 AM, Eric Mill wrote: > I've been thinking about this too - is there any way within ES to limit public access to the read-only endpoints, or is that something that has to be configured within the hosting web server (not sure how the web server world works in Javaland)? > > -- Eric > > On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček \<[lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) ([mailto:lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com))\> wrote: > > > Hi, > > I would rather not do it. Couple of reasons why I think this is not a good idea: > > > > 1. API can change. Although ES API has been very stable for major part > > the release 1.0 is still not here. > > 2. Depending on what part of API you want to expose you should be > > careful. Even if you expose only search related API it would allow > > detailed inspection of your index structure and then anybody could > > create queries that can put unnecessary load on our servers. > > 3. If you want to log your use activity then you should do it before > > the request hits first ES node. > > > > I would personaly recommend the opposite: write your own proxy to > > expose only the minimal function set. > > > > Regards, > > Lukas > > > > Dne neděle, 26. června 2011, Bartosz Pietrzak \<[pietrzak@bartosz.me](mailto:pietrzak@bartosz.me) ([mailto:pietrzak@bartosz.me](mailto:pietrzak@bartosz.me))\> napsal(a): > > > > > Quick question (as I have not seen any opinions on this topic) - using > > > part of ES as a public application's API would be a good idea? > > > > > > Our app relies heavily on search (livesearch, in general) - limiting > > > every layer (ruby app, in this case) would be a huge performance boost > > > and since ES uses JSON - this was an obvious thing that came to my > > > mind: use it as the livesearch underlying backend API directly. Bad > > > idea, good idea? ---
**Author:** ![Eric\_Mill](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_mill/32/3124_2.png) [@Eric\_Mill](https://discuss.elastic.co/u/Eric_Mill)\ **Post date:** [June 27, 2011, 2:37pm UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710/5 "2011-06-27T14:37:23Z")
Elasticsearch also supports JSONP and powers the search on [elasticsearch.org](http://elasticsearch.org). I see your points, and even agree that it is likely a bad idea, but you didn't answer my question. Is there any way within ES to limit destructive endpoints to credentialed users, or should this be done at the web server level? -- Eric On Mon, Jun 27, 2011 at 2:46 AM, Tim Hawkins [tim.thawkins@gmail.com](mailto:tim.thawkins@gmail.com) wrote: > Guys, this is seriously not a good idea, ES is designed as a systems > component, It is designed to operate inside a closed network as part of a > back-end system. It does not have the security and intrusion hardening > exposure needed to operate as a direct connected internet service. > > As stated before, use a service between ES and the web to buffer, rate > limit, log and allow blocking of dangerous options. > > Also remember that even with a readonly query system, if you allow > arbitrary queries from hosts that have little or not involvement in the > design and construction of the data sets provided, then you WILL get major > performance issues, again a intervening service would limit queries to the > operations that your datasets can safely provide. > > -- > Tim Hawkins > Sent with Sparrow ([http://bit.ly/sigsprw](http://bit.ly/sigsprw)) > > On Monday, June 27, 2011 at 9:33 AM, Eric Mill wrote: > > > I've been thinking about this too - is there any way within ES to limit > > public access to the read-only endpoints, or is that something that has to > > be configured within the hosting web server (not sure how the web server > > world works in Javaland)? > > > > -- Eric > > > > On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček \<[lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)(mailto: > > [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com))\> wrote: > > > > > Hi, > > > I would rather not do it. Couple of reasons why I think this is not a > > > good idea: > > > > > > 1. API can change. Although ES API has been very stable for major part > > > the release 1.0 is still not here. > > > 2. Depending on what part of API you want to expose you should be > > > careful. Even if you expose only search related API it would allow > > > detailed inspection of your index structure and then anybody could > > > create queries that can put unnecessary load on our servers. > > > 3. If you want to log your use activity then you should do it before > > > the request hits first ES node. > > > > > > I would personaly recommend the opposite: write your own proxy to > > > expose only the minimal function set. > > > > > > Regards, > > > Lukas > > > > > > Dne neděle, 26. června 2011, Bartosz Pietrzak \<[pietrzak@bartosz.me](mailto:pietrzak@bartosz.me)(mailto: > > > [pietrzak@bartosz.me](mailto:pietrzak@bartosz.me))\> napsal(a): > > > > > > > Quick question (as I have not seen any opinions on this topic) - > > > > using > > > > part of ES as a public application's API would be a good idea? > > > > > > > > Our app relies heavily on search (livesearch, in general) - limiting > > > > every layer (ruby app, in this case) would be a huge performance > > > > boost > > > > and since ES uses JSON - this was an obvious thing that came to my > > > > mind: use it as the livesearch underlying backend API directly. Bad > > > > idea, good idea? ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [June 27, 2011, 2:46pm UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710/6 "2011-06-27T14:46:58Z")
Hi, there is nothing in ES out of the box to support user roles, authorization, authentication and secure access. You need to handle that yourself. Regards, Lukas On Mon, Jun 27, 2011 at 4:37 PM, Eric Mill [kprojection@gmail.com](mailto:kprojection@gmail.com) wrote: > Elasticsearch also supports JSONP and powers the search on > [elasticsearch.org](http://elasticsearch.org). I see your points, and even agree that it is likely a > bad idea, but you didn't answer my question. Is there any way within ES to > limit destructive endpoints to credentialed users, or should this be done at > the web server level? > > -- Eric > > On Mon, Jun 27, 2011 at 2:46 AM, Tim Hawkins [tim.thawkins@gmail.com](mailto:tim.thawkins@gmail.com)wrote: > > > Guys, this is seriously not a good idea, ES is designed as a systems > > component, It is designed to operate inside a closed network as part of a > > back-end system. It does not have the security and intrusion hardening > > exposure needed to operate as a direct connected internet service. > > > > As stated before, use a service between ES and the web to buffer, rate > > limit, log and allow blocking of dangerous options. > > > > Also remember that even with a readonly query system, if you allow > > arbitrary queries from hosts that have little or not involvement in the > > design and construction of the data sets provided, then you WILL get major > > performance issues, again a intervening service would limit queries to the > > operations that your datasets can safely provide. > > > > -- > > Tim Hawkins > > Sent with Sparrow ([http://bit.ly/sigsprw](http://bit.ly/sigsprw)) > > > > On Monday, June 27, 2011 at 9:33 AM, Eric Mill wrote: > > > > > I've been thinking about this too - is there any way within ES to limit > > > public access to the read-only endpoints, or is that something that has to > > > be configured within the hosting web server (not sure how the web server > > > world works in Javaland)? > > > > > > -- Eric > > > > > > On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček \<[lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)(mailto: > > > [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com))\> wrote: > > > > > > > Hi, > > > > I would rather not do it. Couple of reasons why I think this is not a > > > > good idea: > > > > > > > > 1. API can change. Although ES API has been very stable for major > > > > part > > > > the release 1.0 is still not here. > > > > 2. Depending on what part of API you want to expose you should be > > > > careful. Even if you expose only search related API it would allow > > > > detailed inspection of your index structure and then anybody could > > > > create queries that can put unnecessary load on our servers. > > > > 3. If you want to log your use activity then you should do it before > > > > the request hits first ES node. > > > > > > > > I would personaly recommend the opposite: write your own proxy to > > > > expose only the minimal function set. > > > > > > > > Regards, > > > > Lukas > > > > > > > > Dne neděle, 26. června 2011, Bartosz Pietrzak \<[pietrzak@bartosz.me](mailto:pietrzak@bartosz.me)(mailto: > > > > [pietrzak@bartosz.me](mailto:pietrzak@bartosz.me))\> napsal(a): > > > > > > > > > Quick question (as I have not seen any opinions on this topic) - > > > > > using > > > > > part of ES as a public application's API would be a good idea? > > > > > > > > > > Our app relies heavily on search (livesearch, in general) - limiting > > > > > every layer (ruby app, in this case) would be a huge performance > > > > > boost > > > > > and since ES uses JSON - this was an obvious thing that came to my > > > > > mind: use it as the livesearch underlying backend API directly. Bad > > > > > idea, good idea? ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [July 4, 2011, 8:00am UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710/7 "2011-07-04T08:00:54Z")
a) It is very easy to support authorization, authentication, secure access and _much more_ on the HTTP level. See this gist for inspiration: [Route requests to ElasticSearch to authenticated user's own index with an Nginx reverse-proxy · GitHub](https://gist.github.com/986390) b) It is a not convenient to _directly_ expose the ES API to your users. I don't believe a thin Ruby wrapper around ES adds any significant overhead. And you'll need things like authentication, throttling, storing analytics anyway... Karel On Jun 27, 4:46 pm, Lukáš Vlček [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) wrote: > Hi, > > there is nothing in ES out of the box to support user roles, authorization, > authentication and secure access. You need to handle that yourself. > > Regards, > Lukas > > On Mon, Jun 27, 2011 at 4:37 PM, Eric Mill [kproject...@gmail.com](mailto:kproject...@gmail.com) wrote: > > > Elasticsearch also supports JSONP and powers the search on > > [elasticsearch.org](http://elasticsearch.org). I see your points, and even agree that it is likely a > > bad idea, but you didn't answer my question. Is there any way within ES to > > limit destructive endpoints to credentialed users, or should this be done at > > the web server level? > > > -- Eric > > > On Mon, Jun 27, 2011 at 2:46 AM, Tim Hawkins [tim.thawk...@gmail.com](mailto:tim.thawk...@gmail.com)wrote: > > > > Guys, this is seriously not a good idea, ES is designed as a systems > > > component, It is designed to operate inside a closed network as part of a > > > back-end system. It does not have the security and intrusion hardening > > > exposure needed to operate as a direct connected internet service. > > > > As stated before, use a service between ES and the web to buffer, rate > > > limit, log and allow blocking of dangerous options. > > > > Also remember that even with a readonly query system, if you allow > > > arbitrary queries from hosts that have little or not involvement in the > > > design and construction of the data sets provided, then you WILL get major > > > performance issues, again a intervening service would limit queries to the > > > operations that your datasets can safely provide. > > > > -- > > > Tim Hawkins > > > Sent with Sparrow ([http://bit.ly/sigsprw](http://bit.ly/sigsprw)) > > > > On Monday, June 27, 2011 at 9:33 AM, Eric Mill wrote: > > > > > I've been thinking about this too - is there any way within ES to limit > > > > public access to the read-only endpoints, or is that something that has to > > > > be configured within the hosting web server (not sure how the web server > > > > world works in Javaland)? > > > > > -- Eric > > > > > On Sun, Jun 26, 2011 at 4:56 PM, Lukáš Vlček \<[lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com)(mailto: > > > > [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com))\> wrote: > > > > > > > > > Hi, > > > > > I would rather not do it. Couple of reasons why I think this is not a > > > > > good idea: > > > > > > 1. API can change. Although ES API has been very stable for major > > > > > part > > > > > the release 1.0 is still not here. > > > > > 2. Depending on what part of API you want to expose you should be > > > > > careful. Even if you expose only search related API it would allow > > > > > detailed inspection of your index structure and then anybody could > > > > > create queries that can put unnecessary load on our servers. > > > > > 3. If you want to log your use activity then you should do it before > > > > > the request hits first ES node. > > > > > > I would personaly recommend the opposite: write your own proxy to > > > > > expose only the minimal function set. > > > > > > Regards, > > > > > Lukas > > > > > > Dne neděle, 26. června 2011, Bartosz Pietrzak \<[pietr...@bartosz.me](mailto:pietr...@bartosz.me)(mailto: > > > > > [pietr...@bartosz.me](mailto:pietr...@bartosz.me))\> napsal(a): > > > > > > > > > > > Quick question (as I have not seen any opinions on this topic) - > > > > > > using > > > > > > part of ES as a public application's API would be a good idea? > > > > > > > Our app relies heavily on search (livesearch, in general) - limiting > > > > > > every layer (ruby app, in this case) would be a huge performance > > > > > > boost > > > > > > and since ES uses JSON - this was an obvious thing that came to my > > > > > > mind: use it as the livesearch underlying backend API directly. Bad > > > > > > idea, good idea? ---
**Author:** ![Mahendra\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mahendra_m/32/3128_2.png) [@Mahendra\_M](https://discuss.elastic.co/u/Mahendra_M)\ **Post date:** [July 4, 2011, 11:02am UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710/8 "2011-07-04T11:02:44Z")
Hi, On Sun, Jun 26, 2011 at 9:43 PM, Bartosz Pietrzak [pietrzak@bartosz.me](mailto:pietrzak@bartosz.me) wrote: > Quick question (as I have not seen any opinions on this topic) - using > part of ES as a public application's API would be a good idea? > > Our app relies heavily on search (livesearch, in general) - limiting > every layer (ruby app, in this case) would be a huge performance boost > and since ES uses JSON - this was an obvious thing that came to my > mind: use it as the livesearch underlying backend API directly. Bad > idea, good idea? In my case, I have started off by exposing CouchDB APIs to external clients, but over time I have exposed Elasticsearch APIs to our data. My setup is something like this: 1. nginx (security, https, limiting to GET/POST) etc. This also does load balancing to the below layers. 2. A django (spawning + eventlet) server handling auth and acls. 3. Elasticsearch beneath django. The plan is to slowly remove the Django setup in between. Also planning to use the "aliases" feature to restrict what queries can be run on ES. ( [Aliases: add an ability to specify filters on aliases · Issue #971 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/971) ). Have not tried the aliases feature myself. Just my idea. Regards, Mahendra [http://twitter.com/mahendra](http://twitter.com/mahendra) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:01am UTC](https://discuss.elastic.co/t/using-part-of-es-as-a-public-frontend-api/4710/9 "2017-07-06T04:01:53Z")