We are in the process of building out an Elasticsearch cluster, and we need
security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port (9200) -- teh
jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it as well (or
it defeats the purpose).
First, obviously -- has anyone out there already done this?? And if so, is
the code out there in the wild...
Second, my goggling says that we'll probably need to build this ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
The transport port range (9300-9399) is for internal cluster use. Note
there may be more ports, for example, if you install transport plugins.
If you put your whole cluster into a private network - see Private network - Wikipedia - you can control access to
ES just by adding your favorite proxy/router/firewall in front of it.
Jörg
Am 10.04.13 22:30, schrieb Chris Berry:
Greetings,
We are in the process of building out an Elasticsearch cluster, and we
need security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port (9200) --
teh jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it as
well (or it defeats the purpose).
First, obviously -- has anyone out there already done this?? And if
so, is the code out there in the wild...
Second, my goggling says that we'll probably need to build this ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
We built an OAuth2 layer with full index level ACL support on top of
elasticsearch (opposed to using the plugin architecture). This allows us to
control every aspect and elasticsearch embeds really well. We restrict
access to the transport though so you have to use HTTPS (no TCP). We
support SPDY and use an embedded instance of Jetty with async servlets (to
support Spring Security).
-Eric
On Wednesday, April 10, 2013 4:30:38 PM UTC-4, Chris Berry wrote:
Greetings,
We are in the process of building out an Elasticsearch cluster, and we
need security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port (9200) -- teh
jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it as well
(or it defeats the purpose).
First, obviously -- has anyone out there already done this?? And if so, is
the code out there in the wild...
Second, my goggling says that we'll probably need to build this ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
Could you elaborate a bit. You say you embed elastic rather than using
plugin strategy. Do you run it within a servlet container and replicate
complete API with your ACL checks before making the same elastic call? Do
you call elastic via java API or pass http request on to elastic http
endpoint. Also you said you use embedded jetty. I am confused a bit what's
embedded in what - elastic or jetty?
Thanks,
Alex
On Wednesday, April 10, 2013 6:22:30 PM UTC-4, egaumer wrote:
We built an OAuth2 layer with full index level ACL support on top of
elasticsearch (opposed to using the plugin architecture). This allows us to
control every aspect and elasticsearch embeds really well. We restrict
access to the transport though so you have to use HTTPS (no TCP). We
support SPDY and use an embedded instance of Jetty with async servlets (to
support Spring Security).
-Eric
On Wednesday, April 10, 2013 4:30:38 PM UTC-4, Chris Berry wrote:
Greetings,
We are in the process of building out an Elasticsearch cluster, and we
need security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port (9200) --
teh jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it as well
(or it defeats the purpose).
First, obviously -- has anyone out there already done this?? And if so,
is the code out there in the wild...
Second, my goggling says that we'll probably need to build this ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
This lets us intercept every call and run custom security logic before
handing it off to elasticsearch and we don't have to reimplement every
endpoint via the Java API.
In terms of Jetty, what I mean is that we don't produce a WAR, rather we
embed an instance of Jetty inside the app. So we embed elasticsearch inside
our app along side of Jetty. We debated of over using servlets but we
decided Spring Security was worth it. We're using Servlet 3.0 (async) and
we're happy with the choice because it gives us a lot of capability.
-Eric
On Friday, April 12, 2013 9:28:46 AM UTC-4, AlexR wrote:
Eric,
Could you elaborate a bit. You say you embed elastic rather than using
plugin strategy. Do you run it within a servlet container and replicate
complete API with your ACL checks before making the same elastic call? Do
you call elastic via java API or pass http request on to elastic http
endpoint. Also you said you use embedded jetty. I am confused a bit what's
embedded in what - elastic or jetty?
Thanks,
Alex
On Wednesday, April 10, 2013 6:22:30 PM UTC-4, egaumer wrote:
We built an OAuth2 layer with full index level ACL support on top of
elasticsearch (opposed to using the plugin architecture). This allows us to
control every aspect and elasticsearch embeds really well. We restrict
access to the transport though so you have to use HTTPS (no TCP). We
support SPDY and use an embedded instance of Jetty with async servlets (to
support Spring Security).
-Eric
On Wednesday, April 10, 2013 4:30:38 PM UTC-4, Chris Berry wrote:
Greetings,
We are in the process of building out an Elasticsearch cluster, and we
need security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port (9200) --
teh jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it as well
(or it defeats the purpose).
First, obviously -- has anyone out there already done this?? And if so,
is the code out there in the wild...
Second, my goggling says that we'll probably need to build this
ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
It is likely we will need to go this route in the future. Do you hide all
your cluster behind a firewall so nodes (or ingesting servers/nodes if any)
can communicate via IP safely and only expose your protected endpoint to
consuming applications? If you have just one "public" endpoint what happens
if that node fails. Or it is not an issue because this node IS your
application. so if it is down both the node and your app are down...
Thanks,
Alex
On Friday, April 12, 2013 9:57:42 PM UTC-4, egaumer wrote:
We run inside a servlet container by implementing our own RestChannel.
This lets us intercept every call and run custom security logic before
handing it off to elasticsearch and we don't have to reimplement every
endpoint via the Java API.
In terms of Jetty, what I mean is that we don't produce a WAR, rather we
embed an instance of Jetty inside the app. So we embed elasticsearch inside
our app along side of Jetty. We debated of over using servlets but we
decided Spring Security was worth it. We're using Servlet 3.0 (async) and
we're happy with the choice because it gives us a lot of capability.
-Eric
On Friday, April 12, 2013 9:28:46 AM UTC-4, AlexR wrote:
Eric,
Could you elaborate a bit. You say you embed elastic rather than using
plugin strategy. Do you run it within a servlet container and replicate
complete API with your ACL checks before making the same elastic call? Do
you call elastic via java API or pass http request on to elastic http
endpoint. Also you said you use embedded jetty. I am confused a bit what's
embedded in what - elastic or jetty?
Thanks,
Alex
On Wednesday, April 10, 2013 6:22:30 PM UTC-4, egaumer wrote:
We built an OAuth2 layer with full index level ACL support on top of
elasticsearch (opposed to using the plugin architecture). This allows us to
control every aspect and elasticsearch embeds really well. We restrict
access to the transport though so you have to use HTTPS (no TCP). We
support SPDY and use an embedded instance of Jetty with async servlets (to
support Spring Security).
-Eric
On Wednesday, April 10, 2013 4:30:38 PM UTC-4, Chris Berry wrote:
Greetings,
We are in the process of building out an Elasticsearch cluster, and we
need security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port (9200) --
teh jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it as well
(or it defeats the purpose).
First, obviously -- has anyone out there already done this?? And if so,
is the code out there in the wild...
Second, my goggling says that we'll probably need to build this
ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
Any node in our cluster could be the "public" endpoint so we typically
just have a load balancer pick one of the nodes. The transport ports
(9300, etc) are all blocked by firewall rules (except from internal
network) and the http server (9200) is completely disabled since we
only allow access though our secure servlet.
It is likely we will need to go this route in the future. Do you hide all
your cluster behind a firewall so nodes (or ingesting servers/nodes if any)
can communicate via IP safely and only expose your protected endpoint to
consuming applications? If you have just one "public" endpoint what happens
if that node fails. Or it is not an issue because this node IS your
application. so if it is down both the node and your app are down...
Thanks,
Alex
On Friday, April 12, 2013 9:57:42 PM UTC-4, egaumer wrote:
We run inside a servlet container by implementing our own RestChannel.
This lets us intercept every call and run custom security logic before
handing it off to elasticsearch and we don't have to reimplement every
endpoint via the Java API.
In terms of Jetty, what I mean is that we don't produce a WAR, rather we
embed an instance of Jetty inside the app. So we embed elasticsearch inside
our app along side of Jetty. We debated of over using servlets but we
decided Spring Security was worth it. We're using Servlet 3.0 (async) and
we're happy with the choice because it gives us a lot of capability.
-Eric
On Friday, April 12, 2013 9:28:46 AM UTC-4, AlexR wrote:
Eric,
Could you elaborate a bit. You say you embed elastic rather than using
plugin strategy. Do you run it within a servlet container and replicate
complete API with your ACL checks before making the same elastic call? Do
you call elastic via java API or pass http request on to elastic http
endpoint. Also you said you use embedded jetty. I am confused a bit what's
embedded in what - elastic or jetty?
Thanks,
Alex
On Wednesday, April 10, 2013 6:22:30 PM UTC-4, egaumer wrote:
We built an OAuth2 layer with full index level ACL support on top of
elasticsearch (opposed to using the plugin architecture). This allows us to
control every aspect and elasticsearch embeds really well. We restrict
access to the transport though so you have to use HTTPS (no TCP). We support
SPDY and use an embedded instance of Jetty with async servlets (to support
Spring Security).
-Eric
On Wednesday, April 10, 2013 4:30:38 PM UTC-4, Chris Berry wrote:
Greetings,
We are in the process of building out an Elasticsearch cluster, and we
need security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port (9200) --
teh jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it as well
(or it defeats the purpose).
First, obviously -- has anyone out there already done this?? And if so,
is the code out there in the wild...
Second, my goggling says that we'll probably need to build this
ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
I see Matt thanks it makes sense.
I wish I could pick your brain on RestChannel if we had to go this route
(our first phase does not require any index or document level ACL just deny
update/admin functionality to the REST client which I do with apache
mod_proxy for now.
It would be nice if ES had a security or at least an interceptor framework
allowing to register specialized plugins rather than doing it on transport
level like jetty plugin (or your rest channel implementation if it is much
different)
On Wednesday, April 17, 2013 5:36:55 PM UTC-4, Matt Weber wrote:
Hey Alex,
Any node in our cluster could be the "public" endpoint so we typically
just have a load balancer pick one of the nodes. The transport ports
(9300, etc) are all blocked by firewall rules (except from internal
network) and the http server (9200) is completely disabled since we
only allow access though our secure servlet.
Thanks,
Matt Weber
On Tue, Apr 16, 2013 at 10:24 AM, AlexR <royt...@gmail.com <javascript:>>
wrote:
Thanks Eric for the tip!
It is likely we will need to go this route in the future. Do you hide
all
your cluster behind a firewall so nodes (or ingesting servers/nodes if
any)
can communicate via IP safely and only expose your protected endpoint to
consuming applications? If you have just one "public" endpoint what
happens
if that node fails. Or it is not an issue because this node IS your
application. so if it is down both the node and your app are down...
Thanks,
Alex
On Friday, April 12, 2013 9:57:42 PM UTC-4, egaumer wrote:
We run inside a servlet container by implementing our own RestChannel.
This lets us intercept every call and run custom security logic before
handing it off to elasticsearch and we don't have to reimplement every
endpoint via the Java API.
In terms of Jetty, what I mean is that we don't produce a WAR, rather
we
embed an instance of Jetty inside the app. So we embed elasticsearch
inside
our app along side of Jetty. We debated of over using servlets but we
decided Spring Security was worth it. We're using Servlet 3.0 (async)
and
we're happy with the choice because it gives us a lot of capability.
-Eric
On Friday, April 12, 2013 9:28:46 AM UTC-4, AlexR wrote:
Eric,
Could you elaborate a bit. You say you embed elastic rather than using
plugin strategy. Do you run it within a servlet container and
replicate
complete API with your ACL checks before making the same elastic call?
Do
you call elastic via java API or pass http request on to elastic http
endpoint. Also you said you use embedded jetty. I am confused a bit
what's
embedded in what - elastic or jetty?
Thanks,
Alex
On Wednesday, April 10, 2013 6:22:30 PM UTC-4, egaumer wrote:
We built an OAuth2 layer with full index level ACL support on top of
elasticsearch (opposed to using the plugin architecture). This allows
us to
control every aspect and elasticsearch embeds really well. We
restrict
access to the transport though so you have to use HTTPS (no TCP). We
support
SPDY and use an embedded instance of Jetty with async servlets (to
support
Spring Security).
-Eric
On Wednesday, April 10, 2013 4:30:38 PM UTC-4, Chris Berry wrote:
Greetings,
We are in the process of building out an Elasticsearch cluster, and
we
need security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port (9200)
--
teh jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it as
well
(or it defeats the purpose).
First, obviously -- has anyone out there already done this?? And if
so,
is the code out there in the wild...
Second, my goggling says that we'll probably need to build this
ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
Thanks,
-- Chris
--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.
The hard part is integrating spring security with oauth2 and using
elasticsearch vs. a database for the acl information. I am sure Eric
is going to do a writeup about that at some point.
I see Matt thanks it makes sense.
I wish I could pick your brain on RestChannel if we had to go this route
(our first phase does not require any index or document level ACL just deny
update/admin functionality to the REST client which I do with apache
mod_proxy for now.
It would be nice if ES had a security or at least an interceptor framework
allowing to register specialized plugins rather than doing it on transport
level like jetty plugin (or your rest channel implementation if it is much
different)
On Wednesday, April 17, 2013 5:36:55 PM UTC-4, Matt Weber wrote:
Hey Alex,
Any node in our cluster could be the "public" endpoint so we typically
just have a load balancer pick one of the nodes. The transport ports
(9300, etc) are all blocked by firewall rules (except from internal
network) and the http server (9200) is completely disabled since we
only allow access though our secure servlet.
It is likely we will need to go this route in the future. Do you hide
all
your cluster behind a firewall so nodes (or ingesting servers/nodes if
any)
can communicate via IP safely and only expose your protected endpoint to
consuming applications? If you have just one "public" endpoint what
happens
if that node fails. Or it is not an issue because this node IS your
application. so if it is down both the node and your app are down...
Thanks,
Alex
On Friday, April 12, 2013 9:57:42 PM UTC-4, egaumer wrote:
We run inside a servlet container by implementing our own RestChannel.
This lets us intercept every call and run custom security logic before
handing it off to elasticsearch and we don't have to reimplement every
endpoint via the Java API.
In terms of Jetty, what I mean is that we don't produce a WAR, rather
we
embed an instance of Jetty inside the app. So we embed elasticsearch
inside
our app along side of Jetty. We debated of over using servlets but we
decided Spring Security was worth it. We're using Servlet 3.0 (async)
and
we're happy with the choice because it gives us a lot of capability.
-Eric
On Friday, April 12, 2013 9:28:46 AM UTC-4, AlexR wrote:
Eric,
Could you elaborate a bit. You say you embed elastic rather than using
plugin strategy. Do you run it within a servlet container and
replicate
complete API with your ACL checks before making the same elastic call?
Do
you call elastic via java API or pass http request on to elastic http
endpoint. Also you said you use embedded jetty. I am confused a bit
what's
embedded in what - elastic or jetty?
Thanks,
Alex
On Wednesday, April 10, 2013 6:22:30 PM UTC-4, egaumer wrote:
We built an OAuth2 layer with full index level ACL support on top of
elasticsearch (opposed to using the plugin architecture). This allows
us to
control every aspect and elasticsearch embeds really well. We
restrict
access to the transport though so you have to use HTTPS (no TCP). We
support
SPDY and use an embedded instance of Jetty with async servlets (to
support
Spring Security).
-Eric
On Wednesday, April 10, 2013 4:30:38 PM UTC-4, Chris Berry wrote:
Greetings,
We are in the process of building out an Elasticsearch cluster, and
we
need security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port (9200)
teh jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it as
well
(or it defeats the purpose).
First, obviously -- has anyone out there already done this?? And if
so,
is the code out there in the wild...
Second, my goggling says that we'll probably need to build this
ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
The hard part is integrating spring security with oauth2 and using
elasticsearch vs. a database for the acl information. I am sure Eric
is going to do a writeup about that at some point.
Thanks,
Matt Weber
On Wed, Apr 17, 2013 at 3:24 PM, AlexR <royt...@gmail.com <javascript:>>
wrote:
I see Matt thanks it makes sense.
I wish I could pick your brain on RestChannel if we had to go this route
(our first phase does not require any index or document level ACL just
deny
update/admin functionality to the REST client which I do with apache
mod_proxy for now.
It would be nice if ES had a security or at least an interceptor
framework
allowing to register specialized plugins rather than doing it on
transport
level like jetty plugin (or your rest channel implementation if it is
much
different)
On Wednesday, April 17, 2013 5:36:55 PM UTC-4, Matt Weber wrote:
Hey Alex,
Any node in our cluster could be the "public" endpoint so we typically
just have a load balancer pick one of the nodes. The transport ports
(9300, etc) are all blocked by firewall rules (except from internal
network) and the http server (9200) is completely disabled since we
only allow access though our secure servlet.
It is likely we will need to go this route in the future. Do you hide
all
your cluster behind a firewall so nodes (or ingesting servers/nodes
if
any)
can communicate via IP safely and only expose your protected endpoint
to
consuming applications? If you have just one "public" endpoint what
happens
if that node fails. Or it is not an issue because this node IS your
application. so if it is down both the node and your app are down...
Thanks,
Alex
On Friday, April 12, 2013 9:57:42 PM UTC-4, egaumer wrote:
We run inside a servlet container by implementing our own
RestChannel.
This lets us intercept every call and run custom security logic
before
handing it off to elasticsearch and we don't have to reimplement
every
endpoint via the Java API.
In terms of Jetty, what I mean is that we don't produce a WAR,
rather
we
embed an instance of Jetty inside the app. So we embed elasticsearch
inside
our app along side of Jetty. We debated of over using servlets but
we
decided Spring Security was worth it. We're using Servlet 3.0
(async)
and
we're happy with the choice because it gives us a lot of capability.
-Eric
On Friday, April 12, 2013 9:28:46 AM UTC-4, AlexR wrote:
Eric,
Could you elaborate a bit. You say you embed elastic rather than
using
plugin strategy. Do you run it within a servlet container and
replicate
complete API with your ACL checks before making the same elastic
call?
Do
you call elastic via java API or pass http request on to elastic
http
endpoint. Also you said you use embedded jetty. I am confused a bit
what's
embedded in what - elastic or jetty?
Thanks,
Alex
On Wednesday, April 10, 2013 6:22:30 PM UTC-4, egaumer wrote:
We built an OAuth2 layer with full index level ACL support on top
of
elasticsearch (opposed to using the plugin architecture). This
allows
us to
control every aspect and elasticsearch embeds really well. We
restrict
access to the transport though so you have to use HTTPS (no TCP).
We
support
SPDY and use an embedded instance of Jetty with async servlets (to
support
Spring Security).
-Eric
On Wednesday, April 10, 2013 4:30:38 PM UTC-4, Chris Berry wrote:
Greetings,
We are in the process of building out an Elasticsearch cluster,
and
we
need security. (it's too easy to do scary things to the cluster)
Currently we use OAuth2 for Service-to-service security in our
SOA
(although only 2-legged instead of 3).
And we need to add this functionality for elasticsearch.
We understand that we can pretty easily secure the HTTP port
(9200)
teh jetty plugin, etc.
But we will be using the TCP port (9300), and need to secure it
as
well
(or it defeats the purpose).
First, obviously -- has anyone out there already done this?? And
if
so,
is the code out there in the wild...
Second, my goggling says that we'll probably need to build this
ourselves.
Is creating/wiring in a Transport Plugin the correct approach??
--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.