Securing Data in Elasticsearch


(Harvii Dent) #1

Hello,

I'm planning to use Elasticsearch with Logstash for logs management and
search, however, one thing I'm unable to find an answer for is making sure
that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at once,
but they can still be deleted. Are there any options to prevent deleting
indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index files
on disk are not changed (so they can be monitored using a file integrity
monitoring solution)? as many regulatory and compliance bodies have
requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

There are a lot of methods to tamper with ES files, and physically,
everything is possible to modify in files as long as your operating system
permits more than something like "append-only" mode for ES files (not that
I know this would work)

So it depends on your requirements about the security level you want to
reach, if ES settings alone can help you or if you need more (paranoid)
configurations.

Jörg

On Thu, Jun 12, 2014 at 4:48 PM, Harvii Dent harviident@gmail.com wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management and
search, however, one thing I'm unable to find an answer for is making sure
that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at once,
but they can still be deleted. Are there any options to prevent deleting
indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index files
on disk are not changed (so they can be monitored using a file integrity
monitoring solution)? as many regulatory and compliance bodies have
requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGzpQ_A3vwMB%3DF0YBHA3dpQGVuXBLUAVWddGa8HZJ2U7g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Harvii Dent) #3

ES settings alone would be great, are there other options that I could have
missed? right now the main priority is preventing document updates/deletes
(and index deletes) via the ES rest api.

Thanks

On Thursday, June 12, 2014 6:21:36 PM UTC+3, Jörg Prante wrote:

There are a lot of methods to tamper with ES files, and physically,
everything is possible to modify in files as long as your operating system
permits more than something like "append-only" mode for ES files (not that
I know this would work)

So it depends on your requirements about the security level you want to
reach, if ES settings alone can help you or if you need more (paranoid)
configurations.

Jörg

On Thu, Jun 12, 2014 at 4:48 PM, Harvii Dent <harvi...@gmail.com
<javascript:>> wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management and
search, however, one thing I'm unable to find an answer for is making sure
that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at once,
but they can still be deleted. Are there any options to prevent deleting
indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index files
on disk are not changed (so they can be monitored using a file integrity
monitoring solution)? as many regulatory and compliance bodies have
requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #4

If you want ES-level security, you should first reduce attack vectors, by
closing down all the open ports and resources that are not necessary.

One step would be to disable HTTP REST API completely (port 9200) and run
Logstash Elasticsearch output only
http://logstash.net/docs/1.4.1/outputs/elasticsearch

As a consequence, you could only kill the ES process on a node, or send
Java API commands. It is not possible to block Java API commands over port
9300, this is how nodes talk to each other. You could imagine a
self-written tool for administering your cluster that uses the Java API
only (from a J2EE web app for example)

On the node on OS level, you would have to protect the OS user of ES node
is running under from being accessed by third party users.

Jörg

On Thu, Jun 12, 2014 at 5:30 PM, Harvii Dent harviident@gmail.com wrote:

ES settings alone would be great, are there other options that I could
have missed? right now the main priority is preventing document
updates/deletes (and index deletes) via the ES rest api.

Thanks

On Thursday, June 12, 2014 6:21:36 PM UTC+3, Jörg Prante wrote:

There are a lot of methods to tamper with ES files, and physically,
everything is possible to modify in files as long as your operating system
permits more than something like "append-only" mode for ES files (not that
I know this would work)

So it depends on your requirements about the security level you want to
reach, if ES settings alone can help you or if you need more (paranoid)
configurations.

Jörg

On Thu, Jun 12, 2014 at 4:48 PM, Harvii Dent harvi...@gmail.com wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management and
search, however, one thing I'm unable to find an answer for is making sure
that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at
once, but they can still be deleted. Are there any options to prevent
deleting indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index
files on disk are not changed (so they can be monitored using a file
integrity monitoring solution)? as many regulatory and compliance bodies
have requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHVdFqaPgsjVgBWFtC%3DTm5SpsS6S8LKooSi%3DYP60A8s8g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Harvii Dent) #5

ES nodes would be locked down and accessible only to authorized users on
the OS level; it's the ability to delete and update indices/documents
remotely that's worrisome in this case.

Disabling HTTP REST API completely is not possible since it's required by
Kibana (running behind a reverse proxy), although I suppose I could
restrict the ES node to only accept traffic from Logstash on port 9300 and
from the reverse proxy on port 9200, would this provide sufficient
protection?

Thanks

On Thursday, June 12, 2014 6:44:33 PM UTC+3, Jörg Prante wrote:

If you want ES-level security, you should first reduce attack vectors, by
closing down all the open ports and resources that are not necessary.

One step would be to disable HTTP REST API completely (port 9200) and run
Logstash Elasticsearch output only
http://logstash.net/docs/1.4.1/outputs/elasticsearch

As a consequence, you could only kill the ES process on a node, or send
Java API commands. It is not possible to block Java API commands over port
9300, this is how nodes talk to each other. You could imagine a
self-written tool for administering your cluster that uses the Java API
only (from a J2EE web app for example)

On the node on OS level, you would have to protect the OS user of ES node
is running under from being accessed by third party users.

Jörg

On Thu, Jun 12, 2014 at 5:30 PM, Harvii Dent <harvi...@gmail.com
<javascript:>> wrote:

ES settings alone would be great, are there other options that I could
have missed? right now the main priority is preventing document
updates/deletes (and index deletes) via the ES rest api.

Thanks

On Thursday, June 12, 2014 6:21:36 PM UTC+3, Jörg Prante wrote:

There are a lot of methods to tamper with ES files, and physically,
everything is possible to modify in files as long as your operating system
permits more than something like "append-only" mode for ES files (not that
I know this would work)

So it depends on your requirements about the security level you want to
reach, if ES settings alone can help you or if you need more (paranoid)
configurations.

Jörg

On Thu, Jun 12, 2014 at 4:48 PM, Harvii Dent harvi...@gmail.com wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management and
search, however, one thing I'm unable to find an answer for is making sure
that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at
once, but they can still be deleted. Are there any options to prevent
deleting indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index
files on disk are not changed (so they can be monitored using a file
integrity monitoring solution)? as many regulatory and compliance bodies
have requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #6

You should start HTTP only on localhost then and run Kibana on a selected
number of nodes only.

There are some authentication solutions for Kibana.

I am not able to find security features like audit trails or preventing
writes in Kibana/ES so you have to take care. Assessing Kibana for attacks
over the web (intrusion detection, executing commands etc) is useful, I
don't know if anyone has tried such a thing, but it is a very complex task.

Because this variant is tedious and maybe not successful, I would opt for a
different approach. Keep a checksummed copy of an index at a safe
restricted place on a "private" ES cluster (or burn it even to optical
media) and rsync a copy of it to an unsafe place, to another "public" ES
cluster where Kibana runs. Checksum verification can prove if index is
modified in the meantime at the public place.

Jörg

On Fri, Jun 13, 2014 at 8:18 PM, Harvii Dent harviident@gmail.com wrote:

ES nodes would be locked down and accessible only to authorized users on
the OS level; it's the ability to delete and update indices/documents
remotely that's worrisome in this case.

Disabling HTTP REST API completely is not possible since it's required by
Kibana (running behind a reverse proxy), although I suppose I could
restrict the ES node to only accept traffic from Logstash on port 9300 and
from the reverse proxy on port 9200, would this provide sufficient
protection?

Thanks

On Thursday, June 12, 2014 6:44:33 PM UTC+3, Jörg Prante wrote:

If you want ES-level security, you should first reduce attack vectors, by
closing down all the open ports and resources that are not necessary.

One step would be to disable HTTP REST API completely (port 9200) and run
Logstash Elasticsearch output only http://logstash.net/docs/1.4.
1/outputs/elasticsearch

As a consequence, you could only kill the ES process on a node, or send
Java API commands. It is not possible to block Java API commands over port
9300, this is how nodes talk to each other. You could imagine a
self-written tool for administering your cluster that uses the Java API
only (from a J2EE web app for example)

On the node on OS level, you would have to protect the OS user of ES node
is running under from being accessed by third party users.

Jörg

On Thu, Jun 12, 2014 at 5:30 PM, Harvii Dent harvi...@gmail.com wrote:

ES settings alone would be great, are there other options that I could
have missed? right now the main priority is preventing document
updates/deletes (and index deletes) via the ES rest api.

Thanks

On Thursday, June 12, 2014 6:21:36 PM UTC+3, Jörg Prante wrote:

There are a lot of methods to tamper with ES files, and physically,
everything is possible to modify in files as long as your operating system
permits more than something like "append-only" mode for ES files (not that
I know this would work)

So it depends on your requirements about the security level you want to
reach, if ES settings alone can help you or if you need more (paranoid)
configurations.

Jörg

On Thu, Jun 12, 2014 at 4:48 PM, Harvii Dent harvi...@gmail.com
wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management
and search, however, one thing I'm unable to find an answer for is making
sure that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at
once, but they can still be deleted. Are there any options to prevent
deleting indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index
files on disk are not changed (so they can be monitored using a file
integrity monitoring solution)? as many regulatory and compliance bodies
have requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEhU8w%2BemP0gsNCphJ5c-dQTh25409K8ozGx_aKP5E-TA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Harvii Dent) #7

Thanks Jörg. Something like OSSEC (www.ossec.net) can provide the
checksumming you mention below but I guess it would only work on indices
that have been finalized and marked as read-only (assuming ES does not
modify the files on disk at this point).

As for active indices, is there anything that can be done on the reverse
proxy to prevent delete and update operations coming from Kibana's side? I
believe some requests can be filtered but I'm not sure exactly which
without affecting the main functionality of Kibana.

Regards

On Saturday, June 14, 2014 12:44:31 AM UTC+3, Jörg Prante wrote:

You should start HTTP only on localhost then and run Kibana on a selected
number of nodes only.

There are some authentication solutions for Kibana.

I am not able to find security features like audit trails or preventing
writes in Kibana/ES so you have to take care. Assessing Kibana for attacks
over the web (intrusion detection, executing commands etc) is useful, I
don't know if anyone has tried such a thing, but it is a very complex task.

Because this variant is tedious and maybe not successful, I would opt for
a different approach. Keep a checksummed copy of an index at a safe
restricted place on a "private" ES cluster (or burn it even to optical
media) and rsync a copy of it to an unsafe place, to another "public" ES
cluster where Kibana runs. Checksum verification can prove if index is
modified in the meantime at the public place.

Jörg

On Fri, Jun 13, 2014 at 8:18 PM, Harvii Dent <harvi...@gmail.com
<javascript:>> wrote:

ES nodes would be locked down and accessible only to authorized users on
the OS level; it's the ability to delete and update indices/documents
remotely that's worrisome in this case.

Disabling HTTP REST API completely is not possible since it's required by
Kibana (running behind a reverse proxy), although I suppose I could
restrict the ES node to only accept traffic from Logstash on port 9300 and
from the reverse proxy on port 9200, would this provide sufficient
protection?

Thanks

On Thursday, June 12, 2014 6:44:33 PM UTC+3, Jörg Prante wrote:

If you want ES-level security, you should first reduce attack vectors,
by closing down all the open ports and resources that are not necessary.

One step would be to disable HTTP REST API completely (port 9200) and
run Logstash Elasticsearch output only http://logstash.net/docs/1.4.
1/outputs/elasticsearch

As a consequence, you could only kill the ES process on a node, or send
Java API commands. It is not possible to block Java API commands over port
9300, this is how nodes talk to each other. You could imagine a
self-written tool for administering your cluster that uses the Java API
only (from a J2EE web app for example)

On the node on OS level, you would have to protect the OS user of ES
node is running under from being accessed by third party users.

Jörg

On Thu, Jun 12, 2014 at 5:30 PM, Harvii Dent harvi...@gmail.com wrote:

ES settings alone would be great, are there other options that I
could have missed? right now the main priority is preventing document
updates/deletes (and index deletes) via the ES rest api.

Thanks

On Thursday, June 12, 2014 6:21:36 PM UTC+3, Jörg Prante wrote:

There are a lot of methods to tamper with ES files, and physically,
everything is possible to modify in files as long as your operating system
permits more than something like "append-only" mode for ES files (not that
I know this would work)

So it depends on your requirements about the security level you want
to reach, if ES settings alone can help you or if you need more (paranoid)
configurations.

Jörg

On Thu, Jun 12, 2014 at 4:48 PM, Harvii Dent harvi...@gmail.com
wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management
and search, however, one thing I'm unable to find an answer for is making
sure that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at
once, but they can still be deleted. Are there any options to prevent
deleting indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index
files on disk are not changed (so they can be monitored using a file
integrity monitoring solution)? as many regulatory and compliance bodies
have requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9f7a8514-0e2c-44bf-a798-220ea0c85805%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #8

From what I know about Kibana, it just uses the HTTP API _search endpoint,
but I have not examined it more thoroughly.

It is quite simple to set up an nginx/apache reverse proxy to filter
requests.

You should add

http:
host: 127.0.0.1

to your config/elasticsearch.yml to ensure that HTTP REST ES API is not
exposed to other hosts, so nginx/apache must take the job to accept Kibana
HTTP on port 80 (443) only.

Jörg

On Sun, Jun 15, 2014 at 4:31 PM, Harvii Dent harviident@gmail.com wrote:

Thanks Jörg. Something like OSSEC (www.ossec.net) can provide the
checksumming you mention below but I guess it would only work on indices
that have been finalized and marked as read-only (assuming ES does not
modify the files on disk at this point).

As for active indices, is there anything that can be done on the reverse
proxy to prevent delete and update operations coming from Kibana's side? I
believe some requests can be filtered but I'm not sure exactly which
without affecting the main functionality of Kibana.

Regards

On Saturday, June 14, 2014 12:44:31 AM UTC+3, Jörg Prante wrote:

You should start HTTP only on localhost then and run Kibana on a selected
number of nodes only.

There are some authentication solutions for Kibana.

I am not able to find security features like audit trails or preventing
writes in Kibana/ES so you have to take care. Assessing Kibana for attacks
over the web (intrusion detection, executing commands etc) is useful, I
don't know if anyone has tried such a thing, but it is a very complex task.

Because this variant is tedious and maybe not successful, I would opt for
a different approach. Keep a checksummed copy of an index at a safe
restricted place on a "private" ES cluster (or burn it even to optical
media) and rsync a copy of it to an unsafe place, to another "public" ES
cluster where Kibana runs. Checksum verification can prove if index is
modified in the meantime at the public place.

Jörg

On Fri, Jun 13, 2014 at 8:18 PM, Harvii Dent harvi...@gmail.com wrote:

ES nodes would be locked down and accessible only to authorized users on
the OS level; it's the ability to delete and update indices/documents
remotely that's worrisome in this case.

Disabling HTTP REST API completely is not possible since it's required
by Kibana (running behind a reverse proxy), although I suppose I could
restrict the ES node to only accept traffic from Logstash on port 9300 and
from the reverse proxy on port 9200, would this provide sufficient
protection?

Thanks

On Thursday, June 12, 2014 6:44:33 PM UTC+3, Jörg Prante wrote:

If you want ES-level security, you should first reduce attack vectors,
by closing down all the open ports and resources that are not necessary.

One step would be to disable HTTP REST API completely (port 9200) and
run Logstash Elasticsearch output only http://logstash.net/docs/1.4.
1/outputs/elasticsearch

As a consequence, you could only kill the ES process on a node, or send
Java API commands. It is not possible to block Java API commands over port
9300, this is how nodes talk to each other. You could imagine a
self-written tool for administering your cluster that uses the Java API
only (from a J2EE web app for example)

On the node on OS level, you would have to protect the OS user of ES
node is running under from being accessed by third party users.

Jörg

On Thu, Jun 12, 2014 at 5:30 PM, Harvii Dent harvi...@gmail.com
wrote:

ES settings alone would be great, are there other options that I
could have missed? right now the main priority is preventing document
updates/deletes (and index deletes) via the ES rest api.

Thanks

On Thursday, June 12, 2014 6:21:36 PM UTC+3, Jörg Prante wrote:

There are a lot of methods to tamper with ES files, and physically,
everything is possible to modify in files as long as your operating system
permits more than something like "append-only" mode for ES files (not that
I know this would work)

So it depends on your requirements about the security level you want
to reach, if ES settings alone can help you or if you need more (paranoid)
configurations.

Jörg

On Thu, Jun 12, 2014 at 4:48 PM, Harvii Dent harvi...@gmail.com
wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management
and search, however, one thing I'm unable to find an answer for is making
sure that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at
once, but they can still be deleted. Are there any options to prevent
deleting indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index
files on disk are not changed (so they can be monitored using a file
integrity monitoring solution)? as many regulatory and compliance bodies
have requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18a
c-405e-8929-68be32b01a6c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9f7a8514-0e2c-44bf-a798-220ea0c85805%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9f7a8514-0e2c-44bf-a798-220ea0c85805%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHZYK1gnNFdD4HL8aRK_pR_ao55BPgctTb2BamH1z19cQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Harvii Dent) #9

Interesting, but this assumes that both Logstash and Kibana are on the on
same host as ES, correct?

While this is how everything is running in my test environment, I was
thinking of separating ES from Logstash when going to production since each
would require significant resources, although I'm not sure how accurate
that is; would there be any downsides to running Logstash and ES on the
same node?

Thanks

On Sunday, June 15, 2014 5:40:25 PM UTC+3, Jörg Prante wrote:

From what I know about Kibana, it just uses the HTTP API _search endpoint,
but I have not examined it more thoroughly.

It is quite simple to set up an nginx/apache reverse proxy to filter
requests.

You should add

http:
host: 127.0.0.1

to your config/elasticsearch.yml to ensure that HTTP REST ES API is not
exposed to other hosts, so nginx/apache must take the job to accept Kibana
HTTP on port 80 (443) only.

Jörg

On Sun, Jun 15, 2014 at 4:31 PM, Harvii Dent <harvi...@gmail.com
<javascript:>> wrote:

Thanks Jörg. Something like OSSEC (www.ossec.net) can provide the
checksumming you mention below but I guess it would only work on indices
that have been finalized and marked as read-only (assuming ES does not
modify the files on disk at this point).

As for active indices, is there anything that can be done on the reverse
proxy to prevent delete and update operations coming from Kibana's side? I
believe some requests can be filtered but I'm not sure exactly which
without affecting the main functionality of Kibana.

Regards

On Saturday, June 14, 2014 12:44:31 AM UTC+3, Jörg Prante wrote:

You should start HTTP only on localhost then and run Kibana on a
selected number of nodes only.

There are some authentication solutions for Kibana.

I am not able to find security features like audit trails or preventing
writes in Kibana/ES so you have to take care. Assessing Kibana for attacks
over the web (intrusion detection, executing commands etc) is useful, I
don't know if anyone has tried such a thing, but it is a very complex task.

Because this variant is tedious and maybe not successful, I would opt
for a different approach. Keep a checksummed copy of an index at a safe
restricted place on a "private" ES cluster (or burn it even to optical
media) and rsync a copy of it to an unsafe place, to another "public" ES
cluster where Kibana runs. Checksum verification can prove if index is
modified in the meantime at the public place.

Jörg

On Fri, Jun 13, 2014 at 8:18 PM, Harvii Dent harvi...@gmail.com wrote:

ES nodes would be locked down and accessible only to authorized users
on the OS level; it's the ability to delete and update indices/documents
remotely that's worrisome in this case.

Disabling HTTP REST API completely is not possible since it's required
by Kibana (running behind a reverse proxy), although I suppose I could
restrict the ES node to only accept traffic from Logstash on port 9300 and
from the reverse proxy on port 9200, would this provide sufficient
protection?

Thanks

On Thursday, June 12, 2014 6:44:33 PM UTC+3, Jörg Prante wrote:

If you want ES-level security, you should first reduce attack vectors,
by closing down all the open ports and resources that are not necessary.

One step would be to disable HTTP REST API completely (port 9200) and
run Logstash Elasticsearch output only http://logstash.net/docs/1.4.
1/outputs/elasticsearch

As a consequence, you could only kill the ES process on a node, or
send Java API commands. It is not possible to block Java API commands over
port 9300, this is how nodes talk to each other. You could imagine a
self-written tool for administering your cluster that uses the Java API
only (from a J2EE web app for example)

On the node on OS level, you would have to protect the OS user of ES
node is running under from being accessed by third party users.

Jörg

On Thu, Jun 12, 2014 at 5:30 PM, Harvii Dent harvi...@gmail.com
wrote:

ES settings alone would be great, are there other options that I
could have missed? right now the main priority is preventing document
updates/deletes (and index deletes) via the ES rest api.

Thanks

On Thursday, June 12, 2014 6:21:36 PM UTC+3, Jörg Prante wrote:

There are a lot of methods to tamper with ES files, and physically,
everything is possible to modify in files as long as your operating system
permits more than something like "append-only" mode for ES files (not that
I know this would work)

So it depends on your requirements about the security level you want
to reach, if ES settings alone can help you or if you need more (paranoid)
configurations.

Jörg

On Thu, Jun 12, 2014 at 4:48 PM, Harvii Dent harvi...@gmail.com
wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management
and search, however, one thing I'm unable to find an answer for is making
sure that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices
at once, but they can still be deleted. Are there any options to prevent
deleting indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index
files on disk are not changed (so they can be monitored using a file
integrity monitoring solution)? as many regulatory and compliance bodies
have requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18a
c-405e-8929-68be32b01a6c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9f7a8514-0e2c-44bf-a798-220ea0c85805%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9f7a8514-0e2c-44bf-a798-220ea0c85805%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/899eab77-fd55-4216-80f7-159e85f9248b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #10

No, with the setting, you can run Logstash and Kibana on different hosts.

Only on ES node side, you start an additional nginx/apache, to wrap the
HTTP 9200 port service with a HTTP port 80 reverse proxy service.

On Kibana, you change all port 9200 configs to port 80 configs (also the
remote host name instead of localhost)

On Logstash, you configure the native ES port 9300 output, not the HTTP ES
output.

Jörg

On Sun, Jun 15, 2014 at 4:58 PM, Harvii Dent harviident@gmail.com wrote:

Interesting, but this assumes that both Logstash and Kibana are on the on
same host as ES, correct?

While this is how everything is running in my test environment, I was
thinking of separating ES from Logstash when going to production since each
would require significant resources, although I'm not sure how accurate
that is; would there be any downsides to running Logstash and ES on the
same node?

Thanks

On Sunday, June 15, 2014 5:40:25 PM UTC+3, Jörg Prante wrote:

From what I know about Kibana, it just uses the HTTP API _search
endpoint, but I have not examined it more thoroughly.

It is quite simple to set up an nginx/apache reverse proxy to filter
requests.

You should add

http:
host: 127.0.0.1

to your config/elasticsearch.yml to ensure that HTTP REST ES API is not
exposed to other hosts, so nginx/apache must take the job to accept Kibana
HTTP on port 80 (443) only.

Jörg

On Sun, Jun 15, 2014 at 4:31 PM, Harvii Dent harvi...@gmail.com wrote:

Thanks Jörg. Something like OSSEC (www.ossec.net) can provide the
checksumming you mention below but I guess it would only work on indices
that have been finalized and marked as read-only (assuming ES does not
modify the files on disk at this point).

As for active indices, is there anything that can be done on the reverse
proxy to prevent delete and update operations coming from Kibana's side? I
believe some requests can be filtered but I'm not sure exactly which
without affecting the main functionality of Kibana.

Regards

On Saturday, June 14, 2014 12:44:31 AM UTC+3, Jörg Prante wrote:

You should start HTTP only on localhost then and run Kibana on a
selected number of nodes only.

There are some authentication solutions for Kibana.

I am not able to find security features like audit trails or preventing
writes in Kibana/ES so you have to take care. Assessing Kibana for attacks
over the web (intrusion detection, executing commands etc) is useful, I
don't know if anyone has tried such a thing, but it is a very complex task.

Because this variant is tedious and maybe not successful, I would opt
for a different approach. Keep a checksummed copy of an index at a safe
restricted place on a "private" ES cluster (or burn it even to optical
media) and rsync a copy of it to an unsafe place, to another "public" ES
cluster where Kibana runs. Checksum verification can prove if index is
modified in the meantime at the public place.

Jörg

On Fri, Jun 13, 2014 at 8:18 PM, Harvii Dent harvi...@gmail.com
wrote:

ES nodes would be locked down and accessible only to authorized users
on the OS level; it's the ability to delete and update indices/documents
remotely that's worrisome in this case.

Disabling HTTP REST API completely is not possible since it's required
by Kibana (running behind a reverse proxy), although I suppose I could
restrict the ES node to only accept traffic from Logstash on port 9300 and
from the reverse proxy on port 9200, would this provide sufficient
protection?

Thanks

On Thursday, June 12, 2014 6:44:33 PM UTC+3, Jörg Prante wrote:

If you want ES-level security, you should first reduce attack
vectors, by closing down all the open ports and resources that are not
necessary.

One step would be to disable HTTP REST API completely (port 9200) and
run Logstash Elasticsearch output only http://logstash.net/docs/1.4.
1/outputs/elasticsearch

As a consequence, you could only kill the ES process on a node, or
send Java API commands. It is not possible to block Java API commands over
port 9300, this is how nodes talk to each other. You could imagine a
self-written tool for administering your cluster that uses the Java API
only (from a J2EE web app for example)

On the node on OS level, you would have to protect the OS user of ES
node is running under from being accessed by third party users.

Jörg

On Thu, Jun 12, 2014 at 5:30 PM, Harvii Dent harvi...@gmail.com
wrote:

ES settings alone would be great, are there other options that I
could have missed? right now the main priority is preventing document
updates/deletes (and index deletes) via the ES rest api.

Thanks

On Thursday, June 12, 2014 6:21:36 PM UTC+3, Jörg Prante wrote:

There are a lot of methods to tamper with ES files, and physically,
everything is possible to modify in files as long as your operating system
permits more than something like "append-only" mode for ES files (not that
I know this would work)

So it depends on your requirements about the security level you
want to reach, if ES settings alone can help you or if you need more
(paranoid) configurations.

Jörg

On Thu, Jun 12, 2014 at 4:48 PM, Harvii Dent harvi...@gmail.com
wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs
management and search, however, one thing I'm unable to find an answer for
is making sure that the data cannot be modified once it reaches
Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices
at once, but they can still be deleted. Are there any options to prevent
deleting indices altogether?

And on the document level, is it possible to disable 'delete'
AND 'update' operations without setting the entire index as
read-only (ie. 'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the
index files on disk are not changed (so they can be monitored using a file
integrity monitoring solution)? as many regulatory and compliance bodies
have requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18a
c-405e-8929-68be32b01a6c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfc73db4-18ac-405e-8929-68be32b01a6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/190a707b-9ed
f-4128-9740-79d59f0bc209%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/190a707b-9edf-4128-9740-79d59f0bc209%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/9339cfd0-9300-496e-bc00-4179725e02db%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/9f7a8514-0e2c-44bf-a798-220ea0c85805%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9f7a8514-0e2c-44bf-a798-220ea0c85805%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/899eab77-fd55-4216-80f7-159e85f9248b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/899eab77-fd55-4216-80f7-159e85f9248b%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEjmz4SuyS2_LCwc_rBzNa16OBa9cerh268FP2zEEWXpw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Zennet Wheatcroft) #11

If we want to use Kibana we will run into the same issue. I heard Shay say
that Kibana really was not developed for the use case of exposing to
external customers but he did not elaborate on that. What I was thinking of
doing is wrapping ES in a simple web app that forwards GET requests from
Kibana on to ES (keeping the same API) but blocks DELETE, PUT, and POST
requests returning a 501 Not Implemented. Do you think that would work for
maintaining functionality and disallowing updates and deletes? Would that
work for your requirements?

Zennet

On Thursday, June 12, 2014 7:48:47 AM UTC-7, Harvii Dent wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management and
search, however, one thing I'm unable to find an answer for is making sure
that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at once,
but they can still be deleted. Are there any options to prevent deleting
indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index files
on disk are not changed (so they can be monitored using a file integrity
monitoring solution)? as many regulatory and compliance bodies have
requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e4ce0c9a-30a4-4077-b3eb-e4bb5ab2dc0b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #12

As said, you can wrap HTTP REST, and filter for GET, or just for _search
endpoint but that is only one part, and it is an incomplete solution.

More important is to isolate ES in a private network and to maintain a safe
and trusted environment (where every operation on OS level is logged and
must be authorized, and also the network and the connected devices such as
gateways and routers are known to be secure). This makes it possible to
ensure writings can be prevented. Also, append-only file systems should be
considered. In such an environment, it is possible to create a copy from a
master index, that has not been modified in the meantime, to let others
play with it.

Why is wrapping HTTP REST not enough? If you do not consider to take
special care of port 9300-9400, it is possible for attackers to craft
malevolent binary packets and submit these packets to the ES transport
protocol ports if they are exposed to outer network access.

If Kibana (or other plugins), when exposed for network access from outside,
can be used for such operations (by accident or not), I don't know for
sure. I hope it is not possible. It all looks like Kibana just uses search
methods and can not be used for attacks. But hope is not enough for
security. You have to prove by a security audit that everything is
"secure", up to a certain level of trust.

Jörg

On Wed, Jun 18, 2014 at 8:01 PM, Zennet Wheatcroft zwheatcroft@atypon.com
wrote:

If we want to use Kibana we will run into the same issue. I heard Shay say
that Kibana really was not developed for the use case of exposing to
external customers but he did not elaborate on that. What I was thinking of
doing is wrapping ES in a simple web app that forwards GET requests from
Kibana on to ES (keeping the same API) but blocks DELETE, PUT, and POST
requests returning a 501 Not Implemented. Do you think that would work for
maintaining functionality and disallowing updates and deletes? Would that
work for your requirements?

Zennet

On Thursday, June 12, 2014 7:48:47 AM UTC-7, Harvii Dent wrote:

Hello,

I'm planning to use Elasticsearch with Logstash for logs management and
search, however, one thing I'm unable to find an answer for is making sure
that the data cannot be modified once it reaches Elasticsearch.

"action.destructive_requires_name" prevents deleting all indices at
once, but they can still be deleted. Are there any options to prevent
deleting indices altogether?

And on the document level, is it possible to disable 'delete' AND
'update' operations without setting the entire index as read-only (ie.
'index.blocks.read_only')?

Lastly, does setting 'index.blocks.read_only' ensure that the index files
on disk are not changed (so they can be monitored using a file integrity
monitoring solution)? as many regulatory and compliance bodies have
requirements for ensuring logs integrity.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e4ce0c9a-30a4-4077-b3eb-e4bb5ab2dc0b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e4ce0c9a-30a4-4077-b3eb-e4bb5ab2dc0b%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEmjWGXihmXcObSvzaHduO2%3DXoGdohmUWs3CbBb81SxyQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jaguar) #13

It's good to hear more demands on security, as that's one big concern for
commercial usage.
If you are fine with the model of "open to only given group of people",
you could try plugin( jetty ) and try enable authentication from there(not
something out of box). But that is just some workaround. You don't have
control over which REST API to be exposed. And you still can't secure you
transport interface.

My 2 cents!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAP0hgQ39F-aXZ6nc4VyzvDFx_-PVKcNrkhjaqi19qeUOf_smzw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Harvii Dent) #14

@Zennet: I was thinking of doing something similar via a reverse-proxy in
front of Kibana, however I believe Kibana still uses DELETE, PUT, and POST
requests to save its dashboards, so I'm not sure what to block exactly.

@Jaguar: jetty plugin looks interesting, especially the
"jetty-restrict-writes.xml" part, I'll be taking a look at that.

As Jörg said, it shouldn't be too difficult to create a DELETE request and
spoof its source to appear as if coming from a trusted source; I just wish
there was an option built into ES to disable deletes/updates or at least
authenticate them first.

Thanks everyone

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ec799ca8-a833-49c7-9794-cbb23435e98b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Harvii Dent) #15

Just an update, it should be possible to protect ES from most malicious
requests that are generated from Kibana's end by only allowing requests as
described in this nginx config:
https://github.com/elasticsearch/kibana/blob/master/sample/nginx.conf
I've been looking at the ES API references but I didn't find any other ways
to do update/delete operations that bypass the above config, although it
would be great if someone could confirm that.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3eb483d3-ab1a-425c-b629-c490a67c668a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #16