I am currently investigating the ELK (Elasticsearch, Logstash, Kibana)
stack for centralized log file analysis.
The plan is to store logs of multiple applications in the same
Elasticsearch cluster using logstash and day-based indexes.
All documents contain a field called application, e.g."application":
"superapp".
Now we are looking for a way to implement access control like this:
A) Superuser: is able to see log entries of all applications.
B) Developer: can only see log entries of the applications he is allowed
to. For example the dev team for application "superapp" should only be able
to see the entries for this application.
To wrap it up: we need access control based on the value in the field
application.
While reading the documentation for Elastisearch and Shield I could not
find an obvious way to do it.
Any ideas how we could realize this in a way that would also work with
Kibana 3 and 4?
My first idea was to use aliases which are being automatically assigned to
documents using index templates. I am wondering if this is the right
direction.
I am currently investigating the ELK (Elasticsearch, Logstash, Kibana)
stack for centralized log file analysis.
The plan is to store logs of multiple applications in the same
Elasticsearch cluster using logstash and day-based indexes.
All documents contain a field called application, e.g."application":
"superapp".
Now we are looking for a way to implement access control like this:
A) Superuser: is able to see log entries of all applications.
B) Developer: can only see log entries of the applications he is
allowed to. For example the dev team for application "superapp" should only
be able to see the entries for this application.
To wrap it up: we need access control based on the value in the field
application.
While reading the documentation for Elastisearch and Shield I could not
find an obvious way to do it.
Any ideas how we could realize this in a way that would also work with
Kibana 3 and 4?
My first idea was to use aliases which are being automatically assigned to
documents using index templates. I am wondering if this is the right
direction.
could you please elaborate a little more detailed how this could be done?
Would you suggest indexes per application and tie them together via
aliases? Still wonder how this could be integrated into Kibana...
Best regards,
Philipp
Am Freitag, 27. März 2015 10:17:37 UTC+1 schrieb Mark Walkom:
I am currently investigating the ELK (Elasticsearch, Logstash, Kibana)
stack for centralized log file analysis.
The plan is to store logs of multiple applications in the same
Elasticsearch cluster using logstash and day-based indexes.
All documents contain a field called application, e.g."application":
"superapp".
Now we are looking for a way to implement access control like this:
A) Superuser: is able to see log entries of all applications.
B) Developer: can only see log entries of the applications he is
allowed to. For example the dev team for application "superapp" should only
be able to see the entries for this application.
To wrap it up: we need access control based on the value in the field
application.
While reading the documentation for Elastisearch and Shield I could not
find an obvious way to do it.
Any ideas how we could realize this in a way that would also work with
Kibana 3 and 4?
My first idea was to use aliases which are being automatically assigned
to documents using index templates. I am wondering if this is the right
direction.
This is where it gets hard and you need to leverage your own domain
knowledge.
You can separate out the different types of logs into their own indices
which would make things much easier, you could also setup an alias with a
filter and then provide access to that alias to certain users.
Currently KB isn't multi-tenanted but it is a feature that is going to be
added, you'd have to setup multiple instances with each going to their own
alias.
could you please elaborate a little more detailed how this could be done?
Would you suggest indexes per application and tie them together via
aliases? Still wonder how this could be integrated into Kibana...
Best regards,
Philipp
Am Freitag, 27. März 2015 10:17:37 UTC+1 schrieb Mark Walkom:
I am currently investigating the ELK (Elasticsearch, Logstash, Kibana)
stack for centralized log file analysis.
The plan is to store logs of multiple applications in the same
Elasticsearch cluster using logstash and day-based indexes.
All documents contain a field called application, e.g."application":
"superapp".
Now we are looking for a way to implement access control like this:
A) Superuser: is able to see log entries of all applications.
B) Developer: can only see log entries of the applications he is
allowed to. For example the dev team for application "superapp" should only
be able to see the entries for this application.
To wrap it up: we need access control based on the value in the field
application.
While reading the documentation for Elastisearch and Shield I could not
find an obvious way to do it.
Any ideas how we could realize this in a way that would also work with
Kibana 3 and 4?
My first idea was to use aliases which are being automatically assigned
to documents using index templates. I am wondering if this is the right
direction.
Regarding search performance: would you recommend using aliases+filters or
separate indexes per application?
Am Samstag, 28. März 2015 06:46:00 UTC+1 schrieb Mark Walkom:
This is where it gets hard and you need to leverage your own domain
knowledge.
You can separate out the different types of logs into their own indices
which would make things much easier, you could also setup an alias with a
filter and then provide access to that alias to certain users.
Currently KB isn't multi-tenanted but it is a feature that is going to be
added, you'd have to setup multiple instances with each going to their own
alias.
could you please elaborate a little more detailed how this could be done?
Would you suggest indexes per application and tie them together via
aliases? Still wonder how this could be integrated into Kibana...
Best regards,
Philipp
Am Freitag, 27. März 2015 10:17:37 UTC+1 schrieb Mark Walkom:
I am currently investigating the ELK (Elasticsearch, Logstash, Kibana)
stack for centralized log file analysis.
The plan is to store logs of multiple applications in the same
Elasticsearch cluster using logstash and day-based indexes.
All documents contain a field called application, e.g."application":
"superapp".
Now we are looking for a way to implement access control like this:
A) Superuser: is able to see log entries of all applications.
B) Developer: can only see log entries of the applications he is
allowed to. For example the dev team for application "superapp" should only
be able to see the entries for this application.
To wrap it up: we need access control based on the value in the field
application.
While reading the documentation for Elastisearch and Shield I could not
find an obvious way to do it.
Any ideas how we could realize this in a way that would also work with
Kibana 3 and 4?
My first idea was to use aliases which are being automatically assigned
to documents using index templates. I am wondering if this is the right
direction.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.