I'm looking for a search engine that provides document level access /
filtering. I had looked at ManifoldCF with Solr as one option. Does
elasticsearch offer this kind of functionality? I need to be able to
restrict search results in a granual way, whereby each document can have
permissions set for each potential user of the system.
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even
index.
And when a user query comes , incercept the query in between using a proxy
or something , see if the user have permission and then only let it through.
I'm looking for a search engine that provides document level access /
filtering. I had looked at ManifoldCF with Solr as one option. Does
elasticsearch offer this kind of functionality? I need to be able to
restrict search results in a granual way, whereby each document can have
permissions set for each potential user of the system.
Building on this question, are you considering adding an internal security
model to ES?
If so, have you looked at what was done in the Apache Accumulo project?
Short story, this was a Big Table implementation built out by the US
Intelligence Community and built out a finer grained security model that
still scaled. If you are considering adding ES security, I'm curious if you
think this level of granularity is desirable/feasible.
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even
index.
And when a user query comes , incercept the query in between using a proxy
or something , see if the user have permission and then only let it through.
I'm looking for a search engine that provides document level access /
filtering. I had looked at ManifoldCF with Solr as one option. Does
elasticsearch offer this kind of functionality? I need to be able to
restrict search results in a granual way, whereby each document can have
permissions set for each potential user of the system.
The "small solution" is just to hand over authorization tokens, index them,
and later add filters to all queries automatically. This could be done by a
web front-end proxy placed before the Elasticsearch index/search REST API.
I think the most preferred way to have document-level security would be a
"big solution", i.e. writing an Elasticsearch security plugin with the
following features included
transport layer security between client and server (TLS)
server awareness of client identity (establish ACL/role based security)
secure indexing: only allow authorized clients to write documents
(perform ACL/role-based checks)
secure query: index visibility, document visibility, filter query results
due to client authorization
enforce document / index data integrity (checksums, end-to-end security)
constraint violation alerting
and, really advanced stuff, adjust highlighting to filter out sensitive
(restricted) fields (privacy)
even more "paranoid" stuff
Most of the work could be taken over by front-end web processes, and the
Elasticsearch security plugin could offer some conventions for a security
model, i.e. rules for where index/document-level TLS/ACL/Role stuff is
stored and how it is used by clients via a REST API.
The "small solution" is just to hand over authorization tokens, index
them, and later add filters to all queries automatically. This could be
done by a web front-end proxy placed before the Elasticsearch index/search
REST API.
I think the most preferred way to have document-level security would be a
"big solution", i.e. writing an Elasticsearch security plugin with the
following features included
transport layer security between client and server (TLS)
server awareness of client identity (establish ACL/role based security)
secure indexing: only allow authorized clients to write documents
(perform ACL/role-based checks)
secure query: index visibility, document visibility, filter query
results due to client authorization
enforce document / index data integrity (checksums, end-to-end security)
constraint violation alerting
and, really advanced stuff, adjust highlighting to filter out sensitive
(restricted) fields (privacy)
even more "paranoid" stuff
Most of the work could be taken over by front-end web processes, and the
Elasticsearch security plugin could offer some conventions for a security
model, i.e. rules for where index/document-level TLS/ACL/Role stuff is
stored and how it is used by clients via a REST API.
Sorry for delay in response, i was out of the country.
My requirements are not about filtering the incoming search queries, but
rather about restricting the indexed documents to ones a user has
permission to search... i think it has to be done by the search engine or
paging etc would not work properly.
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even
index.
And when a user query comes , incercept the query in between using a proxy
or something , see if the user have permission and then only let it through.
I'm looking for a search engine that provides document level access /
filtering. I had looked at ManifoldCF with Solr as one option. Does
elasticsearch offer this kind of functionality? I need to be able to
restrict search results in a granual way, whereby each document can have
permissions set for each potential user of the system.
Sorry for delay in response, i was out of the country.
My requirements are not about filtering the incoming search queries, but
rather about restricting the indexed documents to ones a user has
permission to search... i think it has to be done by the search engine or
paging etc would not work properly.
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even
index.
And when a user query comes , incercept the query in between using a
proxy or something , see if the user have permission and then only let it
through.
I'm looking for a search engine that provides document level access /
filtering. I had looked at ManifoldCF with Solr as one option. Does
elasticsearch offer this kind of functionality? I need to be able to
restrict search results in a granual way, whereby each document can have
permissions set for each potential user of the system.
Did you find a solution for this problem ? I have quite the same
requirments/ questions about differents sources (out of ManifoldCF that
seems to manage the whole bunch)....
Thanks.
Yann
Le jeudi 5 avril 2012 20:17:09 UTC+2, Richard Brown a écrit :
Hi Vineeth,
Sorry for delay in response, i was out of the country.
My requirements are not about filtering the incoming search queries, but
rather about restricting the indexed documents to ones a user has
permission to search... i think it has to be done by the search engine or
paging etc would not work properly.
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even
index.
And when a user query comes , incercept the query in between using a
proxy or something , see if the user have permission and then only let it
through.
Thanks
Vineeth
On Wed, Mar 21, 2012 at 5:53 PM, Richard Brown <richard...@gmail.com<javascript:>
wrote:
Hi,
I'm looking for a search engine that provides document level access /
filtering. I had looked at ManifoldCF with Solr as one option. Does
elasticsearch offer this kind of functionality? I need to be able to
restrict search results in a granual way, whereby each document can have
permissions set for each potential user of the system.
Did you find a solution for this problem ? I have quite the same
requirments/ questions about differents sources (out of ManifoldCF that
seems to manage the whole bunch)....
Thanks.
Yann
Le jeudi 5 avril 2012 20:17:09 UTC+2, Richard Brown a écrit :
Hi Vineeth,
Sorry for delay in response, i was out of the country.
My requirements are not about filtering the incoming search queries, but
rather about restricting the indexed documents to ones a user has
permission to search... i think it has to be done by the search engine or
paging etc would not work properly.
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even
index.
And when a user query comes , incercept the query in between using a
proxy or something , see if the user have permission and then only let it
through.
I'm looking for a search engine that provides document level access /
filtering. I had looked at ManifoldCF with Solr as one option. Does
elasticsearch offer this kind of functionality? I need to be able to
restrict search results in a granual way, whereby each document can have
permissions set for each potential user of the system.
May be not useful idea in your particular use case but in many
organizations document-user visibility is driven by organization
structure/groups, i.e. not really at the level of individual documents. If
that would fit to your situation then you can consider indexing documents
into indices by "organization departments" and then allow users to search
across relevant indices only. Also index aliases could be useful in this
scenario.
Am Sonntag, 25. März 2012 13:46:53 UTC+2 schrieb Jörg Prante:
Security in search engines is a wide subject.
The "small solution" is just to hand over authorization tokens, index
them, and later add filters to all queries automatically. This could be
done by a web front-end proxy placed before the Elasticsearch index/search
REST API.
I think the most preferred way to have document-level security would be a
"big solution", i.e. writing an Elasticsearch security plugin with the
following features included
transport layer security between client and server (TLS)
server awareness of client identity (establish ACL/role based security)
secure indexing: only allow authorized clients to write documents
(perform ACL/role-based checks)
secure query: index visibility, document visibility, filter query
results due to client authorization
enforce document / index data integrity (checksums, end-to-end security)
constraint violation alerting
and, really advanced stuff, adjust highlighting to filter out sensitive
(restricted) fields (privacy)
even more "paranoid" stuff
Most of the work could be taken over by front-end web processes, and the
Elasticsearch security plugin could offer some conventions for a security
model, i.e. rules for where index/document-level TLS/ACL/Role stuff is
stored and how it is used by clients via a REST API.
Did you find a solution for this problem ? I have quite the same
requirments/ questions about differents sources (out of ManifoldCF that
seems to manage the whole bunch)....
Thanks.
Yann
Le jeudi 5 avril 2012 20:17:09 UTC+2, Richard Brown a écrit :
Hi Vineeth,
Sorry for delay in response, i was out of the country.
My requirements are not about filtering the incoming search queries, but
rather about restricting the indexed documents to ones a user has
permission to search... i think it has to be done by the search engine or
paging etc would not work properly.
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or
even index.
And when a user query comes , incercept the query in between using a
proxy or something , see if the user have permission and then only let it
through.
I'm looking for a search engine that provides document level access /
filtering. I had looked at ManifoldCF with Solr as one option. Does
elasticsearch offer this kind of functionality? I need to be able to
restrict search results in a granual way, whereby each document can have
permissions set for each potential user of the system.
Have you implemented this? Can you please give me an idea. I am planning to use kibana for search. If we index user_id or someother data of a document, how do we filter it in user search.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.