It makes a lot of sence focusing on the core, and to let the user
community supply patches to extend the functionality and mashup
ElasticSearch with the other open source frameworks out there.
Looking at this from an enteprise search perspective there are two
really promising connector frameworks out there in various stages of
development, that makes it possible to index enterprise sources
The task here would be to create a middle-layer that translates the
connector protocols to something that works with ElasticSearch's apis.
For more advanced scenarios you would want to add a processing
pipeline, that lets you normalize and extend the source data. Even
more ambitious would be to have a persistent index queue that lets you
"replay" the indexing queue on demand, for fast reindexing. When you
make changes to the setup somewhere and need to reindex, it is
painfully slow doing it from the connectors when you deal with large
sources. Again, this is something that probably does not belong to the
Document level security, which the connectors support, by extracting
acl:s from the source documents, means you make sure you filter the
documents with appropriate acl:s for each request. There are some
major considerations here, how to encrypt and secure the tokens, how
to resolve the security info for each user, and so on. Building this
right into the search engine can make the experience much smoother for
the developers working with the search engine, as well as more secure,
by building in a rock solid security model.
Solr has an initiative described here
There are some patches out there to enable this, however, it is very
immature. I guess this is not prio to build in to the core, however,
maybe worth considering for the roadmap
On Oct 31, 10:18 pm, Shay Banon shay.ba...@elasticsearch.com wrote:
On Sun, Oct 31, 2010 at 11:09 PM, Mark mark.paster...@gmail.com wrote:
Is there a plan to add support for some type of document level based
security scheme to ElasticSearch that works out of the box?
What do you mean by document level security? Filtering docs based on user
roles? This can be built on top of elasticsearch easily. Adding user
management to elasticsearch is certainly possible, but not in the near
future (at least not on my plan), but contribs are welcomed. The reason is
that since this can easily be implemented on top of elasticsearch, the focus
is on more core features. Also, needing to implement it in elasticsearch
requires it to be more product level feature (pluggabble user management,
roles management, and so on).
That, and some support for Google's or Apaches connector framework and
we have a major enterprise search contender....
Can you refer to the relevant sites?
Keeo up the fantastic work!