Caching of filters, merging of indices

Elastic Stack Elasticsearch
1

Hi,
I want users to have the possibility to define filters on the
searchindex and to only search in this domain, what would be the best
approach to realize this?

The different filters can be very long and over many terms, in a way
that it would be impratical to rely on filtering at querytime.
The problem that comes to mind when defining flags at indexing time,
is the inability to change a filter without reindexing.

Is it possible to define filters or secondary indices in elasticsearch
that will automatically be cached, in a way that:

  • When a user changes a filter, the cache will be updated.
  • When new documents are indexed, the cache will be updated. (could I
    use a percolator for this?)
  • Would it be possible to periodically update the index without
    reindexing, updating the flags for the filters. (Could I use a river
    for this?)

Another thing I wondering about, is it possible to merge different indices?

Thanks,
Michel

2

How many terms? Only hundreds? Did you tried it?

is it possible to merge different indices?

you can use the _scan search + feeding to merge two indices

On Jun 1, 6:23 pm, Michel Conrad michel.con...@trendiction.com
wrote:

Hi,
I want users to have the possibility to define filters on the
searchindex and to only search in this domain, what would be the best
approach to realize this?

The different filters can be very long and over many terms, in a way
that it would be impratical to rely on filtering at querytime.
The problem that comes to mind when defining flags at indexing time,
is the inability to change a filter without reindexing.

Is it possible to define filters or secondary indices in elasticsearch
that will automatically be cached, in a way that:

  • When a user changes a filter, the cache will be updated.
  • When new documents are indexed, the cache will be updated. (could I
    use a percolator for this?)
  • Would it be possible to periodically update the index without
    reindexing, updating the flags for the filters. (Could I use a river
    for this?)

Another thing I wondering about, is it possible to merge different indices?

Thanks,
Michel

3

If you want to create "auxiliary" indices, then you will need to manage them yourself. Either periodically reindex into them, or update them on the fly, but, the logic will need to be in your app. A plugin will not have special features compared to something running on "your end".

There isn't an option to merge two indices... .

On Wednesday, June 1, 2011 at 7:23 PM, Michel Conrad wrote:

Hi,
I want users to have the possibility to define filters on the
searchindex and to only search in this domain, what would be the best
approach to realize this?

The different filters can be very long and over many terms, in a way
that it would be impratical to rely on filtering at querytime.
The problem that comes to mind when defining flags at indexing time,
is the inability to change a filter without reindexing.

Is it possible to define filters or secondary indices in elasticsearch
that will automatically be cached, in a way that:

  • When a user changes a filter, the cache will be updated.
  • When new documents are indexed, the cache will be updated. (could I
    use a percolator for this?)
  • Would it be possible to periodically update the index without
    reindexing, updating the flags for the filters. (Could I use a river
    for this?)

Another thing I wondering about, is it possible to merge different indices?

Thanks,
Michel