Caching filters

Tim_Pease · November 7, 2011, 10:05pm

I am attempting to create a cached filter that will be used to filter
out search results according to some "whitelist" and "blacklist" URLs.
The filter I created can be viewed at the following gist:

gist.github.com

https://gist.github.com/TwP/1346329

explicit_cache.json

{
  "filter": {
    "and": { "_cache": true, "filters": [
      { "or": { "_cache": true, "filters": [
        {"prefix": { "_id": "http://www.example.com" }}
      ]}},

      { "not": { "_cache": true, "filter": { "or": { "_cache": true, "filters": [
        {"term": { "_id": "http://www.example.com" }},
        {"term": { "_id": "http://www.example.com/sitemap" }},

This file has been truncated. show original

implicit_cache.json

{
  "filter": {
    "and": { "_cache": true, "filters": [
      { "or": [
        {"prefix": { "_id": "http://www.example.com" }}
      ]},

      { "not": { "filter": { "or": [
        {"term": { "_id": "http://www.example.com" }},
        {"term": { "_id": "http://www.example.com/sitemap" }},

This file has been truncated. show original

My question is do I need to explicitly cache each filter? Or can I
just set { "_cache": true } on the outermost and filter?

The first file in the gist shows the explicit caching, and the second
file in the gist shows the implicit caching (only caching the and
filter). I only care about the collection of all the filters taken
together. I will not be using these same filters individually in other
queries.

Blessings,
TwP

kimchy · November 9, 2011, 7:01am

You can only cache the outmost "and", assuming the same filter will be used
in other requests. Internal filters can be cached, and then, if they are
used on their own in other type of filters, the cached form will be used
(term filters, for example, are automatically cached).

One thing that I would try and see, if possibly use a single bool filter
instead of the combination of and / or / not over the relevant term /
prefix filters. With many elements, it might make more sense.

On Tue, Nov 8, 2011 at 12:05 AM, Tim Pease tim.pease@gmail.com wrote:

I am attempting to create a cached filter that will be used to filter
out search results according to some "whitelist" and "blacklist" URLs.
The filter I created can be viewed at the following gist:

Examples of ES Filter Cache · GitHub

My question is do I need to explicitly cache each filter? Or can I
just set { "_cache": true } on the outermost and filter?

The first file in the gist shows the explicit caching, and the second
file in the gist shows the implicit caching (only caching the and
filter). I only care about the collection of all the filters taken
together. I will not be using these same filters individually in other
queries.

Blessings,
TwP

Tim_Pease · November 9, 2011, 6:01pm

Thanks for the response and clarifying the caching feature. And yes, I did
run across the "bool" filter. It does the job perfectly!

TwP