Post_filter doesn't work on filtered documents - ES 1.0.2

Elastic Stack Elasticsearch
1

Hi,

We have around 1million documents in our index where we are performing search and have defined some script filters. As the script filters work on each document and is costly, we tried using post_filter (which has script filters defined) in order to reduce the documents on which the script filters apply.

However, when we noticed the post_filter is working on all documents rather than limiting to the filtered results
Below is the query snippet:

{
  "from" : 0,
  "size" : 20,
  "query" : {
    "filtered" : {
      "query" : {
        "query_string" : {
          "query" : "test*",
          "default_field" : "fullContent",
          "default_operator" : "and",
          "allow_leading_wildcard" : true,
          "analyze_wildcard" : true
        }
      },
      "filter" : {
        "fquery" : {
          "query" : {
            "query_string" : {
              "query" : "firstName:(\"XY-Z\" OR \"A-B-C\")",
              "default_field" : "fullContent",
              "default_operator" : "and",
              "allow_leading_wildcard" : true
            }
          },
          "_cache" : false
        }
      }
    }
  },
  "post_filter" : {
    "or" : {
      "filters" : [ {
        "fquery" : {
          "query" : {
            "query_string" : {
              "query" : "firstName:A-B-C",
              "default_field" : "fullContent",
              "default_operator" : "and",
              "allow_leading_wildcard" : true
            }
          },
          "_cache" : false
        }
      }, {
        "and" : {
          "filters" : [ {
            "and" : {
              "filters" : [ {
                "and" : {
                  "filters" : [ {
                    "fquery" : {
                      "query" : {
                        "query_string" : {
                          "query" : "NOT HeirarchicalProp:10",
                          "default_field" : "fullContent",
                          "default_operator" : "and",
                          "allow_leading_wildcard" : true
                        }
                      },
                      "_cache" : false
                    }
                  }, {
                    "script" : {
                      "script" : "native_sample_script",
                      "params" : {
                        "userCode" : "1,2,3",
                        "recordCode" : "samplecolumn"
                      },
                      "lang" : "native",
                      "_name" : "sample"
                    }
                  } ]
                }
              }, {
                "script" : {
                  "script" : "native_test_script",
                  "params" : {
                    "userCode" : "7,8,9",
                    "recordCode" : "testcolumn"
                  },
                  "lang" : "native",
                  "_name" : "test"
                }
              } ]
            }
          }, {
            "fquery" : {
              "query" : {
                "query_string" : {
                  "query" : "firstName:XY-Z",
                  "default_field" : "fullContent",
                  "default_operator" : "and",
                  "allow_leading_wildcard" : true
                }
              },
              "_cache" : false
            }
          } ]
        }
      } ]
    }
  },
  "fields" : "*"
}

Am I missing any thing while using post_filter in the above snippte ?

2

Please format your code with the </> button, it’s very hard to read.

4

Any ideas ?

5

Understood why this happens.
As "query" snippet is the last that gets executed, the documents on which the script filter or post filter gets executed is marginally high (depends on the filter conditions); For our case, we pushed the "query" as a "filter" and used match_all for query; it gave us better performance.