Help Understanding custom_filters_score Error

Hi All,

I'm attempting to convert an OR filter query over to a custom_filters_score
query, as I want to boost certain combinations of results higher.

Each document has three name fields, and I want to match any document that
has a specific name in any of the fields (and boost higher if it matches
multiple name fields), but increase the boosting even more if it matches a
specific name field AND a set of address fields. I can't seem to see where
I'm making the mistake, and will admit I probably have completely missed
the mark on the query.

My attempt to re-write this query has resulted in an error, which I'll give
first - and then the query:

nested: QueryParsingException[[cases] [_na] filter malformed, no field after start_object

And, here is the query:

{
"query": {
"custom_filters_score": {
"query": {
"match_all": {

    }
  },
  "score_mode": "multiply",
  "filters": [
    {
      "boost": "1.2",
      "filter": {
        "fquery": {
          "_name": "name1",
          "query": {
            "field": {
              "name1": "John AND Doe"
            }
          }
        }
      }
    },
    {
      "boost": "1.2",
      "filter": {
        "fquery": {
          "_name": "name2",
          "query": {
            "field": {
              "name2": "John AND Doe"
            }
          }
        }
      }
    },
    {
      "boost": "1.2",
      "filter": {
        "and": [
          {
            "fquery": {
              "_name": "name3",
              "query": {
                "field": {
                  "name3": "John AND Doe"
                }
              }
            }
          }
        ]
      }
    },
    {
      "filter": {
        "boost": "1.5",
        "and": [
          {
            "query": {
              "field": {
                "zip": "77001"
              }
            }
          },
          {
            "query": {
              "field": {
                "city": "Houston"
              }
            }
          },
          {
            "query": {
              "field": {
                "address_1": "111 AND Main AND Street"
              }
            }
          },
          {
            "query": {
              "field": {
                "state": "TX"
              }
            }
          },
          {
            "fquery": {
              "_name": "name3",
              "query": {
                "field": {
                  "name3": "John AND Doe"
                }
              }
            }
          }
        ]
      }
    }
  ]
}

},
"size": 200
}

Note: I use fquery's as it is very important that I know which filter matched, for handling the results.

Am I completely off-base on the use of custom_filters_score here?

Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Of course, posting in public results in me (finally) seeing the obvious
error:

"filter": {
"boost": "1.5",

Should be:

"boost": "1.5",
"filter": {

However, unlike my previous OR filter, which only returned results which matched one of the filters, now it returns all records (with the right records boosted to the top) - even those that don't match any filters?

Is it possible to replicate the behavior of an OR filter query with a customer_filters_score query?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Chris,

I'm in the same boat; looking to combine an "or filter" (so 1 or the other
filter matches) with a custom_filters_score in order to boost results which
meet a certain criteria.
Did you have any luck solving this?

On Friday, 3 May 2013 15:26:52 UTC+10, Chris wrote:

Of course, posting in public results in me (finally) seeing the obvious
error:

"filter": {
"boost": "1.5",

Should be:

"boost": "1.5",
"filter": {

However, unlike my previous OR filter, which only returned results which matched one of the filters, now it returns all records (with the right records boosted to the top) - even those that don't match any filters?

Is it possible to replicate the behavior of an OR filter query with a customer_filters_score query?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/11ab4355-3dbd-4809-a61d-f8ffc5c28686%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You should be able to put your "overall query" into the top query part of
your custom_filters_score. FYI, the function_score should now be used in ES
1.0. The function_score has the flexibility to either augment your top
level query scores, or replace them if needed - check the boost_mode
parameter.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/96772985-ccd7-47b5-abd5-c84b6b932eef%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

James,

It's been a little while, but if my memory serves me correctly, the correct
solution is that 'filters' and 'query' must be at the same level in the
custom_filters_score query and you still need a filtering or limiting query
in the query section, e.g.:

custom_filters_score : {
query : {
filtered: {
filter : {
or : <--- your record matching filters go here
},
query : { } <--- or you can issue a query here
},
score_mode : 'score_multiply',
filters : [ <--- your boost filters go here
{
boost : '1.5',
filter : {
and :
}
},
...
]
}

The problem in the query I posted in my original message is that I was
trying to get the scoring filters to do the record filtering, which they
don't.

On Thu, Feb 27, 2014 at 12:11 AM, James Martin jamsii@gmail.com wrote:

Hi Chris,

I'm in the same boat; looking to combine an "or filter" (so 1 or the other
filter matches) with a custom_filters_score in order to boost results which
meet a certain criteria.
Did you have any luck solving this?

On Friday, 3 May 2013 15:26:52 UTC+10, Chris wrote:

Of course, posting in public results in me (finally) seeing the obvious
error:

"filter": {
"boost": "1.5",

Should be:

"boost": "1.5",
"filter": {

However, unlike my previous OR filter, which only returned results which matched one of the filters, now it returns all records (with the right records boosted to the top) - even those that don't match any filters?

Is it possible to replicate the behavior of an OR filter query with a customer_filters_score query?

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/zXCnWFEW7bE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/11ab4355-3dbd-4809-a61d-f8ffc5c28686%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKVTPE36Ht96CNXNCcDJa67G9moXGkh76PfWrFg_KdZSOsHokQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Just trying to figure out where these custom queries get created? In Elasticsearch or can it be done in kibana interface?

Hi Chris,

Thanks for getting back to me. Originally had trouble getting your example to work, but after reading your statement;

"the correct solution is that 'filters' and 'query' must be at the same level"

I realised your example didn’t have them at the same level, oops! Once I sorted that, things started to work.
Once again, thanks.

James

On 28 February 2014 at 1:24:42 am, Chris Church (thisdroneeatspeople@gmail.com) wrote:

James,

It's been a little while, but if my memory serves me correctly, the correct solution is that 'filters' and 'query' must be at the same level in the custom_filters_score query and you still need a filtering or limiting query in the query section, e.g.:

custom_filters_score : {
query : {
filtered: {
filter : {
or : [ ] <--- your record matching filters go here
},
query : { } <--- or you can issue a query here
},
score_mode : 'score_multiply',
filters : [ <--- your boost filters go here
{
boost : '1.5',
filter : {
and : [ ]
}
},
...
]
}

The problem in the query I posted in my original message is that I was trying to get the scoring filters to do the record filtering, which they don't.

On Thu, Feb 27, 2014 at 12:11 AM, James Martin jamsii@gmail.com wrote:
Hi Chris,

I'm in the same boat; looking to combine an "or filter" (so 1 or the other filter matches) with a custom_filters_score in order to boost results which meet a certain criteria.
Did you have any luck solving this?

On Friday, 3 May 2013 15:26:52 UTC+10, Chris wrote:
Of course, posting in public results in me (finally) seeing the obvious error:

"filter": {
"boost": "1.5",

Should be:

"boost": "1.5",
"filter": {

However, unlike my previous OR filter, which only returned results which matched one of the filters, now it returns all records (with the right records boosted to the top) - even those that don't match any filters?

Is it possible to replicate the behavior of an OR filter query with a customer_filters_score query?

You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/zXCnWFEW7bE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/11ab4355-3dbd-4809-a61d-f8ffc5c28686%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/zXCnWFEW7bE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKVTPE36Ht96CNXNCcDJa67G9moXGkh76PfWrFg_KdZSOsHokQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.530fe5fe.515f007c.143%40J.local.
For more options, visit https://groups.google.com/groups/opt_out.