Help Understanding custom_filters_score Error


(Chris-5) #1

Hi All,

I'm attempting to convert an OR filter query over to a custom_filters_score
query, as I want to boost certain combinations of results higher.

Each document has three name fields, and I want to match any document that
has a specific name in any of the fields (and boost higher if it matches
multiple name fields), but increase the boosting even more if it matches a
specific name field AND a set of address fields. I can't seem to see where
I'm making the mistake, and will admit I probably have completely missed
the mark on the query.

My attempt to re-write this query has resulted in an error, which I'll give
first - and then the query:

nested: QueryParsingException[[cases] [_na] filter malformed, no field after start_object

And, here is the query:

{
"query": {
"custom_filters_score": {
"query": {
"match_all": {

    }
  },
  "score_mode": "multiply",
  "filters": [
    {
      "boost": "1.2",
      "filter": {
        "fquery": {
          "_name": "name1",
          "query": {
            "field": {
              "name1": "John AND Doe"
            }
          }
        }
      }
    },
    {
      "boost": "1.2",
      "filter": {
        "fquery": {
          "_name": "name2",
          "query": {
            "field": {
              "name2": "John AND Doe"
            }
          }
        }
      }
    },
    {
      "boost": "1.2",
      "filter": {
        "and": [
          {
            "fquery": {
              "_name": "name3",
              "query": {
                "field": {
                  "name3": "John AND Doe"
                }
              }
            }
          }
        ]
      }
    },
    {
      "filter": {
        "boost": "1.5",
        "and": [
          {
            "query": {
              "field": {
                "zip": "77001"
              }
            }
          },
          {
            "query": {
              "field": {
                "city": "Houston"
              }
            }
          },
          {
            "query": {
              "field": {
                "address_1": "111 AND Main AND Street"
              }
            }
          },
          {
            "query": {
              "field": {
                "state": "TX"
              }
            }
          },
          {
            "fquery": {
              "_name": "name3",
              "query": {
                "field": {
                  "name3": "John AND Doe"
                }
              }
            }
          }
        ]
      }
    }
  ]
}

},
"size": 200
}

Note: I use fquery's as it is very important that I know which filter matched, for handling the results.

Am I completely off-base on the use of custom_filters_score here?

Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Chris-5) #2

Of course, posting in public results in me (finally) seeing the obvious
error:

"filter": {
"boost": "1.5",

Should be:

"boost": "1.5",
"filter": {

However, unlike my previous OR filter, which only returned results which matched one of the filters, now it returns all records (with the right records boosted to the top) - even those that don't match any filters?

Is it possible to replicate the behavior of an OR filter query with a customer_filters_score query?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(James Martin) #3

Hi Chris,

I'm in the same boat; looking to combine an "or filter" (so 1 or the other
filter matches) with a custom_filters_score in order to boost results which
meet a certain criteria.
Did you have any luck solving this?

On Friday, 3 May 2013 15:26:52 UTC+10, Chris wrote:

Of course, posting in public results in me (finally) seeing the obvious
error:

"filter": {
"boost": "1.5",

Should be:

"boost": "1.5",
"filter": {

However, unlike my previous OR filter, which only returned results which matched one of the filters, now it returns all records (with the right records boosted to the top) - even those that don't match any filters?

Is it possible to replicate the behavior of an OR filter query with a customer_filters_score query?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/11ab4355-3dbd-4809-a61d-f8ffc5c28686%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Binh Ly-2) #4

You should be able to put your "overall query" into the top query part of
your custom_filters_score. FYI, the function_score should now be used in ES
1.0. The function_score has the flexibility to either augment your top
level query scores, or replace them if needed - check the boost_mode
parameter.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/96772985-ccd7-47b5-abd5-c84b6b932eef%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Chris-5) #5

James,

It's been a little while, but if my memory serves me correctly, the correct
solution is that 'filters' and 'query' must be at the same level in the
custom_filters_score query and you still need a filtering or limiting query
in the query section, e.g.:

custom_filters_score : {
query : {
filtered: {
filter : {
or : [ ] <--- your record matching filters go here
},
query : { } <--- or you can issue a query here
},
score_mode : 'score_multiply',
filters : [ <--- your boost filters go here
{
boost : '1.5',
filter : {
and : [ ]
}
},
...
]
}

The problem in the query I posted in my original message is that I was
trying to get the scoring filters to do the record filtering, which they
don't.

On Thu, Feb 27, 2014 at 12:11 AM, James Martin jamsii@gmail.com wrote:

Hi Chris,

I'm in the same boat; looking to combine an "or filter" (so 1 or the other
filter matches) with a custom_filters_score in order to boost results which
meet a certain criteria.
Did you have any luck solving this?

On Friday, 3 May 2013 15:26:52 UTC+10, Chris wrote:

Of course, posting in public results in me (finally) seeing the obvious
error:

"filter": {
"boost": "1.5",

Should be:

"boost": "1.5",
"filter": {

However, unlike my previous OR filter, which only returned results which matched one of the filters, now it returns all records (with the right records boosted to the top) - even those that don't match any filters?

Is it possible to replicate the behavior of an OR filter query with a customer_filters_score query?

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/zXCnWFEW7bE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/11ab4355-3dbd-4809-a61d-f8ffc5c28686%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKVTPE36Ht96CNXNCcDJa67G9moXGkh76PfWrFg_KdZSOsHokQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(jdepp99) #6

Just trying to figure out where these custom queries get created? In Elasticsearch or can it be done in kibana interface?


(James Martin) #7

Hi Chris,

Thanks for getting back to me. Originally had trouble getting your example to work, but after reading your statement;

"the correct solution is that 'filters' and 'query' must be at the same level"

I realised your example didn’t have them at the same level, oops! Once I sorted that, things started to work.
Once again, thanks.

James

On 28 February 2014 at 1:24:42 am, Chris Church (thisdroneeatspeople@gmail.com) wrote:

James,

It's been a little while, but if my memory serves me correctly, the correct solution is that 'filters' and 'query' must be at the same level in the custom_filters_score query and you still need a filtering or limiting query in the query section, e.g.:

custom_filters_score : {
query : {
filtered: {
filter : {
or : [ ] <--- your record matching filters go here
},
query : { } <--- or you can issue a query here
},
score_mode : 'score_multiply',
filters : [ <--- your boost filters go here
{
boost : '1.5',
filter : {
and : [ ]
}
},
...
]
}

The problem in the query I posted in my original message is that I was trying to get the scoring filters to do the record filtering, which they don't.

On Thu, Feb 27, 2014 at 12:11 AM, James Martin jamsii@gmail.com wrote:
Hi Chris,

I'm in the same boat; looking to combine an "or filter" (so 1 or the other filter matches) with a custom_filters_score in order to boost results which meet a certain criteria.
Did you have any luck solving this?

On Friday, 3 May 2013 15:26:52 UTC+10, Chris wrote:
Of course, posting in public results in me (finally) seeing the obvious error:

"filter": {
"boost": "1.5",

Should be:

"boost": "1.5",
"filter": {

However, unlike my previous OR filter, which only returned results which matched one of the filters, now it returns all records (with the right records boosted to the top) - even those that don't match any filters?

Is it possible to replicate the behavior of an OR filter query with a customer_filters_score query?

You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/zXCnWFEW7bE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/11ab4355-3dbd-4809-a61d-f8ffc5c28686%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/zXCnWFEW7bE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKVTPE36Ht96CNXNCcDJa67G9moXGkh76PfWrFg_KdZSOsHokQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.530fe5fe.515f007c.143%40J.local.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #8