MoreLikeThis change between ES 0.16 and ES 0.17


(ruflin-2) #1

Hi

@juneym just made a pull request for Elastica with the MoreLikeThis query:

Now it seems like the query is running under 0.16 but does not work under
0.17. The query looks the following:

  •        [more_like_this] => Array*
    
  •            (*
    
  •                [fields] => Array*
    
  •                    (*
    
  •                        [0] => email*
    
  •                        [1] => content*
    
  •                    )*
    
  •                [boost] => 1*
    
  •                [like_text] => fake gmail sample*
    
  •                [max_query_terms] => 1*
    
  •                [percent_terms_to_match] => 0.3*
    
  •                [min_term_freq] => 1*
    
  •                [min_doc_freq] => 1*
    
  •                [max_doc_freq] => *
    
  •                [min_word_len] => 0*
    
  •                [max_word_len] => 0*
    
  •                [boost_terms] => 1*
    
  •            )*
    
  •    )*
    

And elasticsearch throws the following error

[2011-08-05 12:26:05,427][DEBUG][action.search.type ] [Lorna Dane]
[test][0], node[XC2VHqNjQsmuGCUIfVOPVA], [P], s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@58c3d9ac]

org.elasticsearch.search.query.QueryPhaseExecutionException: [test][0]:
query[like:fake gmail sample],from[0],size[10]: Query Failed [Failed to
execute main query]

  • at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:221)*
  • at
    org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:295)
  • at
    org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:224)
  • at
    org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryAndFetchAction.java:71)
  • at
    org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:204)
  • at
    org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:191)
  • at
    org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:177)
  • at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  • at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  • at java.lang.Thread.run(Thread.java:680)*
    Caused by: java.io.IOException: Stream closed

Any idea what is wrong with this query? I tried to figure out what changed
between ES 0.16 and 0.17, but couldn't figure out what it is.


(Vladimir Shkurin) #2

I also meet with this. The problem in "fields" parameter.

Here's a sample query and it don't work:

But if you remove the "fields", the query works :slight_smile:

  • Offtop *

Shay, I have another question about relevance. MLT function in
ElasticSearch working not as good as in Solr, the problem in fields
weight. I have set "boost" for the fields in the mapping, but they are
ignored. Could you make that request could look like for the
query_string { "fields" : ["title^7", "tags^3", "categories^2",
"description"] } ?

Many thanks in advance!


(yark) #3

There is a typo in docs. fields must be a mlt_fields

5 серп. 2011, в 18:09, Vladimir Shkurin написал(а):

I also meet with this. The problem in "fields" parameter.

Here's a sample query and it don't work:

https://gist.github.com/553660

But if you remove the "fields", the query works :slight_smile:

  • Offtop *

Shay, I have another question about relevance. MLT function in
ElasticSearch working not as good as in Solr, the problem in fields
weight. I have set "boost" for the fields in the mapping, but they are
ignored. Could you make that request could look like for the
query_string { "fields" : ["title^7", "tags^3", "categories^2",
"description"] } ?

Many thanks in advance!


(Vladimir Shkurin) #4

Yaroslav, "mlt_fields" or "mlt_fieldsafaf" is work, but it has no
effect :slight_smile: Because the parameter "mlt_fields" for More Like This API,
not for More Like This Query :slight_smile:


(Shay Banon) #5

This looks like this bug in Lucene:
https://issues.apache.org/jira/browse/LUCENE-3326. It will be fixed in 3.4,
I will see if I can workaround it for now.

On Fri, Aug 5, 2011 at 6:30 PM, Vladimir Shkurin vshkurin@gmail.com wrote:

Yaroslav, "mlt_fields" or "mlt_fieldsafaf" is work, but it has no
effect :slight_smile: Because the parameter "mlt_fields" for More Like This API,
not for More Like This Query :slight_smile:


(ruflin-2) #6

It works when I rename fields to mlt_fields. But if I understand you right,
the fields can be completely skipped as they don't have any affect? This
means, they probably should be removed from the docu?
http://www.elasticsearch.org/guide/reference/query-dsl/mlt-query.html


(Vladimir Shkurin) #7

No need to delete anything, this section of the documentation is
correct. We just have to wait when will the new release of Lucene, or
maybe Shay can fix it earlier :slight_smile:


(ruflin-2) #8

Ok, good to know.


(system) #9