Highlight problem


(locojay) #1

Hi

my mapping is

"mongodocid":{"boost":1.0,
"store": "yes",
"type": "string"},
"fulltext": {"boost": 1.0,
"index": "analyzed",
"store": "yes",
"type": "string",
"term_vector" : "with_positions_offsets"}}
to highlight the complete fulltext i am setting number_of_framgments
to 0.

if i do the following lucene like string query:

'{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test"}}, "size": 100}'

for some documents in the resulset the lengh of the highlighted
fulltext is smaller than ten fulltext itself. Since i am setting
number_of_fragments to 0 and pre_tags/post_tags are added this should
not happen.

Now comes the strange behaviour. if i only search for one of this
failing elements by doing this

{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test AND mongodocid:
4d0a861c2ebef6032c00b1ec"}}, "size": 100}

then all works fine.

Any idea?


(Lukáš Vlček) #2

Hi, which version of ES are you using?
Regards,
Lukas
Dne 22.12.2010 20:29 "locojay" jassin.meknassi@googlemail.com napsal(a):

Hi

my mapping is

"mongodocid":{"boost":1.0,
"store": "yes",
"type": "string"},
"fulltext": {"boost": 1.0,
"index": "analyzed",
"store": "yes",
"type": "string",
"term_vector" : "with_positions_offsets"}}
to highlight the complete fulltext i am setting number_of_framgments
to 0.

if i do the following lucene like string query:

'{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test"}}, "size": 100}'

for some documents in the resulset the lengh of the highlighted
fulltext is smaller than ten fulltext itself. Since i am setting
number_of_fragments to 0 and pre_tags/post_tags are added this should
not happen.

Now comes the strange behaviour. if i only search for one of this
failing elements by doing this

{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test AND mongodocid:
4d0a861c2ebef6032c00b1ec"}}, "size": 100}

then all works fine.

Any idea?


(locojay) #3

Hi i m using 0.13.1

On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote:

Hi, which version of ES are you using?
Regards,
Lukas

Dne 22.12.2010 20:29 "locojay" jassin.meknassi@googlemail.com napsal(a):

Hi

my mapping is

"mongodocid":{"boost":1.0,
"store": "yes",
"type": "string"},
"fulltext": {"boost": 1.0,
"index": "analyzed",
"store": "yes",
"type": "string",
"term_vector" : "with_positions_offsets"}}
to highlight the complete fulltext i am setting number_of_framgments
to 0.

if i do the following lucene like string query:

'{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test"}}, "size": 100}'

for some documents in the resulset the lengh of the highlighted
fulltext is smaller than ten fulltext itself. Since i am setting
number_of_fragments to 0 and pre_tags/post_tags are added this should
not happen.

Now comes the strange behaviour. if i only search for one of this
failing elements by doing this

{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test AND mongodocid:
4d0a861c2ebef6032c00b1ec"}}, "size": 100}

then all works fine.

Any idea?


(Lukáš Vlček) #4

Can you try master? There has been fixed the following bug

Lukáš
Dne 22.12.2010 20:39 "Jassin Meknassi" jassin.meknassi@googlemail.com
napsal(a):

Hi i m using 0.13.1

On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote:

Hi, which version of ES are you using?
Regards,
Lukas

Dne 22.12.2010 20:29 "locojay" jassin.meknassi@googlemail.com
napsal(a):

Hi

my mapping is

"mongodocid":{"boost":1.0,
"store": "yes",
"type": "string"},
"fulltext": {"boost": 1.0,
"index": "analyzed",
"store": "yes",
"type": "string",
"term_vector" : "with_positions_offsets"}}
to highlight the complete fulltext i am setting number_of_framgments
to 0.

if i do the following lucene like string query:

'{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test"}}, "size": 100}'

for some documents in the resulset the lengh of the highlighted
fulltext is smaller than ten fulltext itself. Since i am setting
number_of_fragments to 0 and pre_tags/post_tags are added this should
not happen.

Now comes the strange behaviour. if i only search for one of this
failing elements by doing this

{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test AND mongodocid:
4d0a861c2ebef6032c00b1ec"}}, "size": 100}

then all works fine.

Any idea?


(Jassin MEKNASSI) #5

took longer to build the master. elasticsearch-0.14.0-SNAPSHOT

instead of having a smaller size most have no entry:

result['hits'][i]['highlight']['fulltext'] is an empty list

On Dec 22, 2010, at 2:41 PM, Lukáš Vlček wrote:

Can you try master? There has been fixed the following bug https://github.com/elasticsearch/elasticsearch/issues/issue/479

Lukáš

Dne 22.12.2010 20:39 "Jassin Meknassi" jassin.meknassi@googlemail.com napsal(a):

Hi i m using 0.13.1

On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote:

Hi, which version of ES are you using?
Regards,
Lukas

Dne 22.12.2010 20:29 "locojay" jassin.meknassi@googlemail.com napsal(a):

Hi

my mapping is

"mongodocid":{"boost":1.0,
"store": "yes",
"type": "string"},
"fulltext": {"boost": 1.0,
"index": "analyzed",
"store": "yes",
"type": "string",
"term_vector" : "with_positions_offsets"}}
to highlight the complete fulltext i am setting number_of_framgments
to 0.

if i do the following lucene like string query:

'{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test"}}, "size": 100}'

for some documents in the resulset the lengh of the highlighted
fulltext is smaller than ten fulltext itself. Since i am setting
number_of_fragments to 0 and pre_tags/post_tags are added this should
not happen.

Now comes the strange behaviour. if i only search for one of this
failing elements by doing this

{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test AND mongodocid:
4d0a861c2ebef6032c00b1ec"}}, "size": 100}

then all works fine.

Any idea?


(Lukáš Vlček) #6

Can you please gist some curl recreation?

On Wed, Dec 22, 2010 at 9:44 PM, Jassin MEKNASSI
jassin.meknassi@gmail.comwrote:

took longer to build the master. elasticsearch-0.14.0-SNAPSHOT

instead of having a smaller size most have no entry:

result['hits'][i]['highlight']['fulltext'] is an empty list

On Dec 22, 2010, at 2:41 PM, Lukáš Vlček wrote:

Can you try master? There has been fixed the following bug
https://github.com/elasticsearch/elasticsearch/issues/issue/479

Lukáš
Dne 22.12.2010 20:39 "Jassin Meknassi" jassin.meknassi@googlemail.com
napsal(a):

Hi i m using 0.13.1

On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote:

Hi, which version of ES are you using?
Regards,
Lukas

Dne 22.12.2010 20:29 "locojay" jassin.meknassi@googlemail.com
napsal(a):

Hi

my mapping is

"mongodocid":{"boost":1.0,
"store": "yes",
"type": "string"},
"fulltext": {"boost": 1.0,
"index": "analyzed",
"store": "yes",
"type": "string",
"term_vector" : "with_positions_offsets"}}
to highlight the complete fulltext i am setting number_of_framgments
to 0.

if i do the following lucene like string query:

'{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test"}}, "size": 100}'

for some documents in the resulset the lengh of the highlighted
fulltext is smaller than ten fulltext itself. Since i am setting
number_of_fragments to 0 and pre_tags/post_tags are added this should
not happen.

Now comes the strange behaviour. if i only search for one of this
failing elements by doing this

{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test AND mongodocid:
4d0a861c2ebef6032c00b1ec"}}, "size": 100}

then all works fine.

Any idea?


(Jassin MEKNASSI) #7

unfortunately its confidential data.
reindexing on the failing docs using master works.
I will rebuild my index overnight and see if it works

Thanks for the help

On Dec 22, 2010, at 3:49 PM, Lukáš Vlček wrote:

Can you please gist some curl recreation?

On Wed, Dec 22, 2010 at 9:44 PM, Jassin MEKNASSI jassin.meknassi@gmail.com wrote:
took longer to build the master. elasticsearch-0.14.0-SNAPSHOT

instead of having a smaller size most have no entry:

result['hits'][i]['highlight']['fulltext'] is an empty list

On Dec 22, 2010, at 2:41 PM, Lukáš Vlček wrote:

Can you try master? There has been fixed the following bug https://github.com/elasticsearch/elasticsearch/issues/issue/479

Lukáš

Dne 22.12.2010 20:39 "Jassin Meknassi" jassin.meknassi@googlemail.com napsal(a):

Hi i m using 0.13.1

On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote:

Hi, which version of ES are you using?
Regards,
Lukas

Dne 22.12.2010 20:29 "locojay" jassin.meknassi@googlemail.com napsal(a):

Hi

my mapping is

"mongodocid":{"boost":1.0,
"store": "yes",
"type": "string"},
"fulltext": {"boost": 1.0,
"index": "analyzed",
"store": "yes",
"type": "string",
"term_vector" : "with_positions_offsets"}}
to highlight the complete fulltext i am setting number_of_framgments
to 0.

if i do the following lucene like string query:

'{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test"}}, "size": 100}'

for some documents in the resulset the lengh of the highlighted
fulltext is smaller than ten fulltext itself. Since i am setting
number_of_fragments to 0 and pre_tags/post_tags are added this should
not happen.

Now comes the strange behaviour. if i only search for one of this
failing elements by doing this

{"highlight": {"pre_tags": "", "fields": {"fulltext":
{"number_of_fragments": 0}}, "post_tags": "
"}, "query":
{"query_string": {"query": "fulltext:test AND mongodocid:
4d0a861c2ebef6032c00b1ec"}}, "size": 100}

then all works fine.

Any idea?


(system) #8