SpanNotQuery issues

Is anyone using SpanNotQuerys? Judging by a recent issue that was
never uncovered until now, I am assuming not:

The exclude portion of my SpanNotQuerys are having no effect on the query.

Given a document { "title": "Handheld Apple iPad 2 Wi Fi tablet iOS 5
64 GB 9.7 black Buy.com" }

These two queries return the same results:

SpanNotQuery
{
"query" : {
"span_not" : {
"include" : {
"span_term" : {
"title" : "ipad"
}
},
"exclude" : {
"span_term" : {
"title" : "black"
}
}
}
}
}

SpanTermQuery
{
"query" : {
"span_term" : {
"TitleString_en" : {
"value" : "ipad",
"boost" : 1.0
}
}
}
}

Explain between the two queries is identical:
9.180814 = fieldWeight(title:spanNot(ipad, black) in 531555), product of:
0.70710677 = tf(phraseFreq=0.5)
7.4192176 = idf(title: ipad=9370)
1.75 = fieldNorm(field=title, doc=531555)

SpanNotQueries in Lucene are working perfectly: spanNot(title:ipad,
title:black). I haven't traced through the code in ElasticSearch, but
the code seems to be creating the correct Lucene class. Anyone
successfully using SpanNotQuerys?

Cheers,

Ivan

--

Reproducible issue: SpanNotQuery · GitHub

On Tue, Aug 21, 2012 at 11:59 AM, Ivan Brusic ivan@brusic.com wrote:

Is anyone using SpanNotQuerys? Judging by a recent issue that was
never uncovered until now, I am assuming not:
Incorrect application of boost in SpanNotQueryBuilder · Issue #2192 · elastic/elasticsearch · GitHub

The exclude portion of my SpanNotQuerys are having no effect on the query.

Given a document { "title": "Handheld Apple iPad 2 Wi Fi tablet iOS 5
64 GB 9.7 black Buy.com" }

These two queries return the same results:

SpanNotQuery
{
"query" : {
"span_not" : {
"include" : {
"span_term" : {
"title" : "ipad"
}
},
"exclude" : {
"span_term" : {
"title" : "black"
}
}
}
}
}

SpanTermQuery
{
"query" : {
"span_term" : {
"TitleString_en" : {
"value" : "ipad",
"boost" : 1.0
}
}
}
}

Explain between the two queries is identical:
9.180814 = fieldWeight(title:spanNot(ipad, black) in 531555), product of:
0.70710677 = tf(phraseFreq=0.5)
7.4192176 = idf(title: ipad=9370)
1.75 = fieldNorm(field=title, doc=531555)

SpanNotQueries in Lucene are working perfectly: spanNot(title:ipad,
title:black). I haven't traced through the code in Elasticsearch, but
the code seems to be creating the correct Lucene class. Anyone
successfully using SpanNotQuerys?

Cheers,

Ivan

--

Slight error on my part. Without even realizing it, I was using a
custom query parser in Lucene that handled SpanNotQueries differently.
The queries work as expected in Elasticsearch, true to the Lucene
standard.

--
Ivan

On Tue, Aug 21, 2012 at 1:58 PM, Ivan Brusic ivan@brusic.com wrote:

Reproducible issue: SpanNotQuery · GitHub

On Tue, Aug 21, 2012 at 11:59 AM, Ivan Brusic ivan@brusic.com wrote:

Is anyone using SpanNotQuerys? Judging by a recent issue that was
never uncovered until now, I am assuming not:
Incorrect application of boost in SpanNotQueryBuilder · Issue #2192 · elastic/elasticsearch · GitHub

The exclude portion of my SpanNotQuerys are having no effect on the query.

Given a document { "title": "Handheld Apple iPad 2 Wi Fi tablet iOS 5
64 GB 9.7 black Buy.com" }

These two queries return the same results:

SpanNotQuery
{
"query" : {
"span_not" : {
"include" : {
"span_term" : {
"title" : "ipad"
}
},
"exclude" : {
"span_term" : {
"title" : "black"
}
}
}
}
}

SpanTermQuery
{
"query" : {
"span_term" : {
"TitleString_en" : {
"value" : "ipad",
"boost" : 1.0
}
}
}
}

Explain between the two queries is identical:
9.180814 = fieldWeight(title:spanNot(ipad, black) in 531555), product of:
0.70710677 = tf(phraseFreq=0.5)
7.4192176 = idf(title: ipad=9370)
1.75 = fieldNorm(field=title, doc=531555)

SpanNotQueries in Lucene are working perfectly: spanNot(title:ipad,
title:black). I haven't traced through the code in Elasticsearch, but
the code seems to be creating the correct Lucene class. Anyone
successfully using SpanNotQuerys?

Cheers,

Ivan

--

Hello Ivan,

I know this post is pretty old.
I am definitely puzzled with the gist that you provided.
Why is there 2 matches?
"exclude" : {
"span_term" : {
"field1" : "dog"
}
}
I though we should exclude match with dog...
Could you please point me to proper information to understand what is
happening?

Thx,
Jade

Le mercredi 22 août 2012 02:01:09 UTC-4, Ivan Brusic a écrit :

Slight error on my part. Without even realizing it, I was using a
custom query parser in Lucene that handled SpanNotQueries differently.
The queries work as expected in Elasticsearch, true to the Lucene
standard.

--
Ivan

On Tue, Aug 21, 2012 at 1:58 PM, Ivan Brusic <iv...@brusic.com
<javascript:>> wrote:

Reproducible issue: SpanNotQuery · GitHub

On Tue, Aug 21, 2012 at 11:59 AM, Ivan Brusic <iv...@brusic.com
<javascript:>> wrote:

Is anyone using SpanNotQuerys? Judging by a recent issue that was
never uncovered until now, I am assuming not:
Incorrect application of boost in SpanNotQueryBuilder · Issue #2192 · elastic/elasticsearch · GitHub

The exclude portion of my SpanNotQuerys are having no effect on the
query.

Given a document { "title": "Handheld Apple iPad 2 Wi Fi tablet iOS 5
64 GB 9.7 black Buy.com" }

These two queries return the same results:

SpanNotQuery
{
"query" : {
"span_not" : {
"include" : {
"span_term" : {
"title" : "ipad"
}
},
"exclude" : {
"span_term" : {
"title" : "black"
}
}
}
}
}

SpanTermQuery
{
"query" : {
"span_term" : {
"TitleString_en" : {
"value" : "ipad",
"boost" : 1.0
}
}
}
}

Explain between the two queries is identical:
9.180814 = fieldWeight(title:spanNot(ipad, black) in 531555), product
of:
0.70710677 = tf(phraseFreq=0.5)
7.4192176 = idf(title: ipad=9370)
1.75 = fieldNorm(field=title, doc=531555)

SpanNotQueries in Lucene are working perfectly: spanNot(title:ipad,
title:black). I haven't traced through the code in Elasticsearch, but
the code seems to be creating the correct Lucene class. Anyone
successfully using SpanNotQuerys?

Cheers,

Ivan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0b81d1aa-35e4-468d-8e66-6a8b50029e6c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Pretty old indeed. As explained briefly, I was migrating a Lucene system to
Elasticsearch and did not understand why the span not queries were not
working, only to discover we had a custom parser, to support syntaxes such
as the one you are expecting.

Span nots are tricky in Lucene, but basically you are looking for when
"dog" is not a near "quick" near a "dog". Sounds confusing because it is.

I noticed you +1 the pull request I submitted which will probably help your
use case. The code should work as is without a need to merge in the current
codebase, but I guess it would help. Stuck on a laptop with Java 6, so I
can no longer build Elasticsearch. Will try to find the time.

Cheers,

Ivan

On Thu, Nov 6, 2014 at 10:12 AM, Jade Tremblay jadetremblay.ing@gmail.com
wrote:

Hello Ivan,

I know this post is pretty old.
I am definitely puzzled with the gist that you provided.
Why is there 2 matches?
"exclude" : {
"span_term" : {
"field1" : "dog"
}
}
I though we should exclude match with dog...
Could you please point me to proper information to understand what is
happening?

Thx,
Jade

Le mercredi 22 août 2012 02:01:09 UTC-4, Ivan Brusic a écrit :

Slight error on my part. Without even realizing it, I was using a
custom query parser in Lucene that handled SpanNotQueries differently.
The queries work as expected in Elasticsearch, true to the Lucene
standard.

--
Ivan

On Tue, Aug 21, 2012 at 1:58 PM, Ivan Brusic iv...@brusic.com wrote:

Reproducible issue: SpanNotQuery · GitHub

On Tue, Aug 21, 2012 at 11:59 AM, Ivan Brusic iv...@brusic.com
wrote:

Is anyone using SpanNotQuerys? Judging by a recent issue that was
never uncovered until now, I am assuming not:
Incorrect application of boost in SpanNotQueryBuilder · Issue #2192 · elastic/elasticsearch · GitHub

The exclude portion of my SpanNotQuerys are having no effect on the
query.

Given a document { "title": "Handheld Apple iPad 2 Wi Fi tablet iOS 5
64 GB 9.7 black Buy.com" }

These two queries return the same results:

SpanNotQuery
{
"query" : {
"span_not" : {
"include" : {
"span_term" : {
"title" : "ipad"
}
},
"exclude" : {
"span_term" : {
"title" : "black"
}
}
}
}
}

SpanTermQuery
{
"query" : {
"span_term" : {
"TitleString_en" : {
"value" : "ipad",
"boost" : 1.0
}
}
}
}

Explain between the two queries is identical:
9.180814 = fieldWeight(title:spanNot(ipad, black) in 531555), product
of:
0.70710677 = tf(phraseFreq=0.5)
7.4192176 = idf(title: ipad=9370)
1.75 = fieldNorm(field=title, doc=531555)

SpanNotQueries in Lucene are working perfectly: spanNot(title:ipad,
title:black). I haven't traced through the code in Elasticsearch, but
the code seems to be creating the correct Lucene class. Anyone
successfully using SpanNotQuerys?

Cheers,

Ivan

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0b81d1aa-35e4-468d-8e66-6a8b50029e6c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/0b81d1aa-35e4-468d-8e66-6a8b50029e6c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCbVdBcZxVeDQLSUaPOGScaZMit6soFyteSymaT3y3-1g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

@Ivan

I've pick you git push and integrated it into ElasticSearch source
tag:v1.3.5
After a rebuild, it seems to work perfectly (I am still trying to find the
maximum values for pre and post, no luck so far).

I've been able to figure out how span_not works with this post.
I add it here if someone else is looking to understand why the actual gist
return 2 results.

Thanks for the gist and the push request!

Cheers,
Jade

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/776c24aa-0a11-46ce-8c81-5f78e4c187dc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Yes, that post explained it a lot better than I wanted to. :slight_smile: But basically
yes, the exclude portion is only as part of an existing span, but a single
span term is not really a span.

Ultimately, span queries are not very flexible since they do not analyze
terms, which is why I suspect there are rarely any questions about them
(and perhaps why my PR is in limbo). Phrase matches might work better, but
they do not support in order slop.

--
Ivan

On Thu, Nov 6, 2014 at 2:56 PM, Jade Tremblay jadetremblay.ing@gmail.com
wrote:

@Ivan

I've pick you git push and integrated it into Elasticsearch source
tag:v1.3.5
After a rebuild, it seems to work perfectly (I am still trying to find the
maximum values for pre and post, no luck so far).

I've been able to figure out how span_not works with this post.
I add it here if someone else is looking to understand why the actual gist
return 2 results.

lucene - SpanNotQuery giving unexpected results (exclude is ignored) - Stack Overflow

Thanks for the gist and the push request!

Cheers,
Jade

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/776c24aa-0a11-46ce-8c81-5f78e4c187dc%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/776c24aa-0a11-46ce-8c81-5f78e4c187dc%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAipZAp7fTBTp8PRy_RBORkmTTZJdDipqqeiB3HnUHu3g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.