Has_parent filter acting strange

I have two documents, both (should) have a parent (different parents, but
the same type).

However, I don't get the result I expect - especially trying to return
documents without a parent.

For example:

{
"fields":["_parent"],
"query": {
"filtered": {
"query": {
"query_string": {
"query": "kittens*"
}
},
"filter": {
"bool": {
"must_not": [
{
"has_parent": {
"parent_type": "psa",
"query": {
"match_all": {}
}
}
}
]
}
}
}
}
}

I run that, and I get this back:

{
"took": 287,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9104",
"_score": 1,
"fields": {
"_parent": "6_905"
}
}
]
}
}

Clearly it DOES have a parent because it shows in the results!

If I change the initial query from "must_not" to "must", I get the other
document in the result:

{
"took": 75,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9106",
"_score": 1,
"fields": {
"_parent": "6_847"
}
}
]
}
}

One query should return zero results, and the other should return 2.
However each are returning 1.

Am I being thick, or is something amiss here - and how can I troubleshoot
it?

Tom.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c8beeddf-2cfb-4af7-8836-6779baf85933%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Can you share a reproduction of the mismatches you're experiencing? (in a
gist or something like that, so that I can see on what data you're running
these queries)

The has_parent query in a must clause should return all child documents
with a parent and a has_parent in must_not should return all child docs
with no parent.

Also on what ES version are you running this?

On 2 October 2014 20:27, Tom Miller tom.miller@ebiz.co.uk wrote:

I have two documents, both (should) have a parent (different parents, but
the same type).

However, I don't get the result I expect - especially trying to return
documents without a parent.

For example:

{
"fields":["_parent"],
"query": {
"filtered": {
"query": {
"query_string": {
"query": "kittens*"
}
},
"filter": {
"bool": {
"must_not": [
{
"has_parent": {
"parent_type": "psa",
"query": {
"match_all": {}
}
}
}
]
}
}
}
}
}

I run that, and I get this back:

{
"took": 287,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9104",
"_score": 1,
"fields": {
"_parent": "6_905"
}
}
]
}
}

Clearly it DOES have a parent because it shows in the results!

If I change the initial query from "must_not" to "must", I get the other
document in the result:

{
"took": 75,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9106",
"_score": 1,
"fields": {
"_parent": "6_847"
}
}
]
}
}

One query should return zero results, and the other should return 2.
However each are returning 1.

Am I being thick, or is something amiss here - and how can I troubleshoot
it?

Tom.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c8beeddf-2cfb-4af7-8836-6779baf85933%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c8beeddf-2cfb-4af7-8836-6779baf85933%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TwRGkJWWRLeV_Hv4kV1U3-zWN12pVuLsspmQRH3PSM7Kw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Martin,

I'm running this on v1.3.

The mapping for "document" only has one parent type: "psa", so the fact
both documents have a parent means they must be the same type. I'm using
using routing, so the parents and children are on the same shard.

I'm not sure exactly what extra info I can give? My data is huge...

I'll try fiddling a bit more to see if I can replicate it further.

I guess the question is, how can it return a result which clearly contains
a "_parent" field, when I'm doing a "must_not" "has_parent" filter? That
should never be possible.

On 2 October 2014 20:54, Martijn v Groningen martijn.v.groningen@gmail.com
wrote:

Can you share a reproduction of the mismatches you're experiencing? (in a
gist or something like that, so that I can see on what data you're running
these queries)

The has_parent query in a must clause should return all child documents
with a parent and a has_parent in must_not should return all child docs
with no parent.

Also on what ES version are you running this?

On 2 October 2014 20:27, Tom Miller tom.miller@ebiz.co.uk wrote:

I have two documents, both (should) have a parent (different parents, but
the same type).

However, I don't get the result I expect - especially trying to return
documents without a parent.

For example:

{
"fields":["_parent"],
"query": {
"filtered": {
"query": {
"query_string": {
"query": "kittens*"
}
},
"filter": {
"bool": {
"must_not": [
{
"has_parent": {
"parent_type": "psa",
"query": {
"match_all": {}
}
}
}
]
}
}
}
}
}

I run that, and I get this back:

{
"took": 287,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9104",
"_score": 1,
"fields": {
"_parent": "6_905"
}
}
]
}
}

Clearly it DOES have a parent because it shows in the results!

If I change the initial query from "must_not" to "must", I get the other
document in the result:

{
"took": 75,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9106",
"_score": 1,
"fields": {
"_parent": "6_847"
}
}
]
}
}

One query should return zero results, and the other should return 2.
However each are returning 1.

Am I being thick, or is something amiss here - and how can I troubleshoot
it?

Tom.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c8beeddf-2cfb-4af7-8836-6779baf85933%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c8beeddf-2cfb-4af7-8836-6779baf85933%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/N8eKOCz2vdM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TwRGkJWWRLeV_Hv4kV1U3-zWN12pVuLsspmQRH3PSM7Kw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TwRGkJWWRLeV_Hv4kV1U3-zWN12pVuLsspmQRH3PSM7Kw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--

Tom Miller

Managing Director

=======================================================

eBiz

Centurion House, London Road, Staines, TW18 4AX

Tel: 08448 045046

email: tom.miller@ebiz. tom.miller@ebiz.ukuk tom.miller@ebiz.co.uk |
web: ebiz.uk | eBiz | Staines | www.twitter.com/eBizUK

Information contained in this communication may be confidential and/or
legally privileged. It is intended solely for the use of the addressee and
others authorized to receive it. If you are not the intended recipient any
disclosure, copying, distribution or action taken in reliance on its
contents is prohibited and may be unlawful. Neither eBiz | interactive
business solutions nor the author accepts legal liability for the contents
of this message. If you receive this communication in error please advise
us at privacy@ebiz.co.uk

Whilst we run anti-virus software we are not liable for any loss/damage
sustained as a result of software viruses. The recipients are advised to
run their own anti-virus software

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAM9sqB2yMh8_FTmYmaUPb7dkbYJ8w9B2iXAa1UVySn7h4AEe6Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

A child document can point to an id of a parent document that doesn't exist
(yet). So a child document that just has a _parent field can be returned if
you wrap a has_parent in a must_not clause.

If you try to perform this request:
GET /[your-index]/psa/6_905

then I expect no parent document to be returned.

On 2 October 2014 22:14, Tom Miller tom.miller@ebiz.co.uk wrote:

Martin,

I'm running this on v1.3.

The mapping for "document" only has one parent type: "psa", so the fact
both documents have a parent means they must be the same type. I'm using
using routing, so the parents and children are on the same shard.

I'm not sure exactly what extra info I can give? My data is huge...

I'll try fiddling a bit more to see if I can replicate it further.

I guess the question is, how can it return a result which clearly contains
a "_parent" field, when I'm doing a "must_not" "has_parent" filter? That
should never be possible.

On 2 October 2014 20:54, Martijn v Groningen <
martijn.v.groningen@gmail.com> wrote:

Can you share a reproduction of the mismatches you're experiencing? (in a
gist or something like that, so that I can see on what data you're running
these queries)

The has_parent query in a must clause should return all child documents
with a parent and a has_parent in must_not should return all child docs
with no parent.

Also on what ES version are you running this?

On 2 October 2014 20:27, Tom Miller tom.miller@ebiz.co.uk wrote:

I have two documents, both (should) have a parent (different parents,
but the same type).

However, I don't get the result I expect - especially trying to return
documents without a parent.

For example:

{
"fields":["_parent"],
"query": {
"filtered": {
"query": {
"query_string": {
"query": "kittens*"
}
},
"filter": {
"bool": {
"must_not": [
{
"has_parent": {
"parent_type": "psa",
"query": {
"match_all": {}
}
}
}
]
}
}
}
}
}

I run that, and I get this back:

{
"took": 287,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9104",
"_score": 1,
"fields": {
"_parent": "6_905"
}
}
]
}
}

Clearly it DOES have a parent because it shows in the results!

If I change the initial query from "must_not" to "must", I get the other
document in the result:

{
"took": 75,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9106",
"_score": 1,
"fields": {
"_parent": "6_847"
}
}
]
}
}

One query should return zero results, and the other should return 2.
However each are returning 1.

Am I being thick, or is something amiss here - and how can I
troubleshoot it?

Tom.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c8beeddf-2cfb-4af7-8836-6779baf85933%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c8beeddf-2cfb-4af7-8836-6779baf85933%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/N8eKOCz2vdM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TwRGkJWWRLeV_Hv4kV1U3-zWN12pVuLsspmQRH3PSM7Kw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TwRGkJWWRLeV_Hv4kV1U3-zWN12pVuLsspmQRH3PSM7Kw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--

Tom Miller

Managing Director

=======================================================

eBiz

Centurion House, London Road, Staines, TW18 4AX

Tel: 08448 045046

email: tom.miller@ebiz. tom.miller@ebiz.ukuk tom.miller@ebiz.co.uk |
web: ebiz.uk | eBiz | Staines | www.twitter.com/eBizUK

Information contained in this communication may be confidential and/or
legally privileged. It is intended solely for the use of the addressee and
others authorized to receive it. If you are not the intended recipient any
disclosure, copying, distribution or action taken in reliance on its
contents is prohibited and may be unlawful. Neither eBiz | interactive
business solutions nor the author accepts legal liability for the contents
of this message. If you receive this communication in error please advise
us at privacy@ebiz.co.uk

Whilst we run anti-virus software we are not liable for any loss/damage
sustained as a result of software viruses. The recipients are advised to
run their own anti-virus software

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAM9sqB2yMh8_FTmYmaUPb7dkbYJ8w9B2iXAa1UVySn7h4AEe6Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAM9sqB2yMh8_FTmYmaUPb7dkbYJ8w9B2iXAa1UVySn7h4AEe6Q%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TyAWtroBYH0NCgHE9n0fdWrd_wta_dR2b_cFiPCDi7dzw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Martin,

Wow I feel like an idiot! You're totally right!

Doh!

Thanks,

Tom.

On 2 October 2014 21:34, Martijn v Groningen martijn.v.groningen@gmail.com
wrote:

A child document can point to an id of a parent document that doesn't
exist (yet). So a child document that just has a _parent field can be
returned if you wrap a has_parent in a must_not clause.

If you try to perform this request:
GET /[your-index]/psa/6_905

then I expect no parent document to be returned.

On 2 October 2014 22:14, Tom Miller tom.miller@ebiz.co.uk wrote:

Martin,

I'm running this on v1.3.

The mapping for "document" only has one parent type: "psa", so the fact
both documents have a parent means they must be the same type. I'm using
using routing, so the parents and children are on the same shard.

I'm not sure exactly what extra info I can give? My data is huge...

I'll try fiddling a bit more to see if I can replicate it further.

I guess the question is, how can it return a result which clearly
contains a "_parent" field, when I'm doing a "must_not" "has_parent"
filter? That should never be possible.

On 2 October 2014 20:54, Martijn v Groningen <
martijn.v.groningen@gmail.com> wrote:

Can you share a reproduction of the mismatches you're experiencing? (in
a gist or something like that, so that I can see on what data you're
running these queries)

The has_parent query in a must clause should return all child documents
with a parent and a has_parent in must_not should return all child docs
with no parent.

Also on what ES version are you running this?

On 2 October 2014 20:27, Tom Miller tom.miller@ebiz.co.uk wrote:

I have two documents, both (should) have a parent (different parents,
but the same type).

However, I don't get the result I expect - especially trying to return
documents without a parent.

For example:

{
"fields":["_parent"],
"query": {
"filtered": {
"query": {
"query_string": {
"query": "kittens*"
}
},
"filter": {
"bool": {
"must_not": [
{
"has_parent": {
"parent_type": "psa",
"query": {
"match_all": {}
}
}
}
]
}
}
}
}
}

I run that, and I get this back:

{
"took": 287,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9104",
"_score": 1,
"fields": {
"_parent": "6_905"
}
}
]
}
}

Clearly it DOES have a parent because it shows in the results!

If I change the initial query from "must_not" to "must", I get the
other document in the result:

{
"took": 75,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "egroup",
"_type": "document",
"_id": "6_9106",
"_score": 1,
"fields": {
"_parent": "6_847"
}
}
]
}
}

One query should return zero results, and the other should return 2.
However each are returning 1.

Am I being thick, or is something amiss here - and how can I
troubleshoot it?

Tom.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c8beeddf-2cfb-4af7-8836-6779baf85933%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c8beeddf-2cfb-4af7-8836-6779baf85933%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/N8eKOCz2vdM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TwRGkJWWRLeV_Hv4kV1U3-zWN12pVuLsspmQRH3PSM7Kw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TwRGkJWWRLeV_Hv4kV1U3-zWN12pVuLsspmQRH3PSM7Kw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--

Tom Miller

Managing Director

=======================================================

eBiz

Centurion House, London Road, Staines, TW18 4AX

Tel: 08448 045046

email: tom.miller@ebiz. tom.miller@ebiz.ukuk tom.miller@ebiz.co.uk |
web: ebiz.uk | eBiz | Staines | www.twitter.com/eBizUK

Information contained in this communication may be confidential and/or
legally privileged. It is intended solely for the use of the addressee and
others authorized to receive it. If you are not the intended recipient any
disclosure, copying, distribution or action taken in reliance on its
contents is prohibited and may be unlawful. Neither eBiz | interactive
business solutions nor the author accepts legal liability for the contents
of this message. If you receive this communication in error please advise
us at privacy@ebiz.co.uk

Whilst we run anti-virus software we are not liable for any loss/damage
sustained as a result of software viruses. The recipients are advised to
run their own anti-virus software

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAM9sqB2yMh8_FTmYmaUPb7dkbYJ8w9B2iXAa1UVySn7h4AEe6Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAM9sqB2yMh8_FTmYmaUPb7dkbYJ8w9B2iXAa1UVySn7h4AEe6Q%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/N8eKOCz2vdM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TyAWtroBYH0NCgHE9n0fdWrd_wta_dR2b_cFiPCDi7dzw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TyAWtroBYH0NCgHE9n0fdWrd_wta_dR2b_cFiPCDi7dzw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--

Tom Miller

Managing Director

=======================================================

eBiz

Centurion House, London Road, Staines, TW18 4AX

Tel: 08448 045046

email: tom.miller@ebiz. tom.miller@ebiz.ukuk tom.miller@ebiz.co.uk |
web: ebiz.uk | eBiz | Staines | www.twitter.com/eBizUK

Information contained in this communication may be confidential and/or
legally privileged. It is intended solely for the use of the addressee and
others authorized to receive it. If you are not the intended recipient any
disclosure, copying, distribution or action taken in reliance on its
contents is prohibited and may be unlawful. Neither eBiz | interactive
business solutions nor the author accepts legal liability for the contents
of this message. If you receive this communication in error please advise
us at privacy@ebiz.co.uk

Whilst we run anti-virus software we are not liable for any loss/damage
sustained as a result of software viruses. The recipients are advised to
run their own anti-virus software

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAM9sqB0xwxaRPZOJ86D73w4XeF%2B1Fqd2zc7K%2BLxqTb%2B-vqq6DA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.