Different query results when using JDK8 and JDK7

Elastic Stack Elasticsearch
1

I have a weird problem when doing a simple query using different JDK
versions with the same indexed data.

I have 2 docker images based on the official dockerfiles (
http://dockerfile.github.io/#/elasticsearch).
One image has jdk7u67 installed and one has jdk8u20 installed. Both use the
same index data (mounted as a volume) and have the same elasticsearch
installed (1.3.4)
Of course I only run one instance at a time

When I issue following search request I get different results (1 hit with
jdk7 and 5 hits with jdk8):

POST /INDEX/MyType/_search
{
"from": 0,
"size": 50,
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

The “MyType” document has a mapping.
candidategenelist is a nested type:

candidategenelist: {
properties: {
id: {
type: integer
}
name: {
type: string
}

There are a couple of additional nested field types with a similar setup
(id: integer, name: string)

With jdk7u67 I get 1 hit:

Relevant explanation:
description”: “weight(study_.name:flowering in 74274) [PerFieldSimilarity],
result of:”,

With jdk8u20 I get 5 hits:

Relevant explanation:
“description”: “weight(candidategenelist.name:flowering in 74274)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6614)
[PerFieldSimilarity], result of:”,

“description”: “weight(candidategenelist.name:flowering in 56078)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6674)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6646)
[PerFieldSimilarity], result of:”,

So according to the explanation, the only difference is that with JDK7u67
it matches “name” in the nested study_ field whereas with JDK8u20 it
matches the “name” in candidategenelist.

I really don’t know why there are different query results when the only
difference is the JDK version?!
​

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

I would love to reproduce it, do you have some test data available for a
full reproduction?

Thanks,

Jörg

On Fri, Oct 3, 2014 at 4:21 PM, Ümit Seren uemit.seren@gmail.com wrote:

I have a weird problem when doing a simple query using different JDK
versions with the same indexed data.

I have 2 docker images based on the official dockerfiles (
Dockerfile Project - Trusted Automated Docker Builds).
One image has jdk7u67 installed and one has jdk8u20 installed. Both use
the same index data (mounted as a volume) and have the same elasticsearch
installed (1.3.4)
Of course I only run one instance at a time

When I issue following search request I get different results (1 hit with
jdk7 and 5 hits with jdk8):

POST /INDEX/MyType/_search
{
"from": 0,
"size": 50,
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

The “MyType” document has a mapping.
candidategenelist is a nested type:

candidategenelist: {
properties: {
id: {
type: integer
}
name: {
type: string
}

There are a couple of additional nested field types with a similar setup
(id: integer, name: string)

With jdk7u67 I get 1 hit:

Relevant explanation:
description”: “weight(study_.name:flowering in 74274)
[PerFieldSimilarity], result of:”,

With jdk8u20 I get 5 hits:

Relevant explanation:
“description”: “weight(candidategenelist.name:flowering in 74274)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6614)
[PerFieldSimilarity], result of:”,

“description”: “weight(candidategenelist.name:flowering in 56078)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6674)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6646)
[PerFieldSimilarity], result of:”,

So according to the explanation, the only difference is that with JDK7u67
it matches “name” in the nested study_ field whereas with JDK8u20 it
matches the “name” in candidategenelist.

I really don’t know why there are different query results when the only
difference is the JDK version?!
​

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Jörg,
I will try to create some test data ASAP (might take some time as I am not
around next week tough).

cheers
Ümit

On Fri, Oct 3, 2014 at 5:00 PM, joergprante@gmail.com <joergprante@gmail.com

wrote:

I would love to reproduce it, do you have some test data available for a
full reproduction?

Thanks,

Jörg

On Fri, Oct 3, 2014 at 4:21 PM, Ümit Seren uemit.seren@gmail.com wrote:

I have a weird problem when doing a simple query using different JDK
versions with the same indexed data.

I have 2 docker images based on the official dockerfiles (
Dockerfile Project - Trusted Automated Docker Builds).
One image has jdk7u67 installed and one has jdk8u20 installed. Both use
the same index data (mounted as a volume) and have the same elasticsearch
installed (1.3.4)
Of course I only run one instance at a time

When I issue following search request I get different results (1 hit with
jdk7 and 5 hits with jdk8):

POST /INDEX/MyType/_search
{
"from": 0,
"size": 50,
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

The “MyType” document has a mapping.
candidategenelist is a nested type:

candidategenelist: {
properties: {
id: {
type: integer
}
name: {
type: string
}

There are a couple of additional nested field types with a similar setup
(id: integer, name: string)

With jdk7u67 I get 1 hit:

Relevant explanation:
description”: “weight(study_.name:flowering in 74274)
[PerFieldSimilarity], result of:”,

With jdk8u20 I get 5 hits:

Relevant explanation:
“description”: “weight(candidategenelist.name:flowering in 74274)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6614)
[PerFieldSimilarity], result of:”,

“description”: “weight(candidategenelist.name:flowering in 56078)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6674)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6646)
[PerFieldSimilarity], result of:”,

So according to the explanation, the only difference is that with JDK7u67
it matches “name” in the nested study_ field whereas with JDK8u20 it
matches the “name” in candidategenelist.

I really don’t know why there are different query results when the only
difference is the JDK version?!
​

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANBYW4B98yEawQZFGCQJ1eEYkkC5uCf7%3D2x%2Be5dojPDy4WMaqw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Here is the tar.gz folder of the entire elasticsearch data.
let me know if you need anything elese
https://drive.google.com/file/d/0BwYlrRGOfVARbzkwb2FEUmxlRFU/view?usp=sharing

On Fri, Oct 3, 2014 at 5:00 PM, joergprante@gmail.com <joergprante@gmail.com

wrote:

I would love to reproduce it, do you have some test data available for a
full reproduction?

Thanks,

Jörg

On Fri, Oct 3, 2014 at 4:21 PM, Ümit Seren uemit.seren@gmail.com wrote:

I have a weird problem when doing a simple query using different JDK
versions with the same indexed data.

I have 2 docker images based on the official dockerfiles (
Dockerfile Project - Trusted Automated Docker Builds).
One image has jdk7u67 installed and one has jdk8u20 installed. Both use
the same index data (mounted as a volume) and have the same elasticsearch
installed (1.3.4)
Of course I only run one instance at a time

When I issue following search request I get different results (1 hit with
jdk7 and 5 hits with jdk8):

POST /INDEX/MyType/_search
{
"from": 0,
"size": 50,
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

The “MyType” document has a mapping.
candidategenelist is a nested type:

candidategenelist: {
properties: {
id: {
type: integer
}
name: {
type: string
}

There are a couple of additional nested field types with a similar setup
(id: integer, name: string)

With jdk7u67 I get 1 hit:

Relevant explanation:
description”: “weight(study_.name:flowering in 74274)
[PerFieldSimilarity], result of:”,

With jdk8u20 I get 5 hits:

Relevant explanation:
“description”: “weight(candidategenelist.name:flowering in 74274)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6614)
[PerFieldSimilarity], result of:”,

“description”: “weight(candidategenelist.name:flowering in 56078)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6674)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6646)
[PerFieldSimilarity], result of:”,

So according to the explanation, the only difference is that with JDK7u67
it matches “name” in the nested study_ field whereas with JDK8u20 it
matches the “name” in candidategenelist.

I really don’t know why there are different query results when the only
difference is the JDK version?!
​

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

Thanks, I could load the data, but can not find anything.

Your query gave 0 hits but I found this simplified query works:

POST /_search
{
"query" : {
"multi_match" : {
"query" : "Flowering",
"fields": [
"name"
]
}
}
}

JDK 7 --> 1 hit

        "version": "1.7.0_67",
        "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
        "vm_version": "24.65-b04",
        "vm_vendor": "Oracle Corporation"

JDK 8 --> 1 hit

        "version": "1.8.0_20",
        "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
        "vm_version": "25.20-b23",
        "vm_vendor": "Oracle Corporation",

on Mac OS X.

So there must be some other fishy thing going on at your cluster...

Jörg

On Fri, Oct 3, 2014 at 8:19 PM, Ümit Seren uemit.seren@gmail.com wrote:

Here is the tar.gz folder of the entire elasticsearch data.
let me know if you need anything elese

https://drive.google.com/file/d/0BwYlrRGOfVARbzkwb2FEUmxlRFU/view?usp=sharing

On Fri, Oct 3, 2014 at 5:00 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

I would love to reproduce it, do you have some test data available for a
full reproduction?

Thanks,

Jörg

On Fri, Oct 3, 2014 at 4:21 PM, Ümit Seren uemit.seren@gmail.com wrote:

I have a weird problem when doing a simple query using different JDK
versions with the same indexed data.

I have 2 docker images based on the official dockerfiles (
Dockerfile Project - Trusted Automated Docker Builds).
One image has jdk7u67 installed and one has jdk8u20 installed. Both use
the same index data (mounted as a volume) and have the same elasticsearch
installed (1.3.4)
Of course I only run one instance at a time

When I issue following search request I get different results (1 hit
with jdk7 and 5 hits with jdk8):

POST /INDEX/MyType/_search
{
"from": 0,
"size": 50,
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

The “MyType” document has a mapping.
candidategenelist is a nested type:

candidategenelist: {
properties: {
id: {
type: integer
}
name: {
type: string
}

There are a couple of additional nested field types with a similar setup
(id: integer, name: string)

With jdk7u67 I get 1 hit:

Relevant explanation:
description”: “weight(study_.name:flowering in 74274)
[PerFieldSimilarity], result of:”,

With jdk8u20 I get 5 hits:

Relevant explanation:
“description”: “weight(candidategenelist.name:flowering in 74274)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6614)
[PerFieldSimilarity], result of:”,

“description”: “weight(candidategenelist.name:flowering in 56078)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6674)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6646)
[PerFieldSimilarity], result of:”,

So according to the explanation, the only difference is that with
JDK7u67 it matches “name” in the nested study_ field whereas with JDK8u20
it matches the “name” in candidategenelist.

I really don’t know why there are different query results when the only
difference is the JDK version?!
​

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwmxNUuM%3DW8YF7edDAf64vO%2Bco0dy%2BapLm8bT9RR8LJg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

6

Can you try to do the search on
POST /gdpdm/candidate_gene_list_enrichment/_search :

POST /gdpdm/candidate_gene_list_enrichment/_search
{
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

On Fri, Oct 3, 2014 at 9:35 PM, joergprante@gmail.com <joergprante@gmail.com

wrote:

Thanks, I could load the data, but can not find anything.

Your query gave 0 hits but I found this simplified query works:

POST /_search
{
"query" : {
"multi_match" : {
"query" : "Flowering",
"fields": [
"name"
]
}
}
}

JDK 7 --> 1 hit

        "version": "1.7.0_67",
        "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
        "vm_version": "24.65-b04",
        "vm_vendor": "Oracle Corporation"

JDK 8 --> 1 hit

        "version": "1.8.0_20",
        "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
        "vm_version": "25.20-b23",
        "vm_vendor": "Oracle Corporation",

on Mac OS X.

So there must be some other fishy thing going on at your cluster...

Jörg

On Fri, Oct 3, 2014 at 8:19 PM, Ümit Seren uemit.seren@gmail.com wrote:

Here is the tar.gz folder of the entire elasticsearch data.
let me know if you need anything elese

https://drive.google.com/file/d/0BwYlrRGOfVARbzkwb2FEUmxlRFU/view?usp=sharing

On Fri, Oct 3, 2014 at 5:00 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

I would love to reproduce it, do you have some test data available for a
full reproduction?

Thanks,

Jörg

On Fri, Oct 3, 2014 at 4:21 PM, Ümit Seren uemit.seren@gmail.com
wrote:

I have a weird problem when doing a simple query using different JDK
versions with the same indexed data.

I have 2 docker images based on the official dockerfiles (
Dockerfile Project - Trusted Automated Docker Builds).
One image has jdk7u67 installed and one has jdk8u20 installed. Both use
the same index data (mounted as a volume) and have the same elasticsearch
installed (1.3.4)
Of course I only run one instance at a time

When I issue following search request I get different results (1 hit
with jdk7 and 5 hits with jdk8):

POST /INDEX/MyType/_search
{
"from": 0,
"size": 50,
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

The “MyType” document has a mapping.
candidategenelist is a nested type:

candidategenelist: {
properties: {
id: {
type: integer
}
name: {
type: string
}

There are a couple of additional nested field types with a similar
setup (id: integer, name: string)

With jdk7u67 I get 1 hit:

Relevant explanation:
description”: “weight(study_.name:flowering in 74274)
[PerFieldSimilarity], result of:”,

With jdk8u20 I get 5 hits:

Relevant explanation:
“description”: “weight(candidategenelist.name:flowering in 74274)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6614)
[PerFieldSimilarity], result of:”,

“description”: “weight(candidategenelist.name:flowering in 56078)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6674)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6646)
[PerFieldSimilarity], result of:”,

So according to the explanation, the only difference is that with
JDK7u67 it matches “name” in the nested study_ field whereas with JDK8u20
it matches the “name” in candidategenelist.

I really don’t know why there are different query results when the only
difference is the JDK version?!
​

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwmxNUuM%3DW8YF7edDAf64vO%2Bco0dy%2BapLm8bT9RR8LJg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwmxNUuM%3DW8YF7edDAf64vO%2Bco0dy%2BapLm8bT9RR8LJg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANBYW4Dfp%2BgvmwJvWLuFE56D0GSg5NaafFERcQhnpJa8W2qo8Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

7

Yes, I can reproduce it now....

... will investigate. Interesting issue.

Jörg

On Fri, Oct 3, 2014 at 9:48 PM, Ümit Seren uemit.seren@gmail.com wrote:

Can you try to do the search on
POST /gdpdm/candidate_gene_list_enrichment/_search :

POST /gdpdm/candidate_gene_list_enrichment/_search
{
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

On Fri, Oct 3, 2014 at 9:35 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Thanks, I could load the data, but can not find anything.

Your query gave 0 hits but I found this simplified query works:

POST /_search
{
"query" : {
"multi_match" : {
"query" : "Flowering",
"fields": [
"name"
]
}
}
}

JDK 7 --> 1 hit

        "version": "1.7.0_67",
        "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
        "vm_version": "24.65-b04",
        "vm_vendor": "Oracle Corporation"

JDK 8 --> 1 hit

        "version": "1.8.0_20",
        "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
        "vm_version": "25.20-b23",
        "vm_vendor": "Oracle Corporation",

on Mac OS X.

So there must be some other fishy thing going on at your cluster...

Jörg

On Fri, Oct 3, 2014 at 8:19 PM, Ümit Seren uemit.seren@gmail.com wrote:

Here is the tar.gz folder of the entire elasticsearch data.
let me know if you need anything elese

https://drive.google.com/file/d/0BwYlrRGOfVARbzkwb2FEUmxlRFU/view?usp=sharing

On Fri, Oct 3, 2014 at 5:00 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

I would love to reproduce it, do you have some test data available for
a full reproduction?

Thanks,

Jörg

On Fri, Oct 3, 2014 at 4:21 PM, Ümit Seren uemit.seren@gmail.com
wrote:

I have a weird problem when doing a simple query using different JDK
versions with the same indexed data.

I have 2 docker images based on the official dockerfiles (
Dockerfile Project - Trusted Automated Docker Builds).
One image has jdk7u67 installed and one has jdk8u20 installed. Both
use the same index data (mounted as a volume) and have the same
elasticsearch installed (1.3.4)
Of course I only run one instance at a time

When I issue following search request I get different results (1 hit
with jdk7 and 5 hits with jdk8):

POST /INDEX/MyType/_search
{
"from": 0,
"size": 50,
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

The “MyType” document has a mapping.
candidategenelist is a nested type:

candidategenelist: {
properties: {
id: {
type: integer
}
name: {
type: string
}

There are a couple of additional nested field types with a similar
setup (id: integer, name: string)

With jdk7u67 I get 1 hit:

Relevant explanation:
description”: “weight(study_.name:flowering in 74274)
[PerFieldSimilarity], result of:”,

With jdk8u20 I get 5 hits:

Relevant explanation:
“description”: “weight(candidategenelist.name:flowering in 74274)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6614)
[PerFieldSimilarity], result of:”,

“description”: “weight(candidategenelist.name:flowering in 56078)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6674)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6646)
[PerFieldSimilarity], result of:”,

So according to the explanation, the only difference is that with
JDK7u67 it matches “name” in the nested study_ field whereas with JDK8u20
it matches the “name” in candidategenelist.

I really don’t know why there are different query results when the
only difference is the JDK version?!
​

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwmxNUuM%3DW8YF7edDAf64vO%2Bco0dy%2BapLm8bT9RR8LJg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwmxNUuM%3DW8YF7edDAf64vO%2Bco0dy%2BapLm8bT9RR8LJg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANBYW4Dfp%2BgvmwJvWLuFE56D0GSg5NaafFERcQhnpJa8W2qo8Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANBYW4Dfp%2BgvmwJvWLuFE56D0GSg5NaafFERcQhnpJa8W2qo8Q%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGGZYC7qKB9Q_WBDfx4MXeiDW3%3DLtGnHkgWG2X131gcEQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

8

The reason is that field names are stored in a hash where the order of the
keys depend on JVM implementation. I.e. a Java 7 JVM uses other hashes than
a Java 8 JVM.

ES iterates over the field names in a map to find the first matching field
"name" if the field name is not a full path, and you found a case where two
fields are ending with "name" but their key position in the Java map is
different across JVM implementations.

You have two options:

  • never use same field names in ES structured JSON objects

  • always use full path to reference the field you want

Full path example:

POST /gdpdm/candidate_gene_list_enrichment/search
{
"explain": true,
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
},
"query": {
"multi_match": {
"fields": [
"study.name^3.5"
],
"query": "Flowering"
}
}
}
}
}

But in all cases, this is a bug in ES, and I suggest to open an issue.

Jörg

On Fri, Oct 3, 2014 at 10:04 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Yes, I can reproduce it now....

... will investigate. Interesting issue.

Jörg

On Fri, Oct 3, 2014 at 9:48 PM, Ümit Seren uemit.seren@gmail.com wrote:

Can you try to do the search on
POST /gdpdm/candidate_gene_list_enrichment/_search :

POST /gdpdm/candidate_gene_list_enrichment/_search
{
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

On Fri, Oct 3, 2014 at 9:35 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Thanks, I could load the data, but can not find anything.

Your query gave 0 hits but I found this simplified query works:

POST /_search
{
"query" : {
"multi_match" : {
"query" : "Flowering",
"fields": [
"name"
]
}
}
}

JDK 7 --> 1 hit

        "version": "1.7.0_67",
        "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
        "vm_version": "24.65-b04",
        "vm_vendor": "Oracle Corporation"

JDK 8 --> 1 hit

        "version": "1.8.0_20",
        "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
        "vm_version": "25.20-b23",
        "vm_vendor": "Oracle Corporation",

on Mac OS X.

So there must be some other fishy thing going on at your cluster...

Jörg

On Fri, Oct 3, 2014 at 8:19 PM, Ümit Seren uemit.seren@gmail.com
wrote:

Here is the tar.gz folder of the entire elasticsearch data.
let me know if you need anything elese

https://drive.google.com/file/d/0BwYlrRGOfVARbzkwb2FEUmxlRFU/view?usp=sharing

On Fri, Oct 3, 2014 at 5:00 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

I would love to reproduce it, do you have some test data available for
a full reproduction?

Thanks,

Jörg

On Fri, Oct 3, 2014 at 4:21 PM, Ümit Seren uemit.seren@gmail.com
wrote:

I have a weird problem when doing a simple query using different JDK
versions with the same indexed data.

I have 2 docker images based on the official dockerfiles (
Dockerfile Project - Trusted Automated Docker Builds).
One image has jdk7u67 installed and one has jdk8u20 installed. Both
use the same index data (mounted as a volume) and have the same
elasticsearch installed (1.3.4)
Of course I only run one instance at a time

When I issue following search request I get different results (1 hit
with jdk7 and 5 hits with jdk8):

POST /INDEX/MyType/_search
{
"from": 0,
"size": 50,
"query": {
"multi_match": {
"query": "Flowering",
"fields": [
"name^3.5"
]
}
},
"post_filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
}
}

The “MyType” document has a mapping.
candidategenelist is a nested type:

candidategenelist: {
properties: {
id: {
type: integer
}
name: {
type: string
}

There are a couple of additional nested field types with a similar
setup (id: integer, name: string)

With jdk7u67 I get 1 hit:

Relevant explanation:
description”: “weight(study_.name:flowering in 74274)
[PerFieldSimilarity], result of:”,

With jdk8u20 I get 5 hits:

Relevant explanation:
“description”: “weight(candidategenelist.name:flowering in 74274)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6614)
[PerFieldSimilarity], result of:”,

“description”: “weight(candidategenelist.name:flowering in 56078)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6674)
[PerFieldSimilarity], result of:”,
“description”: “weight(candidategenelist.name:flowering in 6646)
[PerFieldSimilarity], result of:”,

So according to the explanation, the only difference is that with
JDK7u67 it matches “name” in the nested study_ field whereas with JDK8u20
it matches the “name” in candidategenelist.

I really don’t know why there are different query results when the
only difference is the JDK version?!
​

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwmxNUuM%3DW8YF7edDAf64vO%2Bco0dy%2BapLm8bT9RR8LJg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwmxNUuM%3DW8YF7edDAf64vO%2Bco0dy%2BapLm8bT9RR8LJg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANBYW4Dfp%2BgvmwJvWLuFE56D0GSg5NaafFERcQhnpJa8W2qo8Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANBYW4Dfp%2BgvmwJvWLuFE56D0GSg5NaafFERcQhnpJa8W2qo8Q%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGz%3DCvR7EtyOaBvmrOnStLVCEkz6M8RsgxOn1tz6iZ%3DuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

9

On Fri, Oct 3, 2014 at 10:19 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

But in all cases, this is a bug in ES, and I suggest to open an issue.

+1 it's bad that the results that you get depend on the iteration order of
a hash table. Thanks for investigating the issue Jörg!

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j4eN661JUZTRfuJx9eBkxqAsQ6weFxefOYVgMVZ4A%2B3%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

10

Created issue:

On Saturday, October 4, 2014 12:21:24 AM UTC+2, Adrien Grand wrote:

On Fri, Oct 3, 2014 at 10:19 PM, joerg...@gmail.com <javascript:> <
joerg...@gmail.com <javascript:>> wrote:

But in all cases, this is a bug in ES, and I suggest to open an issue.

+1 it's bad that the results that you get depend on the iteration order of
a hash table. Thanks for investigating the issue Jörg!

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/44b81f7d-18ce-40c9-8fc0-425f0aa7e11c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.