Possible JVM issue with "missing" filter


(Les Barstow) #1

(Apologies in advance for including sample code - having issues with gist
at the second...)

When running some tests against ElasticSearch 1.3.2, we noticed what
appears to be a problem with the "missing" filter. This problem manifests
with JVM 1.7.0_60 (recommended in the ES documentation), but disappears
under JVM 1.7.0_67.

If we create a simple template with a single property designating a string
field:

{
"template": "empty_ex*",
"settings": { "number_of_shards": 1 },

"mappings": {
    "empty_example": {
       "_source": { "enabled" : true },
        "properties": {
            "testField": { "type": "string" }
        }
    }
}

}

And then enter a simple document:

curl -XPUT http://localhost:9200/empty_ex1/empty_example/1 -d
'{"testField":""}'

Then the following simple filter fails to retrieve the document in JVM
1.7.0_60, but retrieves the document properly in 1.7.0_67.

{
"filter": {
"missing": {
"field": "testField"
}
}
}

Simply switching out the JVM changes the query results on existing indexes,
so this appears to be a query-side issue only. Adding "null_value": true
does not return any hits under JVM _60 either...

Can someone confirm this?

--
Les Barstow, Senior Software Engineer
Return Path, Inc.
The Global Leader In Email Intelligence

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5a68b02e-fc07-477c-9999-a1e26e310802%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

I can not reproduce this on Mac OS X.

The behavior of "missing"/"exists" filter for empty fields has slightly
changed, see https://github.com/elasticsearch/elasticsearch/issues/5659

Jörg

On Tue, Sep 9, 2014 at 11:18 PM, Les Barstow les.barstow@returnpath.com
wrote:

(Apologies in advance for including sample code - having issues with gist
at the second...)

When running some tests against ElasticSearch 1.3.2, we noticed what
appears to be a problem with the "missing" filter. This problem manifests
with JVM 1.7.0_60 (recommended in the ES documentation), but disappears
under JVM 1.7.0_67.

If we create a simple template with a single property designating a string
field:

{
"template": "empty_ex*",
"settings": { "number_of_shards": 1 },

"mappings": {
    "empty_example": {
       "_source": { "enabled" : true },
        "properties": {
            "testField": { "type": "string" }
        }
    }
}

}

And then enter a simple document:

curl -XPUT http://localhost:9200/empty_ex1/empty_example/1 -d
'{"testField":""}'

Then the following simple filter fails to retrieve the document in JVM
1.7.0_60, but retrieves the document properly in 1.7.0_67.

{
"filter": {
"missing": {
"field": "testField"
}
}
}

Simply switching out the JVM changes the query results on existing
indexes, so this appears to be a query-side issue only. Adding
"null_value": true does not return any hits under JVM _60 either...

Can someone confirm this?

--
Les Barstow, Senior Software Engineer
Return Path, Inc.
The Global Leader In Email Intelligence

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5a68b02e-fc07-477c-9999-a1e26e310802%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5a68b02e-fc07-477c-9999-a1e26e310802%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKixVmFBJ6%3DGqH6oxLOnnDBZsXRGd%3D_eM3VBHw2XMLjA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Les Barstow) #3

Thanks. https://github.com/elasticsearch/elasticsearch/issues/7348 is a
recently opened bug that addresses my problem. Don't know why 1.7.0_67
"fixes" it on my Linux box (and others who have replicated it...)

--
Les

On Tue, Sep 9, 2014 at 4:43 PM, joergprante@gmail.com <joergprante@gmail.com

wrote:

I can not reproduce this on Mac OS X.

The behavior of "missing"/"exists" filter for empty fields has slightly
changed, see https://github.com/elasticsearch/elasticsearch/issues/5659

Jörg

On Tue, Sep 9, 2014 at 11:18 PM, Les Barstow les.barstow@returnpath.com
wrote:

(Apologies in advance for including sample code - having issues with gist
at the second...)

When running some tests against ElasticSearch 1.3.2, we noticed what
appears to be a problem with the "missing" filter. This problem manifests
with JVM 1.7.0_60 (recommended in the ES documentation), but disappears
under JVM 1.7.0_67.

If we create a simple template with a single property designating a
string field:

{
"template": "empty_ex*",
"settings": { "number_of_shards": 1 },

"mappings": {
    "empty_example": {
       "_source": { "enabled" : true },
        "properties": {
            "testField": { "type": "string" }
        }
    }
}

}

And then enter a simple document:

curl -XPUT http://localhost:9200/empty_ex1/empty_example/1 -d
'{"testField":""}'

Then the following simple filter fails to retrieve the document in JVM
1.7.0_60, but retrieves the document properly in 1.7.0_67.

{
"filter": {
"missing": {
"field": "testField"
}
}
}

Simply switching out the JVM changes the query results on existing
indexes, so this appears to be a query-side issue only. Adding
"null_value": true does not return any hits under JVM _60 either...

Can someone confirm this?

--
Les Barstow, Senior Software Engineer
Return Path, Inc.
The Global Leader In Email Intelligence

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5a68b02e-fc07-477c-9999-a1e26e310802%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5a68b02e-fc07-477c-9999-a1e26e310802%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKixVmFBJ6%3DGqH6oxLOnnDBZsXRGd%3D_eM3VBHw2XMLjA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKixVmFBJ6%3DGqH6oxLOnnDBZsXRGd%3D_eM3VBHw2XMLjA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOppbCXffaNxNh91tPMzvEqmyCe563_LGAhLW4K%2BrarCOmnCvg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #4