Empty field in native script

Elastic Stack Elasticsearch
1

I have created native script that checks if a specific field (mapped to
type long) in the document is empty, using three different options below,
the two first options sometime return false.
Is there a documentation on the semantics of each option?

From what I see, the first two options use
PackedArrayIndexFieldData.loadDirect(AtomicReaderContext context) which
gets the terms from a Lucene reader (lucene.index.SegmentReader) and since
they're not null, the field value isn't considered empty.

I'm using ElasticSearch 0.90.5 and java 1.7u17.

public class MapScript extends AbstractSearchScript {
@Override
public Object run() {
if (!doc().containsKey("field2")) //sometime false
return new Object();

    if (doc().get("field2") == ScriptDocValues.EMPTY) //sometime false
        return new Object();

    ScriptDocValues value = (ScriptDocValues) doc().get("field2");
    if (value.isEmpty()) //always true
        return new Object();
        
    return new Object();
}

}

Here’s the data for this script:
curl -XPOST localhost:9200/documents/ -d '{ "mappings" : { "type1" :
{ "properties" : { "field1" : { "type" : "long" }
} }}}'
curl -XPUT localhost:9200/documents/type1/1 -d '{ "field1" : 7 }'
curl -XPUT localhost:9200/documents/type1/_mapping -d '{ "type1" :
{ "properties" : { "field2" : {"type" : "long"}
} }}'
curl -XPUT localhost:9200/documents/type1/2 -d '{ "field1" : 77 }'
curl -XPUT localhost:9200/documents/type1/3 -d '{ "field1" : 3, "field2" :
33 }'

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5f77fa65-9f40-4bb2-98ac-9834e7362eb0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

The internals aren't really documented.

But what are you trying to do? are you familiar with Missing Filter?

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Author of RavenDB in Action http://manning.com/synhershko/

On Mon, Mar 24, 2014 at 11:36 AM, Meidan meidan.alon@gmail.com wrote:

I have created native script that checks if a specific field (mapped to
type long) in the document is empty, using three different options below,
the two first options sometime return false.
Is there a documentation on the semantics of each option?

From what I see, the first two options use
PackedArrayIndexFieldData.loadDirect(AtomicReaderContext context) which
gets the terms from a Lucene reader (lucene.index.SegmentReader) and since
they're not null, the field value isn't considered empty.

I'm using Elasticsearch 0.90.5 and java 1.7u17.

public class MapScript extends AbstractSearchScript {
@Override
public Object run() {
if (!doc().containsKey("field2")) //sometime false
return new Object();

    if (doc().get("field2") == ScriptDocValues.EMPTY) //sometime false
        return new Object();

    ScriptDocValues value = (ScriptDocValues) doc().get("field2");
    if (value.isEmpty()) //always true
        return new Object();

    return new Object();
}

}

Here's the data for this script:
curl -XPOST localhost:9200/documents/ -d '{ "mappings" : { "type1" :
{ "properties" : { "field1" : { "type" : "long" }
} }}}'
curl -XPUT localhost:9200/documents/type1/1 -d '{ "field1" : 7 }'
curl -XPUT localhost:9200/documents/type1/_mapping -d '{ "type1" :
{ "properties" : { "field2" : {"type" : "long"}
} }}'
curl -XPUT localhost:9200/documents/type1/2 -d '{ "field1" : 77 }'
curl -XPUT localhost:9200/documents/type1/3 -d '{ "field1" : 3, "field2" :
33 }'

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5f77fa65-9f40-4bb2-98ac-9834e7362eb0%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/5f77fa65-9f40-4bb2-98ac-9834e7362eb0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZsjJW5x1C-BwatL1ETOn%2Bf5HJ8%3DBM1MM6kt2AKQnQSj8w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Sure, I want the exact same behavior as the missing filter, but in a native
script.
This is just a part of a much larger script I have.

On Mon, Mar 24, 2014 at 11:41 AM, Itamar Syn-Hershko itamar@code972.comwrote:

The internals aren't really documented.

But what are you trying to do? are you familiar with Missing Filter?
Elasticsearch Platform — Find real-time answers at scale | Elastic

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Author of RavenDB in Action http://manning.com/synhershko/

On Mon, Mar 24, 2014 at 11:36 AM, Meidan meidan.alon@gmail.com wrote:

I have created native script that checks if a specific field (mapped to
type long) in the document is empty, using three different options below,
the two first options sometime return false.
Is there a documentation on the semantics of each option?

From what I see, the first two options use
PackedArrayIndexFieldData.loadDirect(AtomicReaderContext context) which
gets the terms from a Lucene reader (lucene.index.SegmentReader) and since
they're not null, the field value isn't considered empty.

I'm using Elasticsearch 0.90.5 and java 1.7u17.

public class MapScript extends AbstractSearchScript {
@Override
public Object run() {
if (!doc().containsKey("field2")) //sometime false
return new Object();

    if (doc().get("field2") == ScriptDocValues.EMPTY) //sometime false
        return new Object();

    ScriptDocValues value = (ScriptDocValues) doc().get("field2");
    if (value.isEmpty()) //always true
        return new Object();

    return new Object();
}

}

Here's the data for this script:
curl -XPOST localhost:9200/documents/ -d '{ "mappings" : { "type1" :
{ "properties" : { "field1" : { "type" : "long" }
} }}}'
curl -XPUT localhost:9200/documents/type1/1 -d '{ "field1" : 7 }'
curl -XPUT localhost:9200/documents/type1/_mapping -d '{ "type1" :
{ "properties" : { "field2" : {"type" : "long"}
} }}'
curl -XPUT localhost:9200/documents/type1/2 -d '{ "field1" : 77 }'
curl -XPUT localhost:9200/documents/type1/3 -d '{ "field1" : 3, "field2"
: 33 }'

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5f77fa65-9f40-4bb2-98ac-9834e7362eb0%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/5f77fa65-9f40-4bb2-98ac-9834e7362eb0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/gSlS02Dw2ss/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZsjJW5x1C-BwatL1ETOn%2Bf5HJ8%3DBM1MM6kt2AKQnQSj8w%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAHTr4ZsjJW5x1C-BwatL1ETOn%2Bf5HJ8%3DBM1MM6kt2AKQnQSj8w%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Meidan.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHHFEKny4AkKqAjmg97sc%3DLBxdFwk5N3jbviSTkHLOV%3DLE1Emw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Can you try something like this for each field you want to test?

        if (doc().containsKey(field)) {
              ScriptDocValues fieldValue = (ScriptDocValues)

doc().get(field);
if (fieldValue != null && !fieldValue.isEmpty()) {
//not empty!
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/26c1f284-8a62-4449-b88f-f50e54fbbe57%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

Binh, I did try that.
As I wrote in my original message, I just want to know the differences
between the 3 variants.

On Mon, Mar 24, 2014 at 5:58 PM, Binh Ly binhly_es@yahoo.com wrote:

Can you try something like this for each field you want to test?

        if (doc().containsKey(field)) {
              ScriptDocValues fieldValue = (ScriptDocValues)

doc().get(field);
if (fieldValue != null && !fieldValue.isEmpty()) {
//not empty!
}
}

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/gSlS02Dw2ss/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/26c1f284-8a62-4449-b88f-f50e54fbbe57%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/26c1f284-8a62-4449-b88f-f50e54fbbe57%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Meidan.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHHFEKkT_uKQgzT_8eHb7cBwVBPEtp6Rc2_1iWcgk4D7Xm9h_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.