How to response null value fields when query using "fields"

Elastic Stack Elasticsearch
1

I used "fields" to limit only return selected fields:
{
"fields" : ["Title", "Price"],
"query" :{
...
}
}

The Price fields might be null for some of the documents, I found that only
when "Price" field value is NOT null, it is returned in ES response,
otherwise, it is missing in response. But if I remove "fields" from the
query, the null value field will appear in response.

Is it by design of Elasticsearch? How to return the null value fields in
response even using "fields" in query?

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3c91fd31-0e13-4afb-a9bf-1a0ec795e68f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hi

If you have a field that has null as a value then as far as the inverted
index is concerned for that document it doesn''t exist

Have a look at this

On 26 May 2015 at 08:11, Xudong You xudong.you@gmail.com wrote:

I used "fields" to limit only return selected fields:
{
"fields" : ["Title", "Price"],
"query" :{
...
}
}

The Price fields might be null for some of the documents, I found that
only when "Price" field value is NOT null, it is returned in ES response,
otherwise, it is missing in response. But if I remove "fields" from the
query, the null value field will appear in response.

Is it by design of Elasticsearch? How to return the null value fields in
response even using "fields" in query?

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3c91fd31-0e13-4afb-a9bf-1a0ec795e68f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/3c91fd31-0e13-4afb-a9bf-1a0ec795e68f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAECdJzD0GgOE2wz2uoZ974AZtP4BbB%2BK%3DOUbbz5HTQ8rCKh9oA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks your reply!
But why if I remove "fields" from query DSL, then I can see those null
value fields in ES response?

On Tuesday, May 26, 2015 at 4:19:30 PM UTC+8, Allan Mitchell wrote:

Hi

If you have a field that has null as a value then as far as the inverted
index is concerned for that document it doesn''t exist

Have a look at this

Dealing with Null Values | Elasticsearch: The Definitive Guide [2.x] | Elastic

On 26 May 2015 at 08:11, Xudong You <xudon...@gmail.com <javascript:>>
wrote:

I used "fields" to limit only return selected fields:
{
"fields" : ["Title", "Price"],
"query" :{
...
}
}

The Price fields might be null for some of the documents, I found that
only when "Price" field value is NOT null, it is returned in ES response,
otherwise, it is missing in response. But if I remove "fields" from the
query, the null value field will appear in response.

Is it by design of Elasticsearch? How to return the null value fields in
response even using "fields" in query?

--
Please update your bookmarks! We have moved to
https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3c91fd31-0e13-4afb-a9bf-1a0ec795e68f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/3c91fd31-0e13-4afb-a9bf-1a0ec795e68f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/71e4219a-e9a6-4fa0-a289-9bf3fb6e50a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Hi

Here is a quick example (pinched from the article I sent you and added to)

You can see that we get the values from two different places. Whilst the
field exists at the _all level it does not exist in the inverted index.

POST /my_index/posts/_bulk
{ "index": { "_id": "1" }}
{ "tags" : ["search"] }
{ "index": { "_id": "2" }}
{ "tags" : ["search", "open_source"] }
{ "index": { "_id": "3" }}
{ "other_field" : "some data" }
{ "index": { "_id": "4" }}
{ "tags" : null }
{ "index": { "_id": "5" }}
{ "tags" : ["search", null] }

GET /my_index/_search

GET /my_index/_search
{
"fields": [
"tags"
],
"query": { "match_all": {}}

}

On 26 May 2015 at 09:33, Xudong You xudong.you@gmail.com wrote:

Thanks your reply!
But why if I remove "fields" from query DSL, then I can see those null
value fields in ES response?

On Tuesday, May 26, 2015 at 4:19:30 PM UTC+8, Allan Mitchell wrote:

Hi

If you have a field that has null as a value then as far as the inverted
index is concerned for that document it doesn''t exist

Have a look at this

Dealing with Null Values | Elasticsearch: The Definitive Guide [2.x] | Elastic

On 26 May 2015 at 08:11, Xudong You xudon...@gmail.com wrote:

I used "fields" to limit only return selected fields:
{
"fields" : ["Title", "Price"],
"query" :{
...
}
}

The Price fields might be null for some of the documents, I found that
only when "Price" field value is NOT null, it is returned in ES response,
otherwise, it is missing in response. But if I remove "fields" from the
query, the null value field will appear in response.

Is it by design of Elasticsearch? How to return the null value fields in
response even using "fields" in query?

--
Please update your bookmarks! We have moved to
https://discuss.elastic.co/

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3c91fd31-0e13-4afb-a9bf-1a0ec795e68f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/3c91fd31-0e13-4afb-a9bf-1a0ec795e68f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/71e4219a-e9a6-4fa0-a289-9bf3fb6e50a3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/71e4219a-e9a6-4fa0-a289-9bf3fb6e50a3%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAECdJzCt_%2BVvB8JqWRfT7Xs-9%3DHb_Jg-s6Yxarq%2B01b9mn5ggg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

Thanks. Understand now.

On Tuesday, May 26, 2015 at 4:46:47 PM UTC+8, Allan Mitchell wrote:

Hi

Here is a quick example (pinched from the article I sent you and added to)

You can see that we get the values from two different places. Whilst the
field exists at the _all level it does not exist in the inverted index.

POST /my_index/posts/_bulk
{ "index": { "_id": "1" }}
{ "tags" : ["search"] }
{ "index": { "_id": "2" }}
{ "tags" : ["search", "open_source"] }
{ "index": { "_id": "3" }}
{ "other_field" : "some data" }
{ "index": { "_id": "4" }}
{ "tags" : null }
{ "index": { "_id": "5" }}
{ "tags" : ["search", null] }

GET /my_index/_search

GET /my_index/_search
{
"fields": [
"tags"
],
"query": { "match_all": {}}

}

On 26 May 2015 at 09:33, Xudong You <xudon...@gmail.com <javascript:>>
wrote:

Thanks your reply!
But why if I remove "fields" from query DSL, then I can see those null
value fields in ES response?

On Tuesday, May 26, 2015 at 4:19:30 PM UTC+8, Allan Mitchell wrote:

Hi

If you have a field that has null as a value then as far as the inverted
index is concerned for that document it doesn''t exist

Have a look at this

Dealing with Null Values | Elasticsearch: The Definitive Guide [2.x] | Elastic

On 26 May 2015 at 08:11, Xudong You xudon...@gmail.com wrote:

I used "fields" to limit only return selected fields:
{
"fields" : ["Title", "Price"],
"query" :{
...
}
}

The Price fields might be null for some of the documents, I found that
only when "Price" field value is NOT null, it is returned in ES response,
otherwise, it is missing in response. But if I remove "fields" from the
query, the null value field will appear in response.

Is it by design of Elasticsearch? How to return the null value fields
in response even using "fields" in query?

--
Please update your bookmarks! We have moved to
https://discuss.elastic.co/

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3c91fd31-0e13-4afb-a9bf-1a0ec795e68f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/3c91fd31-0e13-4afb-a9bf-1a0ec795e68f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to
https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/71e4219a-e9a6-4fa0-a289-9bf3fb6e50a3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/71e4219a-e9a6-4fa0-a289-9bf3fb6e50a3%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e80d5718-8dfd-4775-9f80-f02977195329%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.