Hiding and showing fields in search results and _source


(Alexander Reelsen) #1

Hi,

I am trying to hide nested fields per default and only showing them if
the 'fields' parameter is used in the query - which does not work as I
thought it would.

First my setup, a standard mapping

{
"test": {
"properties": {
"name": { "type": "string", "index":
"not_analyzed", "store" : "yes" },
"customPrices": { "store" : "yes", "type" : "object" }
},
"_source" : {
"excludes" : [ "customPrices", "name" ]
}
}
}

The mapping should define, that anything gets included, except name
(for testing purposes), which is a flat field and customPrices which
is a nested object

Now I index some object matching the above criteria

curl -X PUT localhost:9200/test/test/1 -d '{ "name" : "somename" ,
"foo" :"foo", "bar":"bar", "customPrices" : [ { "name" : "customer1",
"price" : 22.0}, { "name" : "customer2", "price" : 1000 } ] }'

Match all querying works

curl -X POST localhost:9200/test/test/_search?pretty=1 -d '{ "query" :
{ "match_all" : {}}}'

Returns the source like this: "_source" : {"foo":"foo","bar":"bar"}

curl -X POST localhost:9200/test/test/_search?pretty=1 -d '{ "query" :
{ "match_all" : {} }, "fields" : [ "name" ]}'

Does not return the source, but the fields array including the name field

However if I do a similar search for the customer, I fail and do not
get any fields, if I use one of these searches

curl -X POST localhost:9200/test/test/_search?pretty=1 -d '{ "query" :
{ "match_all" : {} }, "fields" : [ "customPrice" ]}'
curl -X POST localhost:9200/test/test/_search?pretty=1 -d '{ "query" :
{ "match_all" : {} }, "fields" : [ "customPrice.customer1" ]}'
curl -X POST localhost:9200/test/test/_search?pretty=1 -d '{ "query" :
{ "match_all" : {} }, "fields" : [ "*" ]}'

The last query does include the name field but not the customPrices one...

I am sure this is possible, but I am missing something or have a wrong
mapping configuration :slight_smile:

Any hints greatly appreciated!

--Alexander

--


(Clinton Gormley) #2

Hi Alexander

{
"test": {
"properties": {
"name": { "type": "string", "index":
"not_analyzed", "store" : "yes" },
"customPrices": { "store" : "yes", "type" : "object" }

An object type can't be store'd. Only the scalar fields within the
object.

I am sure this is possible, but I am missing something or have a wrong
mapping configuration :slight_smile:

Perhaps you want to store the whole _source, but when you query you can
use partial fields to just return the parts of the source that you want
to see.

clint

--


(Alexander Reelsen) #3

Hey Clinton

On Mon, Sep 17, 2012 at 9:20 PM, Clinton Gormley clint@traveljury.com wrote:

An object type can't be store'd. Only the scalar fields within the
object.
Ok. No documentation reading after a full work day anymore... it is
pretty obvious :slight_smile:

Perhaps you want to store the whole _source, but when you query you can
use partial fields to just return the parts of the source that you want
to see.
Good point about storing the whole source. However using partial
fields still returns the whole source plus the partial fields if I
recall correctly from my last tests...

--Alexander

--


(Clinton Gormley) #4

Perhaps you want to store the whole _source, but when you query you can
use partial fields to just return the parts of the source that you want
to see.
Good point about storing the whole source. However using partial
fields still returns the whole source plus the partial fields if I
recall correctly from my last tests...

No. If you request any field (or partial field) then it only returns the
_source if you specifically request it.

clint

--


(Alexander Reelsen) #5

Hi Clinton

On Tue, Sep 18, 2012 at 11:24 AM, Clinton Gormley clint@traveljury.com wrote:

Good point about storing the whole source. However using partial
fields still returns the whole source plus the partial fields if I
recall correctly from my last tests...
No. If you request any field (or partial field) then it only returns the
_source if you specifically request it.
If I request a single field, which is not in the source (excluded via
mapping config), the _source will not be returned.
If I request a single field, which excluded from the source plus a
field being stored in it, I'll get back the whole source.

Perhaps this is a bug, I'm not sure. I can provide a minimal sample if needed.

--Alexander

--


(Clinton Gormley) #6

Perhaps this is a bug, I'm not sure. I can provide a minimal sample if needed.

Please do

clint

--


(system) #7