Problem with Fields in search requests

Mustafa_Sener · April 29, 2011, 3:31pm

Hi,
I have following type mapping
XContentBuilder builder = XContentFactory.jsonBuilder();

   builder.startObject().
            startObject("optimizedmetric1").
            startObject("_all").field("enabled", false).endObject().
            startObject("_source").field("enabled", false).endObject().
            startObject("_index").field("enabled", false).endObject().
            startObject("properties").
            startObject("object").
            field("type", "string").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            startObject("metric").
            field("type", "string").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            startObject("value").
            field("type", "double").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            startObject("timestamp").
            field("type", "long").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            startObject("standartdev").
            field("type", "double").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            endObject().
            endObject().
            endObject();

After inserting some documents, I tried to search and get fields of inserted
objects using following CURL request*

curl -XGET 'http://host:port/performance/optimizedmetric1/_search' -d '
{
"fields" : ["object"],
"query" : {
"match_all":{}
}
}'

It returns following result:

{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":10000,"max_score":1.0,"hits":[{"_index":"performance","_type":"optimizedmetric1","_id":"grfsll6yruqlxkojlydjrg","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"p11wzy33teq9h6zcmbooka","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"vfjf8he-soqnvxemmkgira","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"p1wda31dtqyjgbagscxilw","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"oyahsw7cs4iuf2efcp_g6g","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"db0y-k0yqg6qntbfrt1qvg","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"-i5vmsptrfsze81stswoxq","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"1ciy9u95tzetb9snee-_ya","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"rwrkklo2syofeuvhwmy0lg","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"s8amnskbrzkkk8oxgyv38g","_score":1.0}]}}

I expected to see object field but results does not include it. is this
normal?

--
Mustafa Sener
www.ifountain.com

Mustafa_Sener · April 29, 2011, 3:52pm

I am using version v0.16.0

On Fri, Apr 29, 2011 at 6:31 PM, Mustafa Sener mustafa.sener@gmail.comwrote:

Hi,
I have following type mapping
XContentBuilder builder = XContentFactory.jsonBuilder();
   builder.startObject().
            startObject("optimizedmetric1").
            startObject("_all").field("enabled", false).endObject().
            startObject("_source").field("enabled", false).endObject().
            startObject("_index").field("enabled", false).endObject().
            startObject("properties").
            startObject("object").
            field("type", "string").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            startObject("metric").
            field("type", "string").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            startObject("value").
            field("type", "double").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            startObject("timestamp").
            field("type", "long").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            startObject("standartdev").
            field("type", "double").
            field("store", "yes").
            field("index", "not_analyzed").
            field("omit_norms", true).
            endObject().

            endObject().
            endObject().
            endObject();
After inserting some documents, I tried to search and get fields of
inserted objects using following CURL request*

curl -XGET 'http://host:port/performance/optimizedmetric1/_search' -d '
{
"fields" : ["object"],
"query" : {
"match_all":{}
}
}'

It returns following result:

{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":10000,"max_score":1.0,"hits":[{"_index":"performance","_type":"optimizedmetric1","_id":"grfsll6yruqlxkojlydjrg","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"p11wzy33teq9h6zcmbooka","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"vfjf8he-soqnvxemmkgira","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"p1wda31dtqyjgbagscxilw","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"oyahsw7cs4iuf2efcp_g6g","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"db0y-k0yqg6qntbfrt1qvg","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"-i5vmsptrfsze81stswoxq","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"1ciy9u95tzetb9snee-_ya","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"rwrkklo2syofeuvhwmy0lg","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"s8amnskbrzkkk8oxgyv38g","_score":1.0}]}}

I expected to see object field but results does not include it. is this
normal?

--
Mustafa Sener
www.ifountain.com

--
Mustafa Sener
www.ifountain.com
WebRep
Overall rating

kimchy · April 29, 2011, 4:17pm

Can you try and do _source.object and see if it works? Also, if it does not work, can you gist the code that inserts a sample doc?
On Friday, April 29, 2011 at 6:52 PM, Mustafa Sener wrote:

I am using version v0.16.0

On Fri, Apr 29, 2011 at 6:31 PM, Mustafa Sener mustafa.sener@gmail.com wrote:

Hi,
I have following type mapping
XContentBuilder builder = XContentFactory.jsonBuilder();

builder.startObject().
startObject("optimizedmetric1").
startObject("_all").field("enabled", false).endObject().
startObject("_source").field("enabled", false).endObject().
startObject("_index").field("enabled", false).endObject().
startObject("properties").
startObject("object").
field("type", "string").
field("store", "yes").
field("index", "not_analyzed").
field("omit_norms", true).
endObject().

startObject("metric").
field("type", "string").
field("store", "yes").
field("index", "not_analyzed").
field("omit_norms", true).
endObject().

startObject("value").
field("type", "double").
field("store", "yes").
field("index", "not_analyzed").
field("omit_norms", true).
endObject().

startObject("timestamp").
field("type", "long").
field("store", "yes").
field("index", "not_analyzed").
field("omit_norms", true).
endObject().

startObject("standartdev").
field("type", "double").
field("store", "yes").
field("index", "not_analyzed").
field("omit_norms", true).
endObject().

endObject().
endObject().
endObject();

After inserting some documents, I tried to search and get fields of inserted objects using following CURL request

curl -XGET 'http://host:port/performance/optimizedmetric1/_search' -d '
{
"fields" : ["object"],
"query" : {
"match_all":{}
}
}'

It returns following result:

{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":10000,"max_score":1.0,"hits":[{"_index":"performance","_type":"optimizedmetric1","_id":"grfsll6yruqlxkojlydjrg","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"p11wzy33teq9h6zcmbooka","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"vfjf8he-soqnvxemmkgira","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"p1wda31dtqyjgbagscxilw","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"oyahsw7cs4iuf2efcp_g6g","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"db0y-k0yqg6qntbfrt1qvg","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"-i5vmsptrfsze81stswoxq","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"1ciy9u95tzetb9snee-_ya","_score":1.0},{"_index":"performance","_type":"optimizedmetric1","_id":"rwrkklo2syofeuvhwmy0lg","_score":1.0},{"_index
":"performance","_type":"optimizedmetric1","_id":"s8amnskbrzkkk8oxgyv38g","_score":1.0}]}}

I expected to see object field but results does not include it. is this normal?

--
Mustafa Sener
www.ifountain.com

--
Mustafa Sener
www.ifountain.com
WebRep

Overall rating

Clinton_Gormley · April 29, 2011, 4:18pm

Hi Mustafa

After inserting some documents, I tried to search and get fields of
inserted objects using following CURL request

curl -XGET 'http://host:port/performance/optimizedmetric1/_search' -d
'
{
"fields" : ["object"],
"query" : {
"match_all":{}
}
}'

Objects are not storable.

If you want to return the object, then you should enable the _source
field and request the "_source.object" field

curl -XPUT 'http://127.0.0.1:9200/foo/?pretty=1' -d '
{
"mappings" : {
"bar" : {
"properties" : {
"object" : {
"type" : "object",
"properties" : {
"text" : {
"type" : "string"
}
}
}
}
}
}
}
'

curl -XPOST 'http://127.0.0.1:9200/foo/bar?pretty=1' -d '
{
"object" : {
"text" : "bar"
}
}
'

curl -XGET 'http://127.0.0.1:9200/foo/_search?pretty=1' -d '
{
"fields" : "_source.object"
}
'

[Fri Apr 29 18:18:03 2011] Response:

{

"hits" : {

"hits" : [

{

"_score" : 1,

"fields" : {

"_source.object" : {

"text" : "bar"

}

},

"_index" : "foo",

"_id" : "uuSPXe96QJaX_OxJud_VwQ",

"_type" : "bar"

}

],

"max_score" : 1,

"total" : 1

},

"timed_out" : false,

"_shards" : {

"failed" : 0,

"successful" : 5,

"total" : 5

},

"took" : 1

}

Clinton_Gormley · April 29, 2011, 4:20pm

On Fri, 2011-04-29 at 19:17 +0300, Shay Banon wrote:

Can you try and do _source.object and see if it works? Also, if it
does not work, can you gist the code that inserts a sample doc?

He has _source disabled:

startObject("_source").field("enabled",false).endObject().

Mustafa_Sener · April 29, 2011, 4:21pm

The type of object property is string as you can see from mappings. It is
not Object type.

On Fri, Apr 29, 2011 at 7:18 PM, Clinton Gormley clinton@iannounce.co.ukwrote:

Hi Mustafa

After inserting some documents, I tried to search and get fields of
inserted objects using following CURL request

curl -XGET 'http://host:port/performance/optimizedmetric1/_search' -d
'
{
"fields" : ["object"],
"query" : {
"match_all":{}
}
}'

Objects are not storable.

If you want to return the object, then you should enable the _source
field and request the "_source.object" field

curl -XPUT 'http://127.0.0.1:9200/foo/?pretty=1' -d '
{
"mappings" : {
"bar" : {
"properties" : {
"object" : {
"type" : "object",
"properties" : {
"text" : {
"type" : "string"
}
}
}
}
}
}
}
'

curl -XPOST 'http://127.0.0.1:9200/foo/bar?pretty=1' -d '
{
"object" : {
"text" : "bar"
}
}
'

curl -XGET 'http://127.0.0.1:9200/foo/_search?pretty=1' -d '
{
"fields" : "_source.object"
}
'

[Fri Apr 29 18:18:03 2011] Response:

{

"hits" : {

"hits" : [

{

"_score" : 1,

"fields" : {

"_source.object" : {

"text" : "bar"

}

},

"_index" : "foo",

"_id" : "uuSPXe96QJaX_OxJud_VwQ",

"_type" : "bar"

}

],

"max_score" : 1,

"total" : 1

},

"timed_out" : false,

"_shards" : {

"failed" : 0,

"successful" : 5,

"total" : 5

},

"took" : 1

}

--
Mustafa Sener
www.ifountain.com
WebRep
Overall rating

Mustafa_Sener · April 29, 2011, 4:23pm

In my code source is disabled. I just want to store and search fields and
want to make some tests with this configuration.

On Fri, Apr 29, 2011 at 7:21 PM, Mustafa Sener mustafa.sener@gmail.comwrote:

The type of object property is string as you can see from mappings. It is
not Object type.

On Fri, Apr 29, 2011 at 7:18 PM, Clinton Gormley clinton@iannounce.co.ukwrote:

Hi Mustafa

After inserting some documents, I tried to search and get fields of
inserted objects using following CURL request

curl -XGET 'http://host:port/performance/optimizedmetric1/_search' -d
'
{
"fields" : ["object"],
"query" : {
"match_all":{}
}
}'

Objects are not storable.

If you want to return the object, then you should enable the _source
field and request the "_source.object" field

curl -XPUT 'http://127.0.0.1:9200/foo/?pretty=1' -d '
{
"mappings" : {
"bar" : {
"properties" : {
"object" : {
"type" : "object",
"properties" : {
"text" : {
"type" : "string"
}
}
}
}
}
}
}
'

curl -XPOST 'http://127.0.0.1:9200/foo/bar?pretty=1' -d '
{
"object" : {
"text" : "bar"
}
}
'

curl -XGET 'http://127.0.0.1:9200/foo/_search?pretty=1' -d '
{
"fields" : "_source.object"
}
'

[Fri Apr 29 18:18:03 2011] Response:

{

"hits" : {

"hits" : [

{

"_score" : 1,

"fields" : {

"_source.object" : {

"text" : "bar"

}

},

"_index" : "foo",

"_id" : "uuSPXe96QJaX_OxJud_VwQ",

"_type" : "bar"

}

],

"max_score" : 1,

"total" : 1

},

"timed_out" : false,

"_shards" : {

"failed" : 0,

"successful" : 5,

"total" : 5

},

"took" : 1

}

--
Mustafa Sener
www.ifountain.com
WebRep
Overall rating

--
Mustafa Sener
www.ifountain.com
WebRep
Overall rating

Clinton_Gormley · April 29, 2011, 4:24pm

On Fri, 2011-04-29 at 19:21 +0300, Mustafa Sener wrote:

The type of object property is string as you can see from mappings. It
is not Object type.

Oh sorry - I'm not used to reading the Java API

clint

kimchy · April 29, 2011, 8:45pm

Heya, I missed the part where the object is a field and is stored. You should get it back. Can you create a simple test case to recreate it (would be simplest) with data being inserted?
On Friday, April 29, 2011 at 7:24 PM, Clinton Gormley wrote:

On Fri, 2011-04-29 at 19:21 +0300, Mustafa Sener wrote:

The type of object property is string as you can see from mappings. It
is not Object type.

Oh sorry - I'm not used to reading the Java API

clint

Mustafa_Sener · April 30, 2011, 8:37am

Hi,
I could not recreate it with simple data insertion. I will investigate this
problem in our environment and return back if same problem continues.

On Fri, Apr 29, 2011 at 11:45 PM, Shay Banon
shay.banon@elasticsearch.comwrote:

Heya, I missed the part where the object is a field and is stored. You
should get it back. Can you create a simple test case to recreate it (would
be simplest) with data being inserted?

On Friday, April 29, 2011 at 7:24 PM, Clinton Gormley wrote:

On Fri, 2011-04-29 at 19:21 +0300, Mustafa Sener wrote:

The type of object property is string as you can see from mappings. It
is not Object type.

Oh sorry - I'm not used to reading the Java API

clint

--
Mustafa Sener
www.ifountain.com
WebRep
Overall rating