Fields Not Returning When Explicitly Requested

Elastic Stack Elasticsearch
1

Hi,

I'm having problems returning fields in a query when I specifically
request them. If I run this query (without the fields):

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
}
}

I get a full document returned:

{
PID: 0
RSID: 2
Issues: [15 6]
OL: [
{
F: true
A: CNN Web site
Url: http://cnn.com
}
{
F: true
A: BBC Web site
Url: http://bbc.co.uk
}
]
}

If I explicitly request the "OL" field it isn't return. So the query:

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
},
"fields": [
"OL",
"Issues"
]
}

Only returns the "Issues" field but not the "OL" field. The mapping
for the document looks like this:

Page: {
properties: {
OL: {
dynamic: true
properties: {
F: {
type: boolean
}
Url: {
type: string
}
A: {
type: string
}
}
}
PID: {
type: long
}
RSID: {
type: long
}
Issues: {
type: long
}
}

Could someone tell me if this is the expected behavior or am I doing
something wrong?

I'm running ES 0.17.6 and executing these queries using
http://mobz.github.com/elasticsearch-head/

Thanks,

2

Fields are extracted from _source only if they are fields mapped (and not
json objects). You can specify _source.OL and it will only be returned.

On Thu, Sep 8, 2011 at 6:40 PM, mordant asmint3@hotmail.co.uk wrote:

Hi,

I'm having problems returning fields in a query when I specifically
request them. If I run this query (without the fields):

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
}
}

I get a full document returned:

{
PID: 0
RSID: 2
Issues: [15 6]
OL: [
{
F: true
A: CNN Web site
Url: http://cnn.com
}
{
F: true
A: BBC Web site
Url: http://bbc.co.uk
}
]
}

If I explicitly request the "OL" field it isn't return. So the query:

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
},
"fields": [
"OL",
"Issues"
]
}

Only returns the "Issues" field but not the "OL" field. The mapping
for the document looks like this:

Page: {
properties: {
OL: {
dynamic: true
properties: {
F: {
type: boolean
}
Url: {
type: string
}
A: {
type: string
}
}
}
PID: {
type: long
}
RSID: {
type: long
}
Issues: {
type: long
}
}

Could someone tell me if this is the expected behavior or am I doing
something wrong?

I'm running ES 0.17.6 and executing these queries using
http://mobz.github.com/elasticsearch-head/

Thanks,

3

Thanks Shay,

(Sorry for the delayed reply))

So with my current schema is it possible to return an individual
propery of "OL"? For example "OL.A"?

I don't really understand why I can query the field e.g.

"query": {
"query_string": {
"query": "OL.A:bbc"
}
},

But I can't return "OL.A" as a field.

Thanks for your help,

On Sep 8, 6:45 pm, Shay Banon kim...@gmail.com wrote:

Fields are extracted from _source only if they are fields mapped (and not
json objects). You can specify _source.OL and it will only be returned.

On Thu, Sep 8, 2011 at 6:40 PM, mordant asmi...@hotmail.co.uk wrote:

Hi,

I'm having problems returning fields in a query when I specifically
request them. If I run this query (without the fields):

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
}
}

I get a full document returned:

{
PID: 0
RSID: 2
Issues: [15 6]
OL: [
{
F: true
A: CNN Web site
Url:http://cnn.com
}
{
F: true
A: BBC Web site
Url:http://bbc.co.uk
}
]
}

If I explicitly request the "OL" field it isn't return. So the query:

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
},
"fields": [
"OL",
"Issues"
]
}

Only returns the "Issues" field but not the "OL" field. The mapping
for the document looks like this:

Page: {
properties: {
OL: {
dynamic: true
properties: {
F: {
type: boolean
}
Url: {
type: string
}
A: {
type: string
}
}
}
PID: {
type: long
}
RSID: {
type: long
}
Issues: {
type: long
}
}

Could someone tell me if this is the expected behavior or am I doing
something wrong?

I'm running ES 0.17.6 and executing these queries using
http://mobz.github.com/elasticsearch-head/

Thanks,

4

In master (upcoming 0.18), just specifying OL (an object json type) will
work and return it. In 0.17.x, you need to prefix objects with _source., for
example, _source.OL.

On Thu, Oct 13, 2011 at 3:29 PM, mordant asmint3@hotmail.co.uk wrote:

Thanks Shay,

(Sorry for the delayed reply))

So with my current schema is it possible to return an individual
propery of "OL"? For example "OL.A"?

I don't really understand why I can query the field e.g.

"query": {
"query_string": {
"query": "OL.A:bbc"
}
},

But I can't return "OL.A" as a field.

Thanks for your help,

On Sep 8, 6:45 pm, Shay Banon kim...@gmail.com wrote:

Fields are extracted from _source only if they are fields mapped (and not
json objects). You can specify _source.OL and it will only be returned.

On Thu, Sep 8, 2011 at 6:40 PM, mordant asmi...@hotmail.co.uk wrote:

Hi,

I'm having problems returning fields in a query when I specifically
request them. If I run this query (without the fields):

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
}
}

I get a full document returned:

{
PID: 0
RSID: 2
Issues: [15 6]
OL: [
{
F: true
A: CNN Web site
Url:http://cnn.com
}
{
F: true
A: BBC Web site
Url:http://bbc.co.uk
}
]
}

If I explicitly request the "OL" field it isn't return. So the query:

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
},
"fields": [
"OL",
"Issues"
]
}

Only returns the "Issues" field but not the "OL" field. The mapping
for the document looks like this:

Page: {
properties: {
OL: {
dynamic: true
properties: {
F: {
type: boolean
}
Url: {
type: string
}
A: {
type: string
}
}
}
PID: {
type: long
}
RSID: {
type: long
}
Issues: {
type: long
}
}

Could someone tell me if this is the expected behavior or am I doing
something wrong?

I'm running ES 0.17.6 and executing these queries using
http://mobz.github.com/elasticsearch-head/

Thanks,

5

Thanks Shay,

From what you've said I'll still only be able to return the "root"
json object though and not a property of the object. Is that correct?

Is it worth raising a request for this or is it unfeasible?

As a slight aside (but on the same theme) it would make life much
easier if it was possible to return the embedded json object as well
and not its "parent" document. It leads to a lot of post-processing of
results when you start working with embedded json. Is this feasible?

Thanks again,

On Oct 14, 1:13 pm, Shay Banon kim...@gmail.com wrote:

In master (upcoming 0.18), just specifying OL (an object json type) will
work and return it. In 0.17.x, you need to prefix objects with _source., for
example, _source.OL.

On Thu, Oct 13, 2011 at 3:29 PM, mordant asmi...@hotmail.co.uk wrote:

Thanks Shay,

(Sorry for the delayed reply))

So with my current schema is it possible to return an individual
propery of "OL"? For example "OL.A"?

I don't really understand why I can query the field e.g.

"query": {
"query_string": {
"query": "OL.A:bbc"
}
},

But I can't return "OL.A" as a field.

Thanks for your help,

On Sep 8, 6:45 pm, Shay Banon kim...@gmail.com wrote:

Fields are extracted from _source only if they are fields mapped (and not
json objects). You can specify _source.OL and it will only be returned.

On Thu, Sep 8, 2011 at 6:40 PM, mordant asmi...@hotmail.co.uk wrote:

Hi,

I'm having problems returning fields in a query when I specifically
request them. If I run this query (without the fields):

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
}
}

I get a full document returned:

{
PID: 0
RSID: 2
Issues: [15 6]
OL: [
{
F: true
A: CNN Web site
Url:http://cnn.com
}
{
F: true
A: BBC Web site
Url:http://bbc.co.uk
}
]
}

If I explicitly request the "OL" field it isn't return. So the query:

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
},
"fields": [
"OL",
"Issues"
]
}

Only returns the "Issues" field but not the "OL" field. The mapping
for the document looks like this:

Page: {
properties: {
OL: {
dynamic: true
properties: {
F: {
type: boolean
}
Url: {
type: string
}
A: {
type: string
}
}
}
PID: {
type: long
}
RSID: {
type: long
}
Issues: {
type: long
}
}

Could someone tell me if this is the expected behavior or am I doing
something wrong?

I'm running ES 0.17.6 and executing these queries using
http://mobz.github.com/elasticsearch-head/

Thanks,

6

You can return a "path" to any inner object. For example, if you indexed:

"obj1" : {
"obj2" : {
"field3" : "value"
}
}

In 0.17, you can return "just" obj2 by specifying _source.obj1.obj2. In
master (upcoming 0.18), you can just state: obj1.obj2 (and its recommended
once its out).

On Mon, Oct 17, 2011 at 7:34 PM, mordant asmint3@hotmail.co.uk wrote:

Thanks Shay,

From what you've said I'll still only be able to return the "root"
json object though and not a property of the object. Is that correct?

Is it worth raising a request for this or is it unfeasible?

As a slight aside (but on the same theme) it would make life much
easier if it was possible to return the embedded json object as well
and not its "parent" document. It leads to a lot of post-processing of
results when you start working with embedded json. Is this feasible?

Thanks again,

On Oct 14, 1:13 pm, Shay Banon kim...@gmail.com wrote:

In master (upcoming 0.18), just specifying OL (an object json type) will
work and return it. In 0.17.x, you need to prefix objects with _source.,
for
example, _source.OL.

On Thu, Oct 13, 2011 at 3:29 PM, mordant asmi...@hotmail.co.uk wrote:

Thanks Shay,

(Sorry for the delayed reply))

So with my current schema is it possible to return an individual
propery of "OL"? For example "OL.A"?

I don't really understand why I can query the field e.g.

"query": {
"query_string": {
"query": "OL.A:bbc"
}
},

But I can't return "OL.A" as a field.

Thanks for your help,

On Sep 8, 6:45 pm, Shay Banon kim...@gmail.com wrote:

Fields are extracted from _source only if they are fields mapped (and
not
json objects). You can specify _source.OL and it will only be
returned.

On Thu, Sep 8, 2011 at 6:40 PM, mordant asmi...@hotmail.co.uk
wrote:

Hi,

I'm having problems returning fields in a query when I specifically
request them. If I run this query (without the fields):

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
}
}

I get a full document returned:

{
PID: 0
RSID: 2
Issues: [15 6]
OL: [
{
F: true
A: CNN Web site
Url:http://cnn.com
}
{
F: true
A: BBC Web site
Url:http://bbc.co.uk
}
]
}

If I explicitly request the "OL" field it isn't return. So the
query:

{
"size": 1,
"query": {
"filtered": {
"query": {
"term": {
"PID": 0
}
},
"filter": {
"term": {
"RSID": 2
}
}
}
},
"fields": [
"OL",
"Issues"
]
}

Only returns the "Issues" field but not the "OL" field. The mapping
for the document looks like this:

Page: {
properties: {
OL: {
dynamic: true
properties: {
F: {
type: boolean
}
Url: {
type: string
}
A: {
type: string
}
}
}
PID: {
type: long
}
RSID: {
type: long
}
Issues: {
type: long
}
}

Could someone tell me if this is the expected behavior or am I
doing
something wrong?

I'm running ES 0.17.6 and executing these queries using
http://mobz.github.com/elasticsearch-head/

Thanks,