Differences with field _source between 0.17.10 and 0.19.0.RC2

Hi there,

I'm doing some tests with disabling source or not and storing fields or not.

I found that with 0.19.0.RC2, I can't get back the _source as a field, but
it was possible with 0.17.10 (and 0.18.7 I think).

Here is a full recreation :

Let me explain it here inline:

I created two types with same fields, but one with no source and fields
stored. The other one is standard, source enable.

Create the mapping using only fields

curl -XPOST http://localhost:9200/twitter/tweet/_mapping -d '{

"tweet" : {

     "_source" : {"enabled" : false},

    "properties" : {

        "user" : {"type" : "string", "store" : "yes"},

        "message" : {"type" : "string", "store" : "yes"}

    }

}

}'

Create the mapping using source

curl -XPOST http://localhost:9200/twitter/othertweet/_mapping -d '{

"othertweet" : {

    "properties" : {

        "user" : {"type" : "string"},

        "message" : {"type" : "string"}

    }

}

}'

Then I index about the same document with these two types :

Indexing with fields

curl -XPUT http://localhost:9200/twitter/tweet/1 -d '{
"user" : "dadoonet field",
"message" : "Elasticsearch rocks field"
}'

Indexing with source

curl -XPUT http://localhost:9200/twitter/othertweet/1 -d '{
"user" : "dadoonet source",
"message" : "Elasticsearch rocks source"
}'

With 0.17.10 (and 0.18.x), I can get the _source as a field :

_source works with 0.17.10 but not with 0.19.0.RC2

curl -XPOST http://localhost:9200/_search?pretty=true -d '{
"fields" : ["_source"],
"query" : {
"match_all" : { }
}
}'

But it doesn't return _source field with 0.19.0.RC2. I can't find in the
release notes if it's a new feature or not.

So is it a bug ? I will open an issue if needed.

BTW, what I'm looking for is to be able to get "*" (all fields) AND
"_source" field in one single call.

But,

_source doesn't appear even with 0.17.10 nor 0.19.0.RC2

curl -XPOST http://localhost:9200/_search?pretty=true -d '{
"fields" : ["*", "_source"],
"query" : {
"match_all" : { }
}
}'

Doesn't send the _source field back.

May I open an issue for that also ?

Thanks for your help

David.

Hi David

I found that with 0.19.0.RC2, I can’t get back the _source as a field,
but it was possible with 0.17.10 (and 0.18.7 I think).

This is probably related to issue 1705, which has been fixed in
0.19.0RC3: _source field not being returned when a script field is specified · Issue #1705 · elastic/elasticsearch · GitHub

The queries in your gist all seem to work in master

clint

Thanks Clint.

I did not find myself this issue... Thanks for the link.

BTW, there was another question. I'm trying to get “*” (all fields) AND
“_source” field in one single call.

But,

_source doesn't appear even with 0.17.10 nor 0.19.0.RC2

curl -XPOST
http://localhost:9200/_search?pretty
http://localhost:9200/_search?pretty

true -d '{

"fields" : ["*", "_source"],

"query" : {

  "match_all" : { }

}

}'

Doesn’t send the _source field back. If I ask for a specific field and
_source, it works fine. So I imagine that if * takes precedence of any
other field (including _source)

May I open an issue for that ?

Thanks,
David.

Le 21 février 2012 à 10:54, Clinton Gormley clint@traveljury.com a écrit
:

Hi David

I found that with 0.19.0.RC2, I can’t get back the _source as a field,
but it was possible with 0.17.10 (and 0.18.7 I think).

This is probably related to issue 1705, which has been fixed in
0.19.0RC3: _source field not being returned when a script field is specified · Issue #1705 · elastic/elasticsearch · GitHub

The queries in your gist all seem to work in master

clint

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet

David,

Did you see this?

I was trying to do the same thing a while ago and opened an issue. This fix should be in the 0.19.0 RC's, but I have not checked that it actually works.

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Tuesday, February 21, 2012 at 10:02 AM, david@pilato.fr wrote:

Thanks Clint.

I did not find myself this issue... Thanks for the link.

BTW, there was another question. I'm trying to get “*” (all fields) AND “_source” field in one single call.
But,

_source doesn't appear even with 0.17.10 nor 0.19.0.RC2

curl -XPOST http://localhost:9200/_search?pretty = true -d '{
"fields" : ["*", "_source"],
"query" : {
"match_all" : { }
}
}'

Doesn’t send the _source field back. If I ask for a specific field and _source, it works fine. So I imagine that if * takes precedence of any other field (including _source)

May I open an issue for that ?

Thanks,

David.

Le 21 février 2012 à 10:54, Clinton Gormley <clint@traveljury.com (mailto:clint@traveljury.com)> a écrit :

Hi David

I found that with 0.19.0.RC2, I can’t get back the _source as a field,
but it was possible with 0.17.10 (and 0.18.7 I think).

This is probably related to issue 1705, which has been fixed in
0.19.0RC3: _source field not being returned when a script field is specified · Issue #1705 · elastic/elasticsearch · GitHub

The queries in your gist all seem to work in master

clint

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet

Thanks Matt,

Exactly what I'm looking for !

As far as I remember my test yesterday (in France, we said that I have the same memory as a golden fish :wink: ), it was not working in 0.19.0.RC2 but it should work as Shay closed this issue two months ago.

I will make more tests with the RC3 tomorrow and comment the issue if needed.

Thanks again
David :wink:
@dadoonet

Le 21 févr. 2012 à 21:26, Matt Weber matt@mattweber.org a écrit :

David,

Did you see this?

source not returned when * specified in fields list · Issue #1541 · elastic/elasticsearch · GitHub

I was trying to do the same thing a while ago and opened an issue. This fix should be in the 0.19.0 RC's, but I have not checked that it actually works.

--
Matt Weber
Sent with Sparrow

On Tuesday, February 21, 2012 at 10:02 AM, david@pilato.fr wrote:

Thanks Clint.

I did not find myself this issue... Thanks for the link.

BTW, there was another question. I'm trying to get “*” (all fields) AND “_source” field in one single call.

But,

_source doesn't appear even with 0.17.10 nor 0.19.0.RC2

curl -XPOST
http://localhost:9200/_search?pretty

true -d '{

"fields" : ["*", "_source"],
"query" : {
  "match_all" : { }

}
}'

Doesn’t send the _source field back. If I ask for a specific field and _source, it works fine. So I imagine that if * takes precedence of any other field (including _source)

May I open an issue for that ?

Thanks,
David.

Le 21 février 2012 à 10:54, Clinton Gormley clint@traveljury.com a écrit :

Hi David

I found that with 0.19.0.RC2, I can’t get back the _source as a field,
but it was possible with 0.17.10 (and 0.18.7 I think).

This is probably related to issue 1705, which has been fixed in
0.19.0RC3: _source field not being returned when a script field is specified · Issue #1705 · elastic/elasticsearch · GitHub

The queries in your gist all seem to work in master

clint

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet

Matt,

It’s working fine in 0.19.x !

Thanks for your help

David.

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] De la part de David Pilato
Envoyé : mardi 21 février 2012 21:42
À : elasticsearch@googlegroups.com
Objet : Re: Differences with field _source between 0.17.10 and 0.19.0.RC2

Thanks Matt,

Exactly what I'm looking for !

As far as I remember my test yesterday (in France, we said that I have the same memory as a golden fish :wink: ), it was not working in 0.19.0.RC2 but it should work as Shay closed this issue two months ago.

I will make more tests with the RC3 tomorrow and comment the issue if needed.

Thanks again
David :wink:

@dadoonet

Le 21 févr. 2012 à 21:26, Matt Weber matt@mattweber.org a écrit :

David,

Did you see this?

I was trying to do the same thing a while ago and opened an issue. This fix should be in the 0.19.0 RC's, but I have not checked that it actually works.

--

Matt Weber

Sent with Sparrow http://www.sparrowmailapp.com/?sig

On Tuesday, February 21, 2012 at 10:02 AM, david@pilato.fr wrote:

Thanks Clint.

I did not find myself this issue... Thanks for the link.

BTW, there was another question. I'm trying to get “*” (all fields) AND “_source” field in one single call.

But,

_source doesn't appear even with 0.17.10 nor 0.19.0.RC2

curl -XPOST
http://localhost:9200/_search?pretty

true -d '{

"fields" : ["*", "_source"],
"query" : {
  "match_all" : { }

}
}'

Doesn’t send the _source field back. If I ask for a specific field and _source, it works fine. So I imagine that if * takes precedence of any other field (including _source)

May I open an issue for that ?

Thanks,

David.

Le 21 février 2012 à 10:54, Clinton Gormley clint@traveljury.com a écrit :

Hi David

I found that with 0.19.0.RC2, I can’t get back the _source as a field,
but it was possible with 0.17.10 (and 0.18.7 I think).

This is probably related to issue 1705, which has been fixed in
0.19.0RC3: _source field not being returned when a script field is specified · Issue #1705 · elastic/elasticsearch · GitHub

The queries in your gist all seem to work in master

clint

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet