Potential bug? Mapping seems to ignore store='no' and enabled: false

Elastic Stack Elasticsearch
1

Hi all,

There seems to be an issue with ES ignoring "store" : "no" and
"enabled": false.

I couldn't find any other information so I'm coming to the list. Tried
different combinations using 'false' instead of no, etc.

Also, the result of a GET is the same whether "_source" : { "enabled"
: false } or "_source" : { "enabled" : true }.

Search do appear different with "_source" : { "enabled" : false } --
but the GET still returns the full JSON...

Bash script to recreate is at gist: https://gist.github.com/1403154

Any pointers or help would be greatly appreciated.

Cheers,
Tal

2

Its not really ignored, when you do a get, it might fetch the document from
the transaction log, not the index storage itself, and there the document
source exists regardless if you set enabled to false. But, the fact that
its in the transaction log does not mean that we should return it in this
case, since its confusing, will open an issue to no do it:
When _source is disabled, don't return it in realtime get fetching the document from the transaction log · Issue #1509 · elastic/elasticsearch · GitHub.

Btw, what does store has to do with it?

On Tue, Nov 29, 2011 at 5:05 AM, Tal Rotbart redbeard@gmail.com wrote:

Hi all,

There seems to be an issue with ES ignoring "store" : "no" and
"enabled": false.

I couldn't find any other information so I'm coming to the list. Tried
different combinations using 'false' instead of no, etc.

Also, the result of a GET is the same whether "_source" : { "enabled"
: false } or "_source" : { "enabled" : true }.

Search do appear different with "_source" : { "enabled" : false } --
but the GET still returns the full JSON...

Bash script to recreate is at gist: Demonstrates a potential bug in elasticsearch's 'store' and 'enabled' mapping options · GitHub

Any pointers or help would be greatly appreciated.

Cheers,
Tal

3

And btw, your gist have _source enabled set to true.

On Tue, Nov 29, 2011 at 9:14 AM, Shay Banon kimchy@gmail.com wrote:

Its not really ignored, when you do a get, it might fetch the document
from the transaction log, not the index storage itself, and there the
document source exists regardless if you set enabled to false. But, the
fact that its in the transaction log does not mean that we should return it
in this case, since its confusing, will open an issue to no do it:
When _source is disabled, don't return it in realtime get fetching the document from the transaction log · Issue #1509 · elastic/elasticsearch · GitHub.

Btw, what does store has to do with it?

On Tue, Nov 29, 2011 at 5:05 AM, Tal Rotbart redbeard@gmail.com wrote:

Hi all,

There seems to be an issue with ES ignoring "store" : "no" and
"enabled": false.

I couldn't find any other information so I'm coming to the list. Tried
different combinations using 'false' instead of no, etc.

Also, the result of a GET is the same whether "_source" : { "enabled"
: false } or "_source" : { "enabled" : true }.

Search do appear different with "_source" : { "enabled" : false } --
but the GET still returns the full JSON...

Bash script to recreate is at gist: Demonstrates a potential bug in elasticsearch's 'store' and 'enabled' mapping options · GitHub

Any pointers or help would be greatly appreciated.

Cheers,
Tal

4

Thanks for the quick reply!

The transaction log explains the source being returned even with
_source disabled. I guess a clarification in the docs would do the
trick.

Thanks to your comment about store, I've realised that I've been
expecting ES to return the stored fields but I wasn't specifying
'fields'.

It all comes down to indexing 3rd party JSON which I have little
control over, I wanted to be able to avoid storing large amounts of
data in the index so needed fine grained control over which fields I'm
interested in storing and returning for view. Now I understand how to
do it. Thanks!

Cheers,
Tal

On 29 November 2011 18:20, Shay Banon kimchy@gmail.com wrote:

And btw, your gist have _source enabled set to true.

On Tue, Nov 29, 2011 at 9:14 AM, Shay Banon kimchy@gmail.com wrote:

Its not really ignored, when you do a get, it might fetch the document
from the transaction log, not the index storage itself, and there the
document source exists regardless if you set enabled to false. But, the fact
that its in the transaction log does not mean that we should return it in
this case, since its confusing, will open an issue to no do
it: When _source is disabled, don't return it in realtime get fetching the document from the transaction log · Issue #1509 · elastic/elasticsearch · GitHub.

Btw, what does store has to do with it?

On Tue, Nov 29, 2011 at 5:05 AM, Tal Rotbart redbeard@gmail.com wrote:

Hi all,

There seems to be an issue with ES ignoring "store" : "no" and
"enabled": false.

I couldn't find any other information so I'm coming to the list. Tried
different combinations using 'false' instead of no, etc.

Also, the result of a GET is the same whether "_source" : { "enabled"
: false } or "_source" : { "enabled" : true }.

Search do appear different with "_source" : { "enabled" : false } --
but the GET still returns the full JSON...

Bash script to recreate is at gist: Demonstrates a potential bug in elasticsearch's 'store' and 'enabled' mapping options · GitHub

Any pointers or help would be greatly appreciated.

Cheers,
Tal