Search across types with same fields, different stored settings

mattweber · February 18, 2012, 4:48pm

I have an index where I defined some stored fields for a given type. I then had some docs that I didn't need to define a mapping and just threw them in the same index under a different type. I perform a search across both types and ask for the stored fields back. The results for the type with the stored fields has the fields returned, but the results for the type that I did not set the stored field does not return the fields. The field names are the same across each type. I expected the fields that were not stored to be pulled from the source but that does not seem to be the case. I read in the docs that said it is recommended that types with the same fields names is recommend to have the same mappings, so maybe this is just a problem with that. Considering both are string fields, analyzed the same, and the difference is only stored vs. not stored, it might be a bug?

Here is a gist showing the problem:

gist.github.com

https://gist.github.com/mattweber/1860066

mapping

curl -XPUT 'http://localhost:9200/testing' -d '{                                                                           
    "mappings" : {
        "testa" : {
            "_source" : {
                "enabled" : true
            },
            "properties" : {
                "field1" : {"type" : "string", "store" : "yes"},
                "field2" : {"type" : "string", "store" : "yes"}
            }

This file has been truncated. show original

results

{
  "took" : 62,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,

This file has been truncated. show original

test docs

curl -XPUT 'http://localhost:9200/testing/testa/1' -d '{"field1":"testa_field1","field2":"testa_field2","field3":"findme"}'

curl -XPUT 'http://localhost:9200/testing/testb/2' -d '{"field1":"testb_field1","field2":"testb_field2","field3":"findme"}'

There are more than three files. show original

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

kimchy · February 20, 2012, 1:00pm

Yes, there is an optimization to not check each mapping specifically per hit, but try to guess what to fetch on the "search" level (and not per hit) and then use it for all the hits. We can improve it though, open an issue?

On Saturday, February 18, 2012 at 6:48 PM, Matt Weber wrote:

I have an index where I defined some stored fields for a given type. I then had some docs that I didn't need to define a mapping and just threw them in the same index under a different type. I perform a search across both types and ask for the stored fields back. The results for the type with the stored fields has the fields returned, but the results for the type that I did not set the stored field does not return the fields. The field names are the same across each type. I expected the fields that were not stored to be pulled from the source but that does not seem to be the case. I read in the docs that said it is recommended that types with the same fields names is recommend to have the same mappings, so maybe this is just a problem with that. Considering both are string fields, analyzed the same, and the difference is only stored vs. not stored, it might be a bug?

Here is a gist showing the problem:

mapping · GitHub

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

mattweber · February 20, 2012, 3:30pm

Thanks. improve fetch optimizations · Issue #1721 · elastic/elasticsearch · GitHub

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Monday, February 20, 2012 at 5:00 AM, Shay Banon wrote:

Yes, there is an optimization to not check each mapping specifically per hit, but try to guess what to fetch on the "search" level (and not per hit) and then use it for all the hits. We can improve it though, open an issue?

On Saturday, February 18, 2012 at 6:48 PM, Matt Weber wrote:

I have an index where I defined some stored fields for a given type. I then had some docs that I didn't need to define a mapping and just threw them in the same index under a different type. I perform a search across both types and ask for the stored fields back. The results for the type with the stored fields has the fields returned, but the results for the type that I did not set the stored field does not return the fields. The field names are the same across each type. I expected the fields that were not stored to be pulled from the source but that does not seem to be the case. I read in the docs that said it is recommended that types with the same fields names is recommend to have the same mappings, so maybe this is just a problem with that. Considering both are string fields, analyzed the same, and the difference is only stored vs. not stored, it might be a bug?

Here is a gist showing the problem:

mapping · GitHub

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)