Config'd mapping does not seem to take effect


(searchgirl) #1

Here is my setup:

  • index: "en"
  • type: "product"

the mapping file is at ES_HOME/config/mappings/_default/product.json
it contains:

{
"product" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : { "type" : "string", "omit_norms" :
"yes", "store": "no", "index": "not_analyzed"}
}
}
],
"_all" : {"enabled" : false},
"properties" : {
"pid" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"rid" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"pgrp" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"descl" : {
"type" : "string",
"store" : "no",
"index" : "analyzed",
"boost" : 1.0
},
"descs" : {
"type" : "string",
"store" : "no",
"index" : "analyzed",
"boost" : 1.3
}
}}}

Index is created like:

	node = NodeBuilder.nodeBuilder().data(true).node();
	client = node.client();

client.admin().indices().prepareCreate("en").execute().actionGet();

And indexed like:

	final IndexRequestBuilder irb = client.prepareIndex(getIndexName(),

getIndexType(), p.getProductId()).setSource(b);
irb.execute().actionGet();

When I do a get, I get the following:

{
_index: "en",
_type: "product",
_id: "58d2ccf1-abc6-4a61-9eca-7c4d342b48c8",
_version: 2,
exists: true,
_source: {
pid: "58d2ccf1-abc6-4a61-9eca-7c4d342b48c8",
rid: "44fe535f-4531-48bd-82d1-d96b6ef2bbe7",
descl: "The Way Home",
descs: "The Way Home"
}
}

I believe the mapping is not taking effect because "descl" & "descs"
have been defined as "store":"no". My understanding is that these
fields should not have been stored in the index and therefore should
not be returned.

"http://localhost:9200/en/_settings" returned following which also
indicate no mappings

{
en: {
settings: {
index.number_of_shards: "5",
index.number_of_replicas: "1",
index.version.created: "190399"
}
}
}

Questions: what am I doing wrong? Is there other ways to verify the
mappings are properly applied? I am using Java client.


(Shay Banon) #2

When you do a get mapping, do you see the one you defined? It looks good...

On Tue, May 15, 2012 at 7:45 PM, Helen Poon helenchenpoon@gmail.com wrote:

Here is my setup:

  • index: "en"
  • type: "product"

the mapping file is at ES_HOME/config/mappings/_default/product.json
it contains:

{
"product" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : { "type" : "string", "omit_norms" :
"yes", "store": "no", "index": "not_analyzed"}
}
}
],
"_all" : {"enabled" : false},
"properties" : {
"pid" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"rid" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"pgrp" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"descl" : {
"type" : "string",
"store" : "no",
"index" : "analyzed",
"boost" : 1.0
},
"descs" : {
"type" : "string",
"store" : "no",
"index" : "analyzed",
"boost" : 1.3
}
}}}

Index is created like:

           node = NodeBuilder.nodeBuilder().data(true).node();
           client = node.client();

client.admin().indices().prepareCreate("en").execute().actionGet();

And indexed like:

           final IndexRequestBuilder irb =

client.prepareIndex(getIndexName(),
getIndexType(), p.getProductId()).setSource(b);
irb.execute().actionGet();

When I do a get, I get the following:

{
_index: "en",
_type: "product",
_id: "58d2ccf1-abc6-4a61-9eca-7c4d342b48c8",
_version: 2,
exists: true,
_source: {
pid: "58d2ccf1-abc6-4a61-9eca-7c4d342b48c8",
rid: "44fe535f-4531-48bd-82d1-d96b6ef2bbe7",
descl: "The Way Home",
descs: "The Way Home"
}
}

I believe the mapping is not taking effect because "descl" & "descs"
have been defined as "store":"no". My understanding is that these
fields should not have been stored in the index and therefore should
not be returned.

"http://localhost:9200/en/_settings" returned following which also
indicate no mappings

{
en: {
settings: {
index.number_of_shards: "5",
index.number_of_replicas: "1",
index.version.created: "190399"
}
}
}

Questions: what am I doing wrong? Is there other ways to verify the
mappings are properly applied? I am using Java client.


(Igor Motov) #3

When you specify "store":"no" in the mapping, it controls if the field is
stored or not as a separate field. Your get request returns the source, not
individual stored fields, so, the presence of the field in the source
doesn't really indicate that the field was stored individually. You can
disable storing source and remove fields from the source.
See http://www.elasticsearch.org/guide/reference/mapping/source-field.html
for more details.

To verify that mapping was properly applied, simply run

curl 'localhost:9200/en/product/_mapping'

On Tuesday, May 15, 2012 12:45:37 PM UTC-4, Helen Poon wrote:

Here is my setup:

  • index: "en"
  • type: "product"

the mapping file is at ES_HOME/config/mappings/_default/product.json
it contains:

{
"product" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : { "type" : "string", "omit_norms" :
"yes", "store": "no", "index": "not_analyzed"}
}
}
],
"_all" : {"enabled" : false},
"properties" : {
"pid" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"rid" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"pgrp" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"descl" : {
"type" : "string",
"store" : "no",
"index" : "analyzed",
"boost" : 1.0
},
"descs" : {
"type" : "string",
"store" : "no",
"index" : "analyzed",
"boost" : 1.3
}
}}}

Index is created like:

            node = NodeBuilder.nodeBuilder().data(true).node(); 
            client = node.client(); 

client.admin().indices().prepareCreate("en").execute().actionGet();

And indexed like:

            final IndexRequestBuilder irb = 

client.prepareIndex(getIndexName(),
getIndexType(), p.getProductId()).setSource(b);
irb.execute().actionGet();

When I do a get, I get the following:

{
_index: "en",
_type: "product",
_id: "58d2ccf1-abc6-4a61-9eca-7c4d342b48c8",
_version: 2,
exists: true,
_source: {
pid: "58d2ccf1-abc6-4a61-9eca-7c4d342b48c8",
rid: "44fe535f-4531-48bd-82d1-d96b6ef2bbe7",
descl: "The Way Home",
descs: "The Way Home"
}
}

I believe the mapping is not taking effect because "descl" & "descs"
have been defined as "store":"no". My understanding is that these
fields should not have been stored in the index and therefore should
not be returned.

"http://localhost:9200/en/_settings" returned following which also
indicate no mappings

{
en: {
settings: {
index.number_of_shards: "5",
index.number_of_replicas: "1",
index.version.created: "190399"
}
}
}

Questions: what am I doing wrong? Is there other ways to verify the
mappings are properly applied? I am using Java client.

On Tuesday, May 15, 2012 12:45:37 PM UTC-4, Helen Poon wrote:

Here is my setup:

  • index: "en"
  • type: "product"

the mapping file is at ES_HOME/config/mappings/_default/product.json
it contains:

{
"product" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : { "type" : "string", "omit_norms" :
"yes", "store": "no", "index": "not_analyzed"}
}
}
],
"_all" : {"enabled" : false},
"properties" : {
"pid" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"rid" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"pgrp" : {
"type" : "string",
"store" : "yes",
"index" : "not_analyzed"
},
"descl" : {
"type" : "string",
"store" : "no",
"index" : "analyzed",
"boost" : 1.0
},
"descs" : {
"type" : "string",
"store" : "no",
"index" : "analyzed",
"boost" : 1.3
}
}}}

Index is created like:

            node = NodeBuilder.nodeBuilder().data(true).node(); 
            client = node.client(); 

client.admin().indices().prepareCreate("en").execute().actionGet();

And indexed like:

            final IndexRequestBuilder irb = 

client.prepareIndex(getIndexName(),
getIndexType(), p.getProductId()).setSource(b);
irb.execute().actionGet();

When I do a get, I get the following:

{
_index: "en",
_type: "product",
_id: "58d2ccf1-abc6-4a61-9eca-7c4d342b48c8",
_version: 2,
exists: true,
_source: {
pid: "58d2ccf1-abc6-4a61-9eca-7c4d342b48c8",
rid: "44fe535f-4531-48bd-82d1-d96b6ef2bbe7",
descl: "The Way Home",
descs: "The Way Home"
}
}

I believe the mapping is not taking effect because "descl" & "descs"
have been defined as "store":"no". My understanding is that these
fields should not have been stored in the index and therefore should
not be returned.

"http://localhost:9200/en/_settings" returned following which also
indicate no mappings

{
en: {
settings: {
index.number_of_shards: "5",
index.number_of_replicas: "1",
index.version.created: "190399"
}
}
}

Questions: what am I doing wrong? Is there other ways to verify the
mappings are properly applied? I am using Java client.


(system) #4