Don't store certain fields by default

Allan_Johns · June 12, 2013, 9:01am

I can't quite figure out from the docs how to do this. I know I need to use
the exclude functionality shown here
http://www.elasticsearch.org/guide/reference/mapping/source-field/, but the
documentation frustratingly provides the code snippets, but doesn't say
where to put them.

I want to not store fields beginning with an underscore. I tried adding
this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Allan_Johns · June 12, 2013, 9:07am

I also tried this in my default-mapping.json, no luck (see template_2):

{
"default" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "folder",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer": "gas_folder_analyzer"
}
}
},
{
"template_2" : {
"match" : "_",
"mapping" : {
"store": "no"
}
}
},
{
"template_3" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
]
}
}

On Wed, Jun 12, 2013 at 7:01 PM, Allan Johns nerdvegas@gmail.com wrote:

I can't quite figure out from the docs how to do this. I know I need to
use the exclude functionality shown here
Elasticsearch Platform — Find real-time answers at scale | Elastic, but
the documentation frustratingly provides the code snippets, but doesn't say
where to put them.

I want to not store fields beginning with an underscore. I tried adding
this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Allan_Johns · June 13, 2013, 1:35am

Anyone?

I tried:
"store" : false
also, this didn't work either.

You can actually do this, right?

A

On Wed, Jun 12, 2013 at 7:07 PM, Allan Johns nerdvegas@gmail.com wrote:

I also tried this in my default-mapping.json, no luck (see template_2):

{
"default" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "folder",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer": "gas_folder_analyzer"
}
}
},
{
"template_2" : {
"match" : "_",
"mapping" : {
"store": "no"
}
}
},
{
"template_3" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
]
}
}

On Wed, Jun 12, 2013 at 7:01 PM, Allan Johns nerdvegas@gmail.com wrote:

I can't quite figure out from the docs how to do this. I know I need to
use the exclude functionality shown here
Elasticsearch Platform — Find real-time answers at scale | Elastic, but
the documentation frustratingly provides the code snippets, but doesn't say
where to put them.

I want to not store fields beginning with an underscore. I tried adding
this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Allan_Johns · June 13, 2013, 1:56am

On further reading, the "store" property doesn't quite do what I want.

I have a field that I want to be able to search on, but I don't want it in
my _source. Is this possible?
A

On Thu, Jun 13, 2013 at 11:35 AM, Allan Johns nerdvegas@gmail.com wrote:

Anyone?

I tried:
"store" : false
also, this didn't work either.

You can actually do this, right?

A

On Wed, Jun 12, 2013 at 7:07 PM, Allan Johns nerdvegas@gmail.com wrote:

I also tried this in my default-mapping.json, no luck (see template_2):

{
"default" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "folder",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer": "gas_folder_analyzer"
}
}
},
{
"template_2" : {
"match" : "_",
"mapping" : {
"store": "no"
}
}
},
{
"template_3" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
]
}
}

On Wed, Jun 12, 2013 at 7:01 PM, Allan Johns nerdvegas@gmail.com wrote:

I can't quite figure out from the docs how to do this. I know I need to
use the exclude functionality shown here
Elasticsearch Platform — Find real-time answers at scale | Elastic, but
the documentation frustratingly provides the code snippets, but doesn't say
where to put them.

I want to not store fields beginning with an underscore. I tried adding
this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

dadoonet · June 13, 2013, 5:14am

You should know that Elasticsearch stores your source doc as is and don't modify it.
That probably explains that when you get your document back, you are still seeing source with all fields.

But, can you GIST a complete curl recreation that shows what you are doing?

Note that you could probably create default mapping when creating your index:

curl -XPOST localhost:9200/test -d '{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"type1" : {
"_source" : { "enabled" : false },
"properties" : {
"field1" : { "type" : "string", "index" : "not_analyzed" }
}
}
}
}'

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 03:35, Allan Johns nerdvegas@gmail.com a écrit :

Anyone?

I tried:
"store" : false
also, this didn't work either.

You can actually do this, right?

A

On Wed, Jun 12, 2013 at 7:07 PM, Allan Johns nerdvegas@gmail.com wrote:

I also tried this in my default-mapping.json, no luck (see template_2):

{
"default" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "folder",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer": "gas_folder_analyzer"
}
}
},
{
"template_2" : {
"match" : "_",
"mapping" : {
"store": "no"
}
}
},
{
"template_3" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
]
}
}

On Wed, Jun 12, 2013 at 7:01 PM, Allan Johns nerdvegas@gmail.com wrote:

I can't quite figure out from the docs how to do this. I know I need to use the exclude functionality shown here Elasticsearch Platform — Find real-time answers at scale | Elastic, but the documentation frustratingly provides the code snippets, but doesn't say where to put them.

I want to not store fields beginning with an underscore. I tried adding this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

dadoonet · June 13, 2013, 5:16am

Yes you can. See Elasticsearch Platform — Find real-time answers at scale | Elastic
{
"tweet" : {
"_source" : {"enabled" : false}
}
}
Why do you feel you need to disable it?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 03:56, Allan Johns nerdvegas@gmail.com a écrit :

On further reading, the "store" property doesn't quite do what I want.

I have a field that I want to be able to search on, but I don't want it in my _source. Is this possible?
A

On Thu, Jun 13, 2013 at 11:35 AM, Allan Johns nerdvegas@gmail.com wrote:

Anyone?

I tried:
"store" : false
also, this didn't work either.

You can actually do this, right?

A

On Wed, Jun 12, 2013 at 7:07 PM, Allan Johns nerdvegas@gmail.com wrote:

I also tried this in my default-mapping.json, no luck (see template_2):

{
"default" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "folder",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer": "gas_folder_analyzer"
}
}
},
{
"template_2" : {
"match" : "_",
"mapping" : {
"store": "no"
}
}
},
{
"template_3" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
]
}
}

On Wed, Jun 12, 2013 at 7:01 PM, Allan Johns nerdvegas@gmail.com wrote:

I can't quite figure out from the docs how to do this. I know I need to use the exclude functionality shown here Elasticsearch Platform — Find real-time answers at scale | Elastic, but the documentation frustratingly provides the code snippets, but doesn't say where to put them.

I want to not store fields beginning with an underscore. I tried adding this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Allan_Johns · June 13, 2013, 5:44am

David, I cannot.

I need to disable only certain fields from _source, not all of them.

I need to be able to search for documents via an indexed field, but not
actually store that field in _source. I'm a bit surprised this isn't
possible to be honest.

I feel I need to disable, ie remove, a particular field from _source
because I simply never use it in that context and it is just taking up
extra storage space. I only want to search based on the field - the actual
value of the field is not useful to me.

Thanks,
A

On Thu, Jun 13, 2013 at 3:16 PM, David Pilato david@pilato.fr wrote:

Yes you can. See
Elasticsearch Platform — Find real-time answers at scale | Elastic

{
"tweet" : {
"_source" : {"enabled" : false}
}
}

Why do you feel you need to disable it?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 03:56, Allan Johns nerdvegas@gmail.com a écrit :

On further reading, the "store" property doesn't quite do what I want.

I have a field that I want to be able to search on, but I don't want it in
my _source. Is this possible?
A

On Thu, Jun 13, 2013 at 11:35 AM, Allan Johns nerdvegas@gmail.com wrote:

Anyone?

I tried:
"store" : false
also, this didn't work either.

You can actually do this, right?

A

On Wed, Jun 12, 2013 at 7:07 PM, Allan Johns nerdvegas@gmail.com wrote:

I also tried this in my default-mapping.json, no luck (see template_2):

{
"default" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "folder",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer": "gas_folder_analyzer"
}
}
},
{
"template_2" : {
"match" : "_",
"mapping" : {
"store": "no"
}
}
},
{
"template_3" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
]
}
}

On Wed, Jun 12, 2013 at 7:01 PM, Allan Johns nerdvegas@gmail.comwrote:

I can't quite figure out from the docs how to do this. I know I need to
use the exclude functionality shown here
Elasticsearch Platform — Find real-time answers at scale | Elastic,
but the documentation frustratingly provides the code snippets, but doesn't
say where to put them.

I want to not store fields beginning with an underscore. I tried adding
this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

dadoonet · June 13, 2013, 5:52am

For that specific need, you have to disable _source field and field by field define if you want to store it or not.
Then, when searching, ask for fields you want back.

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 07:44, Allan Johns nerdvegas@gmail.com a écrit :

David, I cannot.

I need to disable only certain fields from _source, not all of them.

I need to be able to search for documents via an indexed field, but not actually store that field in _source. I'm a bit surprised this isn't possible to be honest.

I feel I need to disable, ie remove, a particular field from _source because I simply never use it in that context and it is just taking up extra storage space. I only want to search based on the field - the actual value of the field is not useful to me.

Thanks,
A

On Thu, Jun 13, 2013 at 3:16 PM, David Pilato david@pilato.fr wrote:

Yes you can. See Elasticsearch Platform — Find real-time answers at scale | Elastic
{
"tweet" : {
"_source" : {"enabled" : false}
}
}
Why do you feel you need to disable it?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 03:56, Allan Johns nerdvegas@gmail.com a écrit :

On further reading, the "store" property doesn't quite do what I want.

I have a field that I want to be able to search on, but I don't want it in my _source. Is this possible?
A

On Thu, Jun 13, 2013 at 11:35 AM, Allan Johns nerdvegas@gmail.com wrote:

Anyone?

I tried:
"store" : false
also, this didn't work either.

You can actually do this, right?

A

On Wed, Jun 12, 2013 at 7:07 PM, Allan Johns nerdvegas@gmail.com wrote:

I also tried this in my default-mapping.json, no luck (see template_2):

{
"default" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "folder",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer": "gas_folder_analyzer"
}
}
},
{
"template_2" : {
"match" : "_",
"mapping" : {
"store": "no"
}
}
},
{
"template_3" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
]
}
}

On Wed, Jun 12, 2013 at 7:01 PM, Allan Johns nerdvegas@gmail.com wrote:

I can't quite figure out from the docs how to do this. I know I need to use the exclude functionality shown here Elasticsearch Platform — Find real-time answers at scale | Elastic, but the documentation frustratingly provides the code snippets, but doesn't say where to put them.

I want to not store fields beginning with an underscore. I tried adding this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

dadoonet · June 13, 2013, 5:54am

But if it's only to save disk space, I'm wondering if you really want to do it (complexity wise).
Are your fields very huge?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 07:52, David Pilato david@pilato.fr a écrit :

For that specific need, you have to disable _source field and field by field define if you want to store it or not.
Then, when searching, ask for fields you want back.

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 07:44, Allan Johns nerdvegas@gmail.com a écrit :

David, I cannot.

I need to disable only certain fields from _source, not all of them.

I need to be able to search for documents via an indexed field, but not actually store that field in _source. I'm a bit surprised this isn't possible to be honest.

I feel I need to disable, ie remove, a particular field from _source because I simply never use it in that context and it is just taking up extra storage space. I only want to search based on the field - the actual value of the field is not useful to me.

Thanks,
A

On Thu, Jun 13, 2013 at 3:16 PM, David Pilato david@pilato.fr wrote:

Yes you can. See Elasticsearch Platform — Find real-time answers at scale | Elastic
{
"tweet" : {
"_source" : {"enabled" : false}
}
}
Why do you feel you need to disable it?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 03:56, Allan Johns nerdvegas@gmail.com a écrit :

On further reading, the "store" property doesn't quite do what I want.

I have a field that I want to be able to search on, but I don't want it in my _source. Is this possible?
A

On Thu, Jun 13, 2013 at 11:35 AM, Allan Johns nerdvegas@gmail.com wrote:

Anyone?

I tried:
"store" : false
also, this didn't work either.

You can actually do this, right?

A

On Wed, Jun 12, 2013 at 7:07 PM, Allan Johns nerdvegas@gmail.com wrote:

I also tried this in my default-mapping.json, no luck (see template_2):

{
"default" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "folder",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer": "gas_folder_analyzer"
}
}
},
{
"template_2" : {
"match" : "_",
"mapping" : {
"store": "no"
}
}
},
{
"template_3" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
]
}
}

On Wed, Jun 12, 2013 at 7:01 PM, Allan Johns nerdvegas@gmail.com wrote:

I can't quite figure out from the docs how to do this. I know I need to use the exclude functionality shown here Elasticsearch Platform — Find real-time answers at scale | Elastic, but the documentation frustratingly provides the code snippets, but doesn't say where to put them.

I want to not store fields beginning with an underscore. I tried adding this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Allan_Johns · June 17, 2013, 2:34am

No they aren't huge, but the number of documents in the database will
become huge (100s of millions) so any disk space savings I can get become
significant.

I don't think that disabling _source and storing all fields into the
indexes is the right approach, I have read specifically that this adversely
affects performance, especially when you have a lot of fields - each field
needs to be piecemeal extracted from the index, and returning _source all
in one go is faster.

It seems that this should be something fairly trivial to implement... all I
want to be able to do is discard certain fields before the document _source
is written to the database, but after the indexes have been generated.

I'd like to see some new mapping field that can be set, like so:

{
"type" : "string",
"index" : "not_analyzed",
"store" : true,
* "in_source" : false*
}

thx
A

On Thu, Jun 13, 2013 at 3:54 PM, David Pilato david@pilato.fr wrote:

But if it's only to save disk space, I'm wondering if you really want to
do it (complexity wise).
Are your fields very huge?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 07:52, David Pilato david@pilato.fr a écrit :

For that specific need, you have to disable _source field and field by
field define if you want to store it or not.
Then, when searching, ask for fields you want back.

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 07:44, Allan Johns nerdvegas@gmail.com a écrit :

David, I cannot.

I need to disable only certain fields from _source, not all of them.

I need to be able to search for documents via an indexed field, but not
actually store that field in _source. I'm a bit surprised this isn't
possible to be honest.

I feel I need to disable, ie remove, a particular field from _source
because I simply never use it in that context and it is just taking up
extra storage space. I only want to search based on the field - the actual
value of the field is not useful to me.

Thanks,
A

On Thu, Jun 13, 2013 at 3:16 PM, David Pilato david@pilato.fr wrote:

Yes you can. See
Elasticsearch Platform — Find real-time answers at scale | Elastic

{
"tweet" : {
"_source" : {"enabled" : false}
}
}

Why do you feel you need to disable it?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 13 juin 2013 à 03:56, Allan Johns nerdvegas@gmail.com a écrit :

On further reading, the "store" property doesn't quite do what I want.

I have a field that I want to be able to search on, but I don't want it
in my _source. Is this possible?
A

On Thu, Jun 13, 2013 at 11:35 AM, Allan Johns nerdvegas@gmail.comwrote:

Anyone?

I tried:
"store" : false
also, this didn't work either.

You can actually do this, right?

A

On Wed, Jun 12, 2013 at 7:07 PM, Allan Johns nerdvegas@gmail.comwrote:

I also tried this in my default-mapping.json, no luck (see template_2):

{
"default" : {
"dynamic_templates" : [
{
"template_1" : {
"match" : "folder",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer": "gas_folder_analyzer"
}
}
},
{
"template_2" : {
"match" : "_",
"mapping" : {
"store": "no"
}
}
},
{
"template_3" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
]
}
}

On Wed, Jun 12, 2013 at 7:01 PM, Allan Johns nerdvegas@gmail.comwrote:

I can't quite figure out from the docs how to do this. I know I need
to use the exclude functionality shown here
Elasticsearch Platform — Find real-time answers at scale | Elastic,
but the documentation frustratingly provides the code snippets, but doesn't
say where to put them.

I want to not store fields beginning with an underscore. I tried
adding this to my config/elasticsearch.yml:

mapping:
MYTYPE:
source:
excludes: [ "*" ]

But this didn't work.
What do I do?

thx
A

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

q42jaap · June 17, 2013, 6:56pm

Hi Allen,

Have you looked at:

http://www.elasticsearch.org/guide/reference/mapping/source-field/

{
"my_type" : {
"_source" : {
"includes" : ["path1.", "path2."],
"excludes" : ["pat3.*"]
}
}
}

You say you've specified this in a yml file, in my experience this is not
practical, I'd supply the mapping on index creation, I don't know for sure,
but I think the _source mapping info stays the same AFTER the index has
been created. Changing the yml file will not give you an error if the
mapping doesn't merge. If you create a lot of indexes, you can make a
template with the mapping. The template is part of the cluster state so it
is persisted forever in the cluster. You don't need to keep the yml file in
sync for every node in your cluster.

Have you already tried to look what the running mapping is for your index?
http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping/

Jaap

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Allan_Johns · June 18, 2013, 12:04am

Hi Jaap,

That looks to be exactly what I am after, I somehow missed this when going
through the docs.

Thanks!
A

On Tue, Jun 18, 2013 at 4:56 AM, Jaap Taal jaap@q42.nl wrote:

Hi Allen,

Have you looked at:

Elasticsearch Platform — Find real-time answers at scale | Elastic

{
"my_type" : {
"_source" : {
"includes" : ["path1.", "path2."],
"excludes" : ["pat3.*"]
}
}
}

You say you've specified this in a yml file, in my experience this is not
practical, I'd supply the mapping on index creation, I don't know for sure,
but I think the _source mapping info stays the same AFTER the index has
been created. Changing the yml file will not give you an error if the
mapping doesn't merge. If you create a lot of indexes, you can make a
template with the mapping. The template is part of the cluster state so it
is persisted forever in the cluster. You don't need to keep the yml file in
sync for every node in your cluster.

Have you already tried to look what the running mapping is for your index?
Elasticsearch Platform — Find real-time answers at scale | Elastic

Jaap

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.