# Mapping question **URL:** \ **Category:** Elasticsearch\ **Created:** [March 22, 2011, 9:22am UTC](https://discuss.elastic.co/t/mapping-question/4143 "2011-03-22T09:22:59Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Eks\_Dev](https://avatars.discourse-cdn.com/v4/letter/e/f6c823/32.png) [@Eks\_Dev](https://discuss.elastic.co/u/Eks_Dev)\ **Post date:** [March 22, 2011, 9:22am UTC](https://discuss.elastic.co/t/mapping-question/4143/1 "2011-03-22T09:22:59Z")
Hi, I am trying achieve the following: Input document has 2 fields, A1 and A2: I want to have them indexed, but I would also like to have 3rd field FULL that is union of A1 and A2 The problem is that I can map two+ fields to one indexed field by using "index\_name" : "FULL" property, but when I go with "type" : "multi\_field" , they end up mapped to "local namespace" "A1.FULL" and "A2.FULL", is there any way to specify absolute "path" (whatever terminology is appropriate here)? curl -XPUT '[http://localhost:9201/ix/any/1](http://localhost:9201/ix/any/1)' -d '{ "A1" : "aaaa", "A2" : "bbbb" }' curl -XPUT '[http://localhost:9201/ix/any/2](http://localhost:9201/ix/any/2)' -d '{ "A1" : "cccc", "A2" : "dddd" }' with mapping like this: curl -XPOST localhost:9200/ix -d '{ "mappings" : { "any" : { "\_source" : { "enabled" : true }, "properties" : { "A1" : { "type" : "multi\_field", "fields" : { "A1" : { "type" : "string", "index" : "analyzed"}, "FULL" : { "type" : "string", "index" : "analyzed", "index\_name" : "FULL" } } }, "A2" : { "type" : "multi\_field", "fields" : { "A2" : { "type" : "string", "index" : "analyzed"}, "FULL" : { "type" : "string", "index" : "analyzed", "index\_name" : "FULL" } } } } } } }' ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [March 22, 2011, 9:33am UTC](https://discuss.elastic.co/t/mapping-question/4143/2 "2011-03-22T09:33:55Z")
Hi > I am trying achieve the following: > Input document has 2 fields, A1 and A2: > I want to have them indexed, but I would also like to have 3rd field > FULL that is union of A1 and A2 Your suggested mapping won't achieve your goal. If you want 'FULL' to be the union of A1 and A2, then you should store 'FULL' as a separate field and set the value to A1 concatenated with A2, ie A1 + ' ' + A2 Alternatively, by default, A1 and A2 are already stored in the \_all field (as are any other indexed fields, unless you explicitly disable them). Perhaps this will suit your needs? > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. clint ---
**Author:** ![Eks\_Dev](https://avatars.discourse-cdn.com/v4/letter/e/f6c823/32.png) [@Eks\_Dev](https://discuss.elastic.co/u/Eks_Dev)\ **Post date:** [March 22, 2011, 10:50am UTC](https://discuss.elastic.co/t/mapping-question/4143/3 "2011-03-22T10:50:22Z")
thanks Clint, \_all is not indexed, and is not helping here. I need (A1 + A2 searchable in one field, think e.g. NGrams). Searching on two separate fields is also not the same, as frequencies are "per field"... Concatanating fields on a client side would do the job, but this is something I am trying not to do as it: - Duplicates the size of messages (A1, A2 and A1 + A2) - Makes me inflexible as I need to touch clients in order to change the way things get indexed ... This mapping is possible on ES with "index\_name" : "FULL", but it does not work as expected if you use "multi\_field" This mapping here will produce one "concatenated" field, "FULL" from A1 and A2, but will not index A1 and A2 as separate fields (A1 and A2 are going to be used as "synonums" I guess?) See last Query here I am not sure I understand how mapping works, still in learning modus ... try this: curl -XPOST localhost:9200/ix -d '{ "mappings" : { "any" : { "\_source" : { "enabled" : true }, "properties" : { "A1" : { "type" : "string", "index" : "analyzed", "index\_name" : "FULL" }, "A2" : { "type" : "string", "index" : "analyzed", "index\_name" : "FULL" } } } } }' curl -XPUT '[http://localhost:9201/ix/any/1](http://localhost:9201/ix/any/1)' -d '{ "A1" : "aaaa", "A2" : "bbbb" }' curl -XPUT '[http://localhost:9201/ix/any/2](http://localhost:9201/ix/any/2)' -d '{ "A1" : "cccc", "A2" : "dddd" }' curl -XPOST localhost:9200/\_search -d '{ "query":{"term":{"FULL":"dddd"}} }' #Shold be no match, but it is curl -XPOST localhost:9200/\_search -d '{ "query":{"term":{"A2":"aaaa"}} }' On 22 Mrz., 10:33, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) wrote: > Hi > > > I am trying achieve the following: > > Input document has 2 fields, A1 and A2: > > I want to have them indexed, but I would also like to have 3rd field > > FULL that is union of A1 and A2 > > Your suggested mapping won't achieve your goal. If you want 'FULL' to be > the union of A1 and A2, then you should store 'FULL' as a separate field > and set the value to A1 concatenated with A2, ie A1 + ' ' + A2 > > Alternatively, by default, A1 and A2 are already stored in the \_all > field (as are any other indexed fields, unless you explicitly disable > them). Perhaps this will suit your needs? > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/all-field.html) > > clint ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [March 22, 2011, 11:04am UTC](https://discuss.elastic.co/t/mapping-question/4143/4 "2011-03-22T11:04:42Z")
Hiya > \_all is not indexed, and is not helping here. > I need (A1 + A2 searchable in one field, think e.g. NGrams). Searching > on two separate fields is also not the same, as frequencies are "per > field"... The only way that I can see of you getting the frequencies of terms in the union of A1+A2 is to make that a distinct field, which means storing A1, A2, A1+A2. Or by using \_all. I don't know if anybody else has a better suggestion, but I think you're stuck with this. index\_name won't help you here. clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 22, 2011, 11:09am UTC](https://discuss.elastic.co/t/mapping-question/4143/5 "2011-03-22T11:09:33Z")
The index\_name is not meant to allow to concatenate several fields. Its just controls the name of the field created (if it will be appended with the path to the object or not). And you, most times, don't have to change it. Having the ability to create a field that is an aggregation of other several fields is valid, I agree (aside from \_all). You can open an issue for that, though, its a bit tricky to implement (as always 😉 ). On Tuesday, March 22, 2011 at 12:50 PM, eks dev wrote: > thanks Clint, > > \_all is not indexed, and is not helping here. > I need (A1 + A2 searchable in one field, think e.g. NGrams). Searching > on two separate fields is also not the same, as frequencies are "per > field"... > > Concatanating fields on a client side would do the job, but this is > something I am trying not to do as it: > > - Duplicates the size of messages (A1, A2 and A1 + A2) > - Makes me inflexible as I need to touch clients in order to change > the way things get indexed > ... > > This mapping is possible on ES with "index\_name" : "FULL", but it does > not work as expected if you use "multi\_field" > > This mapping here will produce one "concatenated" field, "FULL" from > A1 and A2, but will not index A1 and A2 as separate fields (A1 and A2 > are going to be used as "synonums" I guess?) See last Query here > > I am not sure I understand how mapping works, still in learning > modus ... > > try this: > curl -XPOST localhost:9200/ix -d '{ > "mappings" : { > "any" : { > "\_source" : { "enabled" : true }, > "properties" : { > "A1" : { "type" : "string", "index" : "analyzed", > "index\_name" : "FULL" }, > "A2" : { "type" : "string", "index" : "analyzed", > "index\_name" : "FULL" } > } > } > } > }' > > curl -XPUT '[http://localhost:9201/ix/any/1](http://localhost:9201/ix/any/1)' -d '{ > "A1" : "aaaa", > "A2" : "bbbb" > }' > > curl -XPUT '[http://localhost:9201/ix/any/2](http://localhost:9201/ix/any/2)' -d '{ > "A1" : "cccc", > "A2" : "dddd" > }' > curl -XPOST localhost:9200/\_search -d '{ > "query":{"term":{"FULL":"dddd"}} > }' > > #Shold be no match, but it is > curl -XPOST localhost:9200/\_search -d '{ > "query":{"term":{"A2":"aaaa"}} > }' > > On 22 Mrz., 10:33, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) wrote: > > > Hi > > > > > I am trying achieve the following: > > > Input document has 2 fields, A1 and A2: > > > I want to have them indexed, but I would also like to have 3rd field > > > FULL that is union of A1 and A2 > > > > Your suggested mapping won't achieve your goal. If you want 'FULL' to be > > the union of A1 and A2, then you should store 'FULL' as a separate field > > and set the value to A1 concatenated with A2, ie A1 + ' ' + A2 > > > > Alternatively, by default, A1 and A2 are already stored in the \_all > > field (as are any other indexed fields, unless you explicitly disable > > them). Perhaps this will suit your needs? > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/all-field.html) > > > > clint ---
**Author:** ![Eks\_Dev](https://avatars.discourse-cdn.com/v4/letter/e/f6c823/32.png) [@Eks\_Dev](https://discuss.elastic.co/u/Eks_Dev)\ **Post date:** [March 22, 2011, 11:57am UTC](https://discuss.elastic.co/t/mapping-question/4143/6 "2011-03-22T11:57:34Z")
Hi Shay, it is worth spending some time discussing mapping, far to many "on top of the lucene" projects map lucene analyzers 1-1, It looks to me ES goes further in that sense. Another example: "conditional analysis", mapping M to N fields: Map(STREET, HNO)-\>{street\_name, hno} Example: STREET = "Sleepy street 21" HNO = "null" should be mapped to street\_name = "sleepy street" hno = "21" OR STREET = "22nd street" HNO = "10" should be mapped to street\_name = "22nd street" hno = "10" If we were to support something like this, we would have to stretch ES config capabilities quite a lot, maybe some "request mutators" would be better. Something like Transform(Document) -\> NEW\_Document extensibility plug- in, so we are not pushing ES to provide all possible transformations, we just make it possible for users to "do whatever they want" (something they could anyhow do in client code). The problem with this is that NEW\_DOCUMENT must conform to minimum standards, (e.g. have the same ID as original Document not to screw up routing...) Due to great dynamic mapping in ES, I think it is not a problem to have it relatively simple. At the end of a day, I think of it as opening possibility to "push client code to ES" With all its gotcha-s (User should not wonder if original fields get transformed by "transformer" ).... What I am trying to say, instead of stretching ES mapping features for this, maybe it would be better provide "extension points" for users to reformulate document with their own "transformers". benefit: My example with A1, A2 and FULL and any similar would be my concern, I could indeed make transformer that maps Transform({A1, A2})-\>{A1, A2, FULL, NGramFULL, whatever} as intercepted operation... And it would be my responsibility to know what fields I can use to search. Sometimes are these operation quite complicated, imagine some extractions from unstructured documents where I need to index more then one index document like: TransformBulkText("some longish text")-\> should get transformed to two or more documents (Named Entities e.g.) I do not think it is too hard to make such "API extensions" and make ES even more usable.... but take it with reserve, I still have to dig into ES.... Just analyzing if it could help with problems I already met Thanks, Eks On 22 Mrz., 12:09, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > The index\_name is not meant to allow to concatenate several fields. Its just controls the name of the field created (if it will be appended with the path to the object or not). And you, most times, don't have to change it. > > Having the ability to create a field that is an aggregation of other several fields is valid, I agree (aside from \_all). You can open an issue for that, though, its a bit tricky to implement (as always 😉 ). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:09am UTC](https://discuss.elastic.co/t/mapping-question/4143/7 "2017-07-06T04:09:30Z")