# Supporting query on dynamic columns in elastic search **URL:** \ **Category:** Elasticsearch\ **Created:** [January 1, 2016, 2:30pm UTC](https://discuss.elastic.co/t/supporting-query-on-dynamic-columns-in-elastic-search/38244 "2016-01-01T14:30:48Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Harish\_Kommaraju](https://avatars.discourse-cdn.com/v4/letter/h/b487fb/32.png) [@Harish\_Kommaraju](https://discuss.elastic.co/u/Harish_Kommaraju)\ **Post date:** [January 1, 2016, 2:30pm UTC](https://discuss.elastic.co/t/supporting-query-on-dynamic-columns-in-elastic-search/38244/1 "2016-01-01T14:30:48Z")
I need to support query on dynamic (not predefined) tags in elastic search. Lets say I have a blog document and wanted to support query on different set of columns i.e. tagTypeA=valueX & tagTypeB=ValueY and these tagTypeX columns are not known beforehand. There will be only one value for each of these tags. The user will pass this additional data as Map String::String to my API (no strict model / structure) I am thinking of three ways to support this feature. 1. Declare that I can support a maximum of N type of dynamic tags only per document (say 10) and create internal columns like Tag1,Tag2 ... Tag10. Now have a config to maintain the mapping of TagTypeA=Tag1, TagTypeB=Tag2 etc. In the code, iterate the input key value pair and generate ES search query dynamically by using key to columnName mapping. Pros : Simple to implement Cons : Overhead of maintaining the mapping. This has to modified every-time a new type of document/client is onboarded / new field has to be added for existing client. 2. Create a non-analyzed field in ES with array of strings. When storing the data, store in a concatenated format of key+"Delimiter"+value. So if the input map has TagTypeA=Good & TagTypeB=High, then this will be stored as ["TagTypeA-Good","TagTypeB-High"] in ES. When user queries, construct back the contacted strings and search them. Pros : No code changes required to onboard new clients / to add or update new fields Cons : First of all it doesn't sound clean. The key should not have Delimeter. Changing mapping at later point of time is very tedious as we have to change values of all existing string values. 3. Don't define any schema and let the json key - value pair of tags passthrough to elastic search PUT call. For any new keys which are not already present elastic search will automatically add it to the indices with default type inference (which can be controlled using dynamic tempaltes). Pros : No configuration or manual concatenation of input. Any addition of columns in handled transparently without any manual effort. Cons : We are relying on the default index creation settings of ES which may not suit the requirement always. I feel there will be more cons on this, but can't think of them any. Please suggest. I am personally thinking on aligning to Option #3. Can any one please share your views on above three approaches and if there is a better way to solve this. Thanks, Harish ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 2, 2016, 5:46am UTC](https://discuss.elastic.co/t/supporting-query-on-dynamic-columns-in-elastic-search/38244/2 "2016-01-02T05:46:19Z")
Another option worth considering might be to store them as [nested data types](https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html?q=nested%20documents), each column stored as a document with a 'key' and 'value' field. This avoids having the mappings explode while still giving you a fair amount of flexibility. ---
**Author:** ![Harish\_Kommaraju](https://avatars.discourse-cdn.com/v4/letter/h/b487fb/32.png) [@Harish\_Kommaraju](https://discuss.elastic.co/u/Harish_Kommaraju)\ **Post date:** [January 2, 2016, 12:40pm UTC](https://discuss.elastic.co/t/supporting-query-on-dynamic-columns-in-elastic-search/38244/3 "2016-01-02T12:40:40Z")
Thanks for your suggestion. I also need to do aggregations on the keys i.e. queries like No. of entries having TagTypeA=23 & TagTypeB=45. Will the performance be impacted by having nested structure for these aggregations? (I understand that Option #2 is no longer a valid one when we need aggregations on these column names) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:27pm UTC](https://discuss.elastic.co/t/supporting-query-on-dynamic-columns-in-elastic-search/38244/4 "2017-07-05T23:27:27Z")