# Logstash 2.x : Dynamic Mapping **URL:** \ **Category:** Logstash\ **Created:** [December 4, 2015, 8:07am UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353 "2015-12-04T08:07:40Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Post date:** [December 4, 2015, 8:07am UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/1 "2015-12-04T08:07:40Z")
Hi, Based on the breaking changes in elasticsearch 2,0 and other information: [https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking\_20\_mapping\_changes.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_20_mapping_changes.html) ...its apparent that fields names with leading underscores and/or containing dots are a bad thing. I have observed that logstash's will generate a mapping against fields with leading underscores. Are there plans to make logstash's mapping logic more aware of the elasticsearch schema? And, in the meantime, how can I handle unstructured log data coming in that is quite likely to occasionally break both of the above rules? And finally, are there any other restrictions I should be mindful of with field naming etc? Regards, David ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 4, 2015, 8:46am UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/2 "2015-12-04T08:46:13Z")
You can use the de-dot plugin to help here. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 4, 2015, 12:32pm UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/3 "2015-12-04T12:32:42Z")
What do you mean by "logstash's mapping logic"? Logstash just emits JSON documents according to the rules that you set up. ---
**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Post date:** [December 4, 2015, 12:36pm UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/4 "2015-12-04T12:36:43Z")
I think I must have been misunderstanding something then. I thought that logstash was creating new mappings in elasticsearch for new data streams. It must be elasticsearch doing that itself then... ---
**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Post date:** [December 4, 2015, 12:44pm UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/5 "2015-12-04T12:44:32Z")
The de\_dot plugin only deals with dots. If fields come in that clash with meta-field names that can cause all sorts of problems too as we saw when we received log output that contained an '\_uid' field of type string... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 4, 2015, 1:00pm UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/6 "2015-12-04T13:00:57Z")
> I think I must have been misunderstanding something then. I thought that logstash was creating new mappings in elasticsearch for new data streams. It must be elasticsearch doing that itself then... Yes, ES chooses how to map fields on its own. However, Logstash by default does provide an index template for logstash-\* indexes with rules for the mapping that ES should apply, so it's not completely black and white. You can of course modify the index template so it fits your data. ---
**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Post date:** [December 4, 2015, 2:34pm UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/7 "2015-12-04T14:34:02Z")
The problem we have is that we don't know up front what the format of the inbound data is... ---
**Author:** ![vtst2412](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vtst2412/32/6228_2.png) [@vtst2412](https://discuss.elastic.co/u/vtst2412)\ **Post date:** [December 4, 2015, 7:28pm UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/8 "2015-12-04T19:28:50Z")
> [@dawiro](#): > > The problem we have is that we don't know up front what the format of the inbound data is... This should help with that ``` output{ stdout{ codec => rubydebug } } ``` ---
**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Post date:** [December 5, 2015, 9:27am UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/9 "2015-12-05T09:27:15Z")
We don't know exactly how many log sources we have (a lot) or their format (mostly bespoke). So sending data to std out is likely to lead to data overload aside from the hit on throughput. ---
**Author:** ![vtst2412](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vtst2412/32/6228_2.png) [@vtst2412](https://discuss.elastic.co/u/vtst2412)\ **Post date:** [December 5, 2015, 6:31pm UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/10 "2015-12-05T18:31:04Z")
> [@dawiro](#): > > We don't know exactly how many log sources we have (a lot) or their format (mostly bespoke) You only need to look at one log file (or better yet, one log event) using file input. And you don't have to run this on your production logstash node if performance is a concern to you (I don't imagine running one event through would cause that significant of an impact). p.s. If you don't know the log format...what are you planning on sending to ES? Just the raw message field? May we see your current logstash config. maybe it will make more sense. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:19am UTC](https://discuss.elastic.co/t/logstash-2-x-dynamic-mapping/36353/11 "2017-07-06T05:19:53Z")