# Custom Logs - json - Why do we need to map fields? **URL:** \ **Category:** Elastic Agent\ **Created:** [March 1, 2024, 3:35pm UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542 "2024-03-01T15:35:56Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![jjwallaby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jjwallaby/32/132265_2.png) [@jjwallaby](https://discuss.elastic.co/u/jjwallaby)\ **Post date:** [March 1, 2024, 3:35pm UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542/1 "2024-03-01T15:35:56Z")
Hi, json objects fields are meant to be dynamic. Does 'Custom Logs' from the 'Elastic Agent' have a facility to map json fields on the fly as entry comes in ? If you do not have any mapping an error occurs 'could not create the map from the configuration: missing field accessing 'outputs' Thanks ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [March 3, 2024, 4:39pm UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542/2 "2024-03-03T16:39:44Z")
Hi @jjwallaby Welcome to the community. Apologies, I'm not sure I'm following. Can you provide a little more information please? What version are you on? Can you confirm you're trying to Use the custom logs integration with elastic agent? Can you show the configuration? Provide a couple samples of the logs? Where exactly did you see that error? And yes elastic will dynamically create a mapping for you but there could be conflicts if you have JSON that have conflicting structures. Perhaps if you provide a little more context, we might be able to help a little bit ---
**Author:** ![jjwallaby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jjwallaby/32/132265_2.png) [@jjwallaby](https://discuss.elastic.co/u/jjwallaby)\ **Post date:** [March 6, 2024, 5:03am UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542/3 "2024-03-06T05:03:29Z")
My question was really Do you need to do the mapping of json when doing 'Custom Logs integration' ![mapping_optional](https://us1.discourse-cdn.com/elastic/original/3X/4/4/44eb19f36573582849ed6811b918c1621a8705b3.png) I was just trying to follow an example of the best practice for integration of Custom logs. I did do mappings. But not sure if it is needed. elasticsearch version:8.9.0 agent version: elastic-agent-8.9.0-linux-x86\_64 ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [March 6, 2024, 6:54am UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542/4 "2024-03-06T06:54:19Z")
> [@jjwallaby](#): > > I was just trying to follow an example of the best practice for integration of Custom logs. > I did do mappings. But not sure if it is needed. > [@jjwallaby](#): > > Do you need to do the mapping of json > when doing 'Custom Logs integration' Which example? Not sure what you mean mapping json... so in short Mappings are a best practice for scale, production, efficiency etc etc etc... If you are just messing around they are not necessary. The custom logs will do some good guesses for you with some under the covers magic ... So take a look ... But again, if you're doing production, typically you create a mapping... ---
**Author:** ![jjwallaby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jjwallaby/32/132265_2.png) [@jjwallaby](https://discuss.elastic.co/u/jjwallaby)\ **Post date:** [March 7, 2024, 1:38am UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542/5 "2024-03-07T01:38:46Z")
Thanks stephen, I see that if you do not map the json fields. the fields do not appear in the search field list of building dashboard/visualization. ![if_do_not_add_fields_in_mapping_the_dont_appear](https://us1.discourse-cdn.com/elastic/original/3X/4/2/421d7217533c1f26d2429722a3acf10d35b68ec7.png) ---
**Author:** ![jjwallaby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jjwallaby/32/132265_2.png) [@jjwallaby](https://discuss.elastic.co/u/jjwallaby)\ **Post date:** [March 7, 2024, 2:12am UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542/6 "2024-03-07T02:12:02Z")
They appear now after a couple of hours. Seems it does eventually build its data dictionary. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [March 7, 2024, 2:25am UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542/7 "2024-03-07T02:25:38Z")
What version are you on? Shift Reload of Kibana should be sufficient to see fields. so if your logs are ndjson you need to tell the integration to parse the json if you do not all the json will just end up in the `message` field.. There are 3 ways to do that (of course always more than one way) 1. ndjson parser 2. decode\_json\_fields processor 3. ingest pipeline First 2 you can just set in the Integration Setup there used to be commented out sample but not any more ---
**Author:** ![jjwallaby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jjwallaby/32/132265_2.png) [@jjwallaby](https://discuss.elastic.co/u/jjwallaby)\ **Post date:** [March 11, 2024, 9:32pm UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542/8 "2024-03-11T21:32:05Z")
The version is 8.9.0. The fields do appear after an hour. Much appreciated. Thanks for answering. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 8, 2024, 9:32pm UTC](https://discuss.elastic.co/t/custom-logs-json-why-do-we-need-to-map-fields/354542/9 "2024-04-08T21:32:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.