# Fleet Custom Logs integration add\_fields processor **URL:** \ **Category:** Beats\ **Tags:** fleet\ **Created:** [September 23, 2022, 11:56pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070 "2022-09-23T23:56:24Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Gustavo\_Llermaly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_llermaly/32/92019_2.png) [@Gustavo\_Llermaly](https://discuss.elastic.co/u/Gustavo_Llermaly)\ **Post date:** [September 23, 2022, 11:56pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/1 "2022-09-23T23:56:24Z")
Hello Team, we are trying to add fields to a Custom Logs integration via "Advanced settings" tried many formats but none of them is working: ```auto - processors: add_fields: target: fields: foo:bar ``` `fields.myField: value` ```auto fields: myField: value ``` If we just set : `tags: ["myvalue"]` It adds it to the documents but we want a more complex structure. EDIT: We are using Logstash as Ouput instead of ES cloud. Thanks ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 24, 2022, 4:27pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/2 "2022-09-24T16:27:59Z")
> [@Gustavo\_Llermaly](#): > > ```auto > - processors: > add_fields: > target: > fields: > foo:bar > > ``` Think you got a couple syntax errors.. you got the `-` in the wrong place... name your target fields etc .. see [here](https://www.elastic.co/guide/en/beats/filebeat/current/add-fields.html) for proper syntax (this agent integration is based on filebeat) should look like ```auto processors: - add_fields: target: project fields: name: myproject id: '574734885120952459' ``` Adds these fields to any event: ```auto { "project": { "name": "myproject", "id": "574734885120952459" } } ``` ![Screen Shot 2022-09-24 at 9.25.02 AM](https://us1.discourse-cdn.com/elastic/original/3X/4/d/4da68266b601b054ef30628defdac74bbaebc545.png) results in the doc ![Screen Shot 2022-09-24 at 9.25.34 AM](https://us1.discourse-cdn.com/elastic/original/3X/7/e/7efcef2ca085156778c03d6cfebdaf3273d4e77e.png) ![Screen Shot 2022-09-24 at 9.27.39 AM](https://us1.discourse-cdn.com/elastic/original/3X/a/8/a8d068e47d3ab965fe9a9174f3f29fc8233587ed.png) ---
**Author:** ![Gustavo\_Llermaly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_llermaly/32/92019_2.png) [@Gustavo\_Llermaly](https://discuss.elastic.co/u/Gustavo_Llermaly)\ **Post date:** [September 26, 2022, 9:10pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/3 "2022-09-26T21:10:55Z")
Thanks @stephenb , looks like using Logstash output instead of ES makes the difference. Tried with ES output and all the variants worked, then switched back to Logstash and no luck. Do you have any thoughts about this? ---
**Author:** ![Gustavo\_Llermaly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_llermaly/32/92019_2.png) [@Gustavo\_Llermaly](https://discuss.elastic.co/u/Gustavo_Llermaly)\ **Post date:** [September 26, 2022, 9:26pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/4 "2022-09-26T21:26:19Z")
now works Thank you! ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 26, 2022, 10:14pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/5 "2022-09-26T22:14:19Z")
> [@Gustavo\_Llermaly](#): > > now works Thank you! Good to hear you got it working! What was the solution?... it is good to share with the community for the next person! ---
**Author:** ![Gustavo\_Llermaly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_llermaly/32/92019_2.png) [@Gustavo\_Llermaly](https://discuss.elastic.co/u/Gustavo_Llermaly)\ **Post date:** [September 27, 2022, 6:24pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/6 "2022-09-27T18:24:25Z")
It worked locally but now it is not working , after adding the custom code log files just stop ingesting, ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 27, 2022, 6:26pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/7 "2022-09-27T18:26:28Z")
> [@Gustavo\_Llermaly](#): > > It worked locally but now it is not working , after adding the custom code log files just stop ingesting, I am not sure I Understand Locally vs What... I tried mine on a Fleet Managed Agent and it worked fine... exactly what is your config? and did you look at the agent logs? the syntax is _very_ particular... sure no typos / formatting issues? ---
**Author:** ![Gustavo\_Llermaly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_llermaly/32/92019_2.png) [@Gustavo\_Llermaly](https://discuss.elastic.co/u/Gustavo_Llermaly)\ **Post date:** [September 27, 2022, 6:28pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/8 "2022-09-27T18:28:22Z")
Locally vs client's cluster. I copied and pasted the same was working locally into the Client's Fleet server. Once I paste the the custom config that worked locally logs stops ingesting, I remove it and logs work again. Logstash is not using this enrichment fields at all We see Fleet publish the right event to Logstash, Logstash stdoutput shows the event too but we don't see the data in Elasticsearch ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 27, 2022, 6:44pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/9 "2022-09-27T18:44:42Z")
> [@Gustavo\_Llermaly](#): > > Logstash is not using this enrichment fields at all > > We see Fleet publish the right event to Logstash, Logstash stdoutput shows the event too but we don't see the data in Elasticsearch You are most likely describing a mapping issue..... 1. the new fields do not match the existing mapping and thus the documents can not be ingested because mapping does not match or 2. They have strict mapping setting that does not allow new fields and thus the document can not indexed There should be a pretty descriptive error in the logstash logs... find it... and show me. your can get the mapping of index with `GET /my-index` that should show you as well ---
**Author:** ![Gustavo\_Llermaly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_llermaly/32/92019_2.png) [@Gustavo\_Llermaly](https://discuss.elastic.co/u/Gustavo_Llermaly)\ **Post date:** [September 27, 2022, 7:01pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/10 "2022-09-27T19:01:42Z")
Thought the same, mappings are set as dynamic. Now will try to remove the noise to find some error log from Logstash. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 27, 2022, 7:22pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/11 "2022-09-27T19:22:54Z")
Also is there an ingest pipeline on the elasticsearch side? Also if you put `stdout {codec => json_lines}` in the logstash you can get a sample message and then just try to post via `curl` or kibana Dev Tools and you should easily see the error. > [@Gustavo\_Llermaly](#): > > mappings are set as dynamic. There can still be an issue like if someone already put in a field with the same name that was a keyword and now you are trying to put in as an object ... that is a pretty common hard bug to find 🙂 ---
**Author:** ![Gustavo\_Llermaly](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_llermaly/32/92019_2.png) [@Gustavo\_Llermaly](https://discuss.elastic.co/u/Gustavo_Llermaly)\ **Post date:** [September 27, 2022, 8:25pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/12 "2022-09-27T20:25:42Z")
Mystery solved. Was having an error of fields limit exceeded. Not sure why there are so many fields in there but that's a different topic. Thank you again! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 25, 2022, 10:26pm UTC](https://discuss.elastic.co/t/fleet-custom-logs-integration-add-fields-processor/315070/13 "2022-10-25T22:26:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.