# Big JSON Payloads in Elastic APM **URL:** \ **Category:** APM\ **Tags:** nodejs\ **Created:** [October 26, 2022, 1:37pm UTC](https://discuss.elastic.co/t/big-json-payloads-in-elastic-apm/317527 "2022-10-26T13:37:41Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![rStorms](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@rStorms](https://discuss.elastic.co/u/rStorms)\ **Post date:** [October 26, 2022, 1:37pm UTC](https://discuss.elastic.co/t/big-json-payloads-in-elastic-apm/317527/1 "2022-10-26T13:37:41Z")
Hi there, we are using Elastic for Logs and APM in our application. To track bigger JSON payloads sent via REST between Microservices we selectively log those to a capped MongoDB collection in order to debug. I saw the captureBody Config, which allows the same for APM, which would be much more convenient, however the req.body "will be truncated if larger than 2 KiB.", which is understandable for performance reasons. As I am talking about Json payloads between 0.3 to 10MB, what would be the best approach here? Is this a use case Elasticsearch can be used for? Thanks! ---
**Author:** ![trentm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/trentm/32/77647_2.png) [@trentm](https://discuss.elastic.co/u/trentm)\ **Post date:** [October 26, 2022, 5:01pm UTC](https://discuss.elastic.co/t/big-json-payloads-in-elastic-apm/317527/2 "2022-10-26T17:01:55Z")
Hi @rStorms, I believe that "will be truncated if larger than 2 KiB." is out of date. (I'll open an issue to fix that doc.) The actual default truncate for a captured incoming request body is 10k characters, configurable via the `longFieldMaxLength` ([Configuration options | APM Node.js Agent Reference [4.x] | Elastic](https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html#long-field-max-length)) config var. While you could raise that `longFieldMaxLength` variable to, say, 10M to accommodate your larger request payloads, the config var applies to other fields as well. That may or may not be a concern for your app. I would worry about possible performance issues in Node.js APM agent with 10 _MB_ captured bodies. The APM agent will serialize that request body as a string in its JSON payload to the APM server. You said you "selectively log" some request payloads. You could possibly reproduce this selectivity by using the APM agent's `apm.addTransactionFilter(...)` API ([Agent API | APM Node.js Agent Reference [4.x] | Elastic](https://www.elastic.co/guide/en/apm/agent/nodejs/current/agent-api.html#apm-add-transaction-filter)). Then the next issue in the pipeline will be the maximum event size that the APM server/integration will accept, as documented at [APM input settings | APM User Guide [8.4] | Elastic](https://www.elastic.co/guide/en/apm/guide/8.4/input-apm.html#apm-input-general-settings) > Maximum size per event (int) > Maximum permitted size of an event accepted by the server to be processed (in Bytes). > > **Default:** `307200` Bytes [General recommendations | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/general-recommendations.html) includes a general recommendation for Elasticsearch to avoid large documents. It begins by talking about _100_ MB documents there. I don't have personal experience here, so I can't say if some number of 10MB documents might be problematic. My gut feeling is that there might be real performance concerns here: in the APM agent and by adding (possibly many) large string fields to the Elasticsearch data streams ([Data streams | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/data-streams.html)) created for APM data. As well, depending if/how you want to search over the captured request body documents, you may want to have control over the Elasticsearch index templates for this data. If so, a better design may be to have that data explicitly going to a data stream or index that is _separate_ from the ones used by the APM system. ---
**Author:** ![rStorms](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@rStorms](https://discuss.elastic.co/u/rStorms)\ **Post date:** [October 27, 2022, 6:59am UTC](https://discuss.elastic.co/t/big-json-payloads-in-elastic-apm/317527/3 "2022-10-27T06:59:20Z")
Hi @trentm, wow, this is great advice and confirms my feelings. I will think about a separate data stream and maybe link the data another way. Thanks! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 17, 2022, 3:00am UTC](https://discuss.elastic.co/t/big-json-payloads-in-elastic-apm/317527/4 "2022-11-17T03:00:06Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.