# Elastic Forwarder not decoding json **URL:** \ **Category:** Beats\ **Tags:** functionbeat\ **Created:** [November 18, 2022, 7:43pm UTC](https://discuss.elastic.co/t/elastic-forwarder-not-decoding-json/319330 "2022-11-18T19:43:30Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![eyear](https://avatars.discourse-cdn.com/v4/letter/e/53a042/32.png) [@eyear](https://discuss.elastic.co/u/eyear)\ **Post date:** [November 18, 2022, 7:43pm UTC](https://discuss.elastic.co/t/elastic-forwarder-not-decoding-json/319330/1 "2022-11-18T19:43:30Z")
I'm working on setting up the Elastic Forwarder. The documentation states that it automatically discovers json content - but it's not splitting all of the json content out into fields like it does when using the decode\_json\_fields processor when using functionbeat. Does this processor or something like it exist for the elastic forwarder? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [November 18, 2022, 9:46pm UTC](https://discuss.elastic.co/t/elastic-forwarder-not-decoding-json/319330/2 "2022-11-18T21:46:44Z")
Hi @eyear Welcome to the community! Are you referring to the new [Elastic Serverless Forwarder](https://github.com/elastic/elastic-serverless-forwarder/blob/main/docs/README-AWS.md) (AWS -SAR) There is a section on json discovery [here](https://github.com/elastic/elastic-serverless-forwarder/blob/main/docs/README-AWS.md#json-content-discovery) does that help? ---
**Author:** ![eyear](https://avatars.discourse-cdn.com/v4/letter/e/53a042/32.png) [@eyear](https://discuss.elastic.co/u/eyear)\ **Post date:** [November 21, 2022, 3:26pm UTC](https://discuss.elastic.co/t/elastic-forwarder-not-decoding-json/319330/3 "2022-11-21T15:26:38Z")
Hi @stephenb - Thank you. Yes, I am referring to the new elastic serverless forwarder. I am familiar with that documentation and it doesn't seem to address what I am looking for. That documentation states that the elastic forwarder is able to automatically discover json content in the payload. I am seeing that the forwarder does this, except it keeps the entire json payload in the message field and it doesn't split the message out into individual json events. I currently have the elastic forwarder hooked up to a kinesis data stream (from a cloudwatch log subscription filter), and here is an example of an ingested json document with an application error message: ```auto { "messageType": "DATA_MESSAGE", "owner": "account_id", "logGroup": "/aws/lambda/lambda_name", "logStream": "2022/11/18/[$LATEST]xxx", "subscriptionFilters": ["test_lambda_filter"], "logEvents": [{ "id": "37215743013839263", "timestamp": 1668811633088, "message": "START RequestId: ec57f841-e70b Version: $LATEST\n" }, { "id": "3721574308921578", "timestamp": 1668811636468, "message": "{\"Timestamp\":\"2022-11-18T22:47:16.3716425+00:00\",\"Level\":\"Information\",\"MessageTemplate\":\"Found credentials using the AWS SDK's default credential search\",\"Message\":\"Found credentials using the AWS SDK's default credential search\",\"SourceContext\":\"AWSSDK\",\"Environment\":\"Development\"}\n" }, { "id": "3721574313024915", "timestamp": 1668811638308, "message": "System.ArgumentException: Tenant Not Found: Going away\n at ID.Core.Data.IO.TenantContext.FetchTenantInfo(String orgId)\n at ID.Core.Lambda.LambdaBase.GetTenantInformationFromCatalog(String orgId)\n)" }] } ``` I would like this document to be broken down into individual json fields. In the past, using functionbeat with the `decode_json_fields` processor would handle all of this and break this up into a document with its respective fields after filtering off of a specific field. I have tried using the `expand_event_list_from_field` with the forwarder, and on this specific json document tried to filter on the `logEvents` field - which does narrow down this document to just everything contained in the `logEvents` list - but it still doesn't break it down into individual json events/elasticsearch fields. So - does something like the `decode_json_fields` processor exist for the elastic forwarder that I could filter off of `logEvents` to end up creating multiple key:value fields in elasticsearch? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [November 21, 2022, 3:56pm UTC](https://discuss.elastic.co/t/elastic-forwarder-not-decoding-json/319330/4 "2022-11-21T15:56:52Z")
@Andrea_Spacca any advice/ insight? ---
**Author:** ![Andrea\_Spacca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrea_spacca/32/76976_2.png) [@Andrea\_Spacca](https://discuss.elastic.co/u/Andrea_Spacca)\ **Post date:** [November 29, 2022, 2:32am UTC](https://discuss.elastic.co/t/elastic-forwarder-not-decoding-json/319330/5 "2022-11-29T02:32:03Z")
hello @eyear , sorry for the late reply > [@eyear](#): > > So - does something like the `decode_json_fields` processor exist for the elastic forwarder that I could filter off of `logEvents` to end up creating multiple key:value fields in elasticsearch? The forwarder provides only the `expand_event_list_from_field` setting, that you correctly used to access the events list inside `logEvents` What you are looking to achieve with the `decode_json_fields` processor available on filebeat should be done, with the forwarder, in an ingest pipeline directly on the Elasticsearch cluster instead: using the `json` processor [JSON processor | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/json-processor.html) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 27, 2022, 4:32am UTC](https://discuss.elastic.co/t/elastic-forwarder-not-decoding-json/319330/6 "2022-12-27T04:32:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.