# Remove some fields from jsion **URL:** \ **Category:** Logstash\ **Created:** [July 26, 2018, 12:11pm UTC](https://discuss.elastic.co/t/remove-some-fields-from-jsion/141757 "2018-07-26T12:11:40Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Vivek\_Sharma2](https://avatars.discourse-cdn.com/v4/letter/v/cab0a1/32.png) [@Vivek\_Sharma2](https://discuss.elastic.co/u/Vivek_Sharma2)\ **Post date:** [July 26, 2018, 12:11pm UTC](https://discuss.elastic.co/t/remove-some-fields-from-jsion/141757/1 "2018-07-26T12:11:40Z")
Hi ,I'm trying to read json from s3 and put in Elasticsearch .I put filter in logstash config file.In input I have json, nested json ,list etc. I have to fetch only six fields out of 156 fields how can I put filter? ---
**Author:** ![AurelienG](https://avatars.discourse-cdn.com/v4/letter/a/76d3ee/32.png) [@AurelienG](https://discuss.elastic.co/u/AurelienG)\ **Post date:** [July 27, 2018, 10:42am UTC](https://discuss.elastic.co/t/remove-some-fields-from-jsion/141757/2 "2018-07-27T10:42:34Z")
Hi! The prune filter allows you to create whitelists based on name or regex, but sadly it doesn't work on nested fields. Maybe you can still use it for most of your fields? I have this problem too, but all my fields are nested since they come from a split filter, so I'm looking for a better solution. ---
**Author:** ![Vivek\_Sharma2](https://avatars.discourse-cdn.com/v4/letter/v/cab0a1/32.png) [@Vivek\_Sharma2](https://discuss.elastic.co/u/Vivek_Sharma2)\ **Post date:** [July 27, 2018, 11:40am UTC](https://discuss.elastic.co/t/remove-some-fields-from-jsion/141757/3 "2018-07-27T11:40:37Z")
Thank you for your reply.I'm using mutute plugin, but not able to remove fields. my conf file:- input { s3 { bucket =\> "logstash-backup111" access\_key\_id =\> "A\*\*\*\*\*\*\*\*\*A" secret\_access\_key =\> "0\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*mqouRA" ``` } ``` } filter { json { source =\> "message" target =\> "doc" } mutate { remove\_field =\> ["[extendedOrder][minimumBasketCharge]"] } } output { ``` elasticsearch { hosts => ["localhost:9200"] } stdout { codec => rubydebug } } ``` ---
**Author:** ![AurelienG](https://avatars.discourse-cdn.com/v4/letter/a/76d3ee/32.png) [@AurelienG](https://discuss.elastic.co/u/AurelienG)\ **Post date:** [July 27, 2018, 12:04pm UTC](https://discuss.elastic.co/t/remove-some-fields-from-jsion/141757/4 "2018-07-27T12:04:35Z")
Your field [extendedOrder][minimumBasketCharge] is one of the fields created with the JSON filter, right? The whole field name might be [doc][extendedOrder][minimumBasketCharge] then. I think if you want to use [extendedOrder][minimumBasketCharge] you must not set the target in the JSON filter so the created fields stay at the root of the document. [https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html#plugins-filters-json-target](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html#plugins-filters-json-target) ---
**Author:** ![Vivek\_Sharma2](https://avatars.discourse-cdn.com/v4/letter/v/cab0a1/32.png) [@Vivek\_Sharma2](https://discuss.elastic.co/u/Vivek_Sharma2)\ **Post date:** [July 28, 2018, 2:31am UTC](https://discuss.elastic.co/t/remove-some-fields-from-jsion/141757/5 "2018-07-28T02:31:25Z")
This is json that I'm reading from s3 : { "orderDetails": { "orderNumber": "367388519", "shortOrderNumber": "8647", "customerNumber": "983e8d27-398f2bcf", "deliveryDate": "01/11/2017", "vanNumber": "2L", "customerSupportNumber": "88888 4040", "storeId": "9165", "voucherPromotionName": "d clkujkubs" }, "orderLines": [ { "productId": "uid:1c1f0b3c-239e-4218-8441-acdf3677c9fb", "productType": "Chilled", "productDescription": " 120g ", "productQuantity": "2", "productUnitPrice": "1.20", "productTotalPrice": "2.40", "productSubGroup": "F16AB", "pickingZone": "CH", "isApplicableForVat": "N", "useBy": "01/13/2017", "flagToCustomer":"Poor Quality", "productAmountAfterPromotion":"1.300" }, { "productId": "b00b6c-acfc-439d-833f-2633c39a516a", "productType": "Cupboard" "productSubGroup": "N53DG", "pickingZone": "AM", "isApplicableForVat": "N", "flagToCustomer":"Poor Quality", "productAmountAfterPromotion":"1.300" }, { "productId": "c78a9-8175-4976-df94e545a4a0", "productType": "Frozen", "productDescription": "Terate Orange 3 X 90ml \*", "productQuantity": "1", "productUnitPrice": "2.99", "productTotalPrice": "2.9900", "productSubGroup": "F76AJ", "pickingZone": "FZ", "isApplicableForVat": "Y", "flagToCustomer":"Poor Quality", "productAmountAfterPromotion":"1.300" }, { "productId": "6ba31-e023-4dba-b4dd-d26acff38c72", "productType": "Frozen", "productDescription": " Chew Ice Cream 480ml \*", "productQuantity": "1", "productUnitPrice": "2.00", "productTotalPrice": "2.0000", "productSubGroup": "F76DA", "pickingZone": "FZ", "isApplicableForVat": "Y", "flagToCustomer":"Poor Quality", "productAmountAfterPromotion":"1.300" }, ], "substitutions": [ { "orderLineId": "45878501", "promotionName": "", "orderedProductActualUnitPrice": "3.50", "orderedProductPromoTotalPrice": "3.50", "substitutedProducts": [ { "productProvidedQuantity": "1", "productProvidedActualUnitPrice": "3.50", "productProvidedActualTotalPrice": "3.5000", "productProvidedPromoUnitPrice": "3.5000", "priceMatchRefunds": "0.00", "isEssentialSubstitution": "N" } ] }, { "orderLineId": "45878786", "promotionName": "", "orderedProductId": "18-8441-acdf3677c9fb", "orderedProductType": "Chilled", "orderedProductSubgroup": "F16AB", "orderedProductPromoTotalPrice": "5.25", "substitutedProducts": [ { ``` "productProvidedDescription": "Mixed Leaf 150g ", "productProvidedQuantity": "1", "productProvidedActualUnitPrice": "1.50", "productProvidedActualTotalPrice": "3.00", "productProvidedPromoUnitPrice": "2.45", "priceMatchRefunds": "0.00", "isEssentialSubstitution": "N" } ] }, { "orderLineId": "45887889", "promotionName": "", "orderedProductId": "3c5-f418-4001-b0g", "orderedProductQuantity": "1", "orderedProductActualUnitPrice": "1.44", "orderedProductPromoTotalPrice": "1.44", "substitutedProducts": [ { "productIdProvided": "d9b9309-cf5f-4a4d-8ab7-6b463a1e3b2c", "productProvidedType": "Cupboard", "productProvidedSubgroup": "G51AR", "productProvidedPickingZone": "AM", "productProvidedActualTotalPrice": "2.3900", "productProvidedPromoUnitPrice": "1.4400", "priceMatchRefunds": "0.95", "isEssentialSubstitution": "N" } ] }, { "orderLineId": "45887890", "promotionName": "", "orderedProductId": "4d11a-7b00-4695-9a21-8d1907510908", "orderedProductType": "Cupboard", "orderedProductActualUnitPrice": "2.88", "orderedProductPromoTotalPrice": "2.88", "substitutedProducts": [ { "productIdProvided": "9b9309-cf5f-4a4d-8ab7-6b463a1e3b2c", "productProvidedType": "Cupboard", "productProvidedSubgroup": "G51AR", "productProvidedPickingZone": "AM", "productProvidedIsApplicableForVat": "Y", "productProvidedDescription": "Nature Valley Crunchy Bars Oat & Hon 5pack 210g", } ] }, { "orderLineId": "27295703", "orderedProductPromoTotalPrice": "0.77", "substitutedProducts": [ { "productProvidedSubgroup": "B33ED", "productProvidedDescription": " 6 Whlemeal Pitta Bread ", "productProvidedActualTotalPrice": "0.5000 "isEssentialSubstitution": "N", "productProvidedType": "Cupboard" } ], "orderedProductId": "a975d9-ff4b-4474-94bb-fa6baeb03034", "orderedProductSubgroup": "B33ED", "promotionName": "", "orderedProductQuantity": "1", "orderedProductActualUnitPrice": "0.38", "orderedProductDescription": " White Pitta Bread " } ``` ], "customerDetails": { "title": "Mrs", "firstName": "vivek", "middleName": "", "lastName": "Freeman-Bown", "mobileNumber": "", "emailId": "" }, "outOfStockProducts": [ { "productId": "81655524", "productDescription": "Tv Times ", "quantity": "1", "isCustomerRequestedSubstitution": "TS" }, { "productDescription": " White Pitta Bread ", "productId": "d:cba975d9-ff4b-4474-94bb-fa6baeb03034", "quantity": "1", "isCustomerRequestedSubstitution": "TS" } ], "eCoupons": { "eCoupon": "0.0000" }, "extendedOrder": { "currency": { "currencyCode": "GBP", "currencySymbol": "£" } } Now I try to fetch orderNumber,StoreId,emailId,firstname+lastname. ---
**Author:** ![AurelienG](https://avatars.discourse-cdn.com/v4/letter/a/76d3ee/32.png) [@AurelienG](https://discuss.elastic.co/u/AurelienG)\ **Post date:** [July 28, 2018, 7:46am UTC](https://discuss.elastic.co/t/remove-some-fields-from-jsion/141757/6 "2018-07-28T07:46:28Z")
Ok that's quite like my problem in the end. You only want some nested fields but prune filter doesn't work on those. Maybe you could remove some not nested fields though, **such as orderLines, substitutions, outOfStockProducts, eCoupons, extendedOrder** , that would still help if it removes all the fields below those root fields. But if you want exactly those 5 fields, you'll have to make a mutate filter with remove\_field and a list of all the fields you do not want. This is supposed to work even for nested fields, so that's the best solution for now even if it's not nice to write. ---
**Author:** ![Vivek\_Sharma2](https://avatars.discourse-cdn.com/v4/letter/v/cab0a1/32.png) [@Vivek\_Sharma2](https://discuss.elastic.co/u/Vivek_Sharma2)\ **Post date:** [July 28, 2018, 7:59am UTC](https://discuss.elastic.co/t/remove-some-fields-from-jsion/141757/7 "2018-07-28T07:59:46Z")
I'm not able to remove any fields or tags.I can add new fields but can't remove or extract.May be issue with nested json.Can you please share with me logstash.conf file so I can see how to put mutate inside filter . ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 25, 2018, 7:59am UTC](https://discuss.elastic.co/t/remove-some-fields-from-jsion/141757/8 "2018-08-25T07:59:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.