# Selecting values from JSON **URL:** \ **Category:** Logstash\ **Created:** [July 24, 2018, 2:23pm UTC](https://discuss.elastic.co/t/selecting-values-from-json/141400 "2018-07-24T14:23:49Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Post date:** [July 24, 2018, 2:23pm UTC](https://discuss.elastic.co/t/selecting-values-from-json/141400/1 "2018-07-24T14:23:49Z")
Hello. I am parsing twitter with following settings: input { twitter { consumer\_key =\> "XX" consumer\_secret =\> "XX" oauth\_token =\> "XX" oauth\_token\_secret =\> "XX" full\_tweet =\> true use\_samples =\> true languages =\> ["en", "de"] } } output { elasticsearch { hosts =\> ["10.0.20.51:9200"] index =\> "tweets-%{+YYYY.MM.dd}" } } I do not need the massive json with more than 900 fields being in my ES. For example: ~{ "\_index": "tweets-2018.07.24", "\_type": "doc", "\_id": "wE6hzGQB2mGdQWLhJvXj", "\_version": 1, "\_score": null, "\_source": { "entities": { "hashtags": [], "urls": [ { "expanded\_url": "https://twitter.com/i/web/status/1021759356671598592", "display\_url": "twitter.com/i/web/status/1…", "url": "https://t.co/TVqHFnvmUG", "indices": [ 117, 140 ] } ], "user\_mentions": [], "symbols": [] }, "text": "En todo lo que va del año hasta ahora entraba a gim 10.20 pensando que era ese el horario (y encima llegaba tarde),… https://t.co/TVqHFnvmUG", "in\_reply\_to\_user\_id\_str": null, "extended\_tweet": { "full\_text": "En todo lo que va del año hasta ahora entraba a gim 10.20 pensando que era ese el horario (y encima llegaba tarde), hoy me enteré que entrábamos a las 11🤦", "display\_text\_range": [ 0, 154 ], "entities": { "hashtags": [], "urls": [], "user\_mentions": [], "symbols": [] } }, "quote\_count": 0, "geo": null, "timestamp\_ms": "1532441388658", "@timestamp": "2018-07-24T14:09:48.000Z", "favorited": false, "reply\_count": 0, "truncated": true, "contributors": null, "in\_reply\_to\_status\_id\_str": null, "place": null, "lang": "es", "is\_quote\_status": false, "@version": "1", "retweet\_count": 0, "favorite\_count": 0, "source": "\Twitter for Android", "filter\_level": "low" } How can I extract only following fields: "@timestamp": "lang": etc. using filter? filter { json { source =\> "@timestamp" } } It is so confusing for me. If anyone could point me to the right place, or show how to filter the given fields, would be amazing. Thanks! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 24, 2018, 2:33pm UTC](https://discuss.elastic.co/t/selecting-values-from-json/141400/2 "2018-07-24T14:33:13Z")
I would use [mutate+remove\_field](https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-mutate.html#plugins-filters-mutate-remove_field) to remove the unwanted fields. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 25, 2018, 8:18pm UTC](https://discuss.elastic.co/t/selecting-values-from-json/141400/3 "2018-07-25T20:18:29Z")
Have a look at the prune filter. ---
**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Post date:** [July 26, 2018, 8:09am UTC](https://discuss.elastic.co/t/selecting-values-from-json/141400/4 "2018-07-26T08:09:19Z")
Hello, @magnusbaeck thanks for pointing the prune filter. first I have installed it: ./logstash-plugin install logstash-filter-prune **I have managed to prototype the filter:** filter { prune { whitelist\_names =\> [ "entities.hashtags.text", "entities.user\_mentions.name", "entities.user\_mentions.id", "lang", "coordinates", "retweeted\_status.entities.hashtags.text", "retweeted\_status.entities.user\_mentions.name", "retweeted\_status.entities.user\_mentions.id", "text", "extended\_tweet.full\_text", "extended\_tweet.entities.hashtags.text", "extended\_tweet.entities.urls.url", "extended\_tweet.entities.urls.expanded\_url", "@timestamp" ] } } **The JSON that I am parsing is:** [https://pastebin.com/n69pHb7H](https://pastebin.com/n69pHb7H) However, I am accessing only not nested objects: **/kibana output** { "lang": "en", "coordinates": null, "text": "RT @liamosaur: I just heard mansplaining referred to as "correctile dysfunction" and I'm pretty shook 🤣🤣🤣", "@timestamp": "2018-07-26T08:03:23.000Z" }, "fields": { "@timestamp": [ "2018-07-26T08:03:23.000Z" ] }, "sort": [ 1532592203000 ] } I am working on this in the background. If anyone have have idea how to access those nested objects, please comment. ---
**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Post date:** [July 26, 2018, 9:04am UTC](https://discuss.elastic.co/t/selecting-values-from-json/141400/5 "2018-07-26T09:04:47Z")
At the moment I am doing following: filter { prune { whitelist\_names =\> [ "^entities$", "^lang$", "^coordinates$", "^retweeted\_status$", "^text$", "^extended\_tweet$", "^@timestamp$", "^user$" ] } } The reason to figure out the very accurate filtering is to limit the data load into ES cluster per day to the absolute minimum. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 23, 2018, 9:11am UTC](https://discuss.elastic.co/t/selecting-values-from-json/141400/6 "2018-08-23T09:11:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.