Good day to you all,
Recently I installed ES on my laptop and I have been playing around with it.
At the moment I have got a Twitter pipeline running. All Twitter fields are being loaded into ES.
I would like to remove some of the fields and tried via different settings in the conf file.
For example, how do I remove these fields from loading into ES?
"_id" is not part of an array
"in_reply_to_status_id_str" is part of the array "_source"
"urls" is part of the arrays "_source" and array "entities"
Can someone tell me what code to use to filter out these three fields?
Thanks!
Sharon
"_id": "AV8DKinuBGKL_Y3fmyHR",
"_score": 5.068332,
"_source": {
"in_reply_to_status_id_str": null,
"in_reply_to_status_id": null,
"created_at": "Mon Oct 09 22:03:35 +0000 2017",
"in_reply_to_user_id_str": null,
"entities": {
"urls": [],
"hashtags": [],
"_id" is not part of an array
It's not even part of the document. _source points to the contents of the event sent by Logstash (and it's not an array).
"in_reply_to_status_id_str" is part of the array "_source"
Use a mutate filter's remove_field option to remove that field.
"urls" is part of the arrays "_source" and array "entities"
Use a mutate filter just like above, but keep in mind that it's a nested field so you need to use the [entitites][urls] notation to reference it. See Accessing event data and fields | Logstash Reference [8.11] | Elastic.
Dear Magnus,
Thanks for your reply. Am going to try tomorrow.
Thanks again,Sharon
magnusbaeck Magnus Bäck Logstash Plugins Community Maintainer
October 16 |
"_id" is not part of an array
It's not even part of the document. _source points to the contents of the event sent by Logstash (and it's not an array).
"in_reply_to_status_id_str" is part of the array "_source"
Use a mutate filter's remove_field option to remove that field.
"urls" is part of the arrays "_source" and array "entities"
Use a mutate filter just like above, but keep in mind that it's a nested field so you need to use the [entitites][urls] notation to reference it. See Accessing event data and fields | Logstash Reference [8.11] | Elastic.
Visit Topic or reply to this email to respond.
To unsubscribe from these emails, click here.