I dont want (or expect) you guys to solve these issues over this medium. We are more so looking for conceptual knowledge that only Elastic personnel would know so we will be able to improve our own trouble shooting methods. With the general knowledge you can provide we can take that and run with it.
The zeek.conf file we are speaking of has been in place for over 3 years and has worked without issue (it has also NEVER had a filter plug-in added to it).
We have used Filebeat processors to successfully add fields (We have Filebeat-zeek module- running on SERVER A- which is at the very "top" of the data stream) These new fields will successfully populate in Kibana. Giving credence that the filebeat processors are working properly.
The only singular change we have now made is attempting to add Logstash filters to further adjust those fields (the ones that were created by the Filebeat processors)
The attempted manipulation from the Logstash filters are not working.
This would leave two possible reasons why (unless you can give insight for additional reasons)
#1) The syntax is incorrect
1b) We tried a VARIETY of syntax variations and none worked. (it still very well could be a syntax error, but we wanted to see if you guys could give insight on #2
#2) The filebeat processor does not create the fields until further "down" the data pipeline
2b) this is where your Elastic expertise could come into play giving some insight on when the fields are created.
I know you are asking for config files but if the zeek.conf file has worked for over 3 years and the only addition we are making is a logstash filter, why would the inputs and the outputs matter? (again this is conceptual knowledge that you may have that we dont so any information could assist us and help us where we may be ignorant)
Information like what legoguy1000 provided (if true) is exactly what were looking for. Certain fields are created in Filebeat then the remainder in the ingest pipeline. We could then determine which fields fall into either of the categories and go from there.