Beats input w/ json codec

[joshday]

So I know events coming from filebeat into logstash have a "source" field related to the full path of the file. The problem is my json file that I am sending will occasionally also have a 'source' field, which seems to behave in inconsistent ways. Previously (version 6.1 I think) I could rename 'source' field from JSON to keep that (dropping filebeat file path 'source') but now in 6.2 I can only get the filebeat 'source' field and cannot access JSON 'source' field. Any ideas?

For reference, I'm ingesting JSON bro logs (files.log)

[joshday]

I'll just add that for some fields with collisions I get behavior that makes sense (I keep the field values from JSON input) but other times I keep the filebeat fields instead. For example, when host is present in JSON and from filebeat I get to keep JSON value but when source is present in both I get to keep the filebeat value.

[joshday]

Fixed this problem by writing a bro script to add an additional field in files.log that is a copy of 'source' field. Source below if anyone's interested.

export {
	redef record Files::Info += {
		file_source: string &optional &log;
	};
}
event file_state_remove(f: fa_file) &priority=3
{
  # add 'file_source' if 'source' exists
  if ( f$info?$source )
  	f$info$file_source = f$info$source;
}