How anonymize some fields?


(sampasei) #1

Hi,
i have several logs as input: each record log has different field.
I want to pass some fields and change the remaning ones: some fields are common to all records, others are totally different.
There's a way to do this without a lot of "if" ??
Thanks


(Magnus Bäck) #2

Could you give an example with a couple of if conditionals so that it's more clear what you want to do?


(sampasei) #3

here is an example

{"local": "1.2.3.4", "timestamp": 14432434.495, "event": "conne", "sensor": "test"}
{"event": "dce", "uuid": "sadfsdfsd", "timestamp": 1448234223.2078185, "sensor": "test", "dcerpcbind_transf": "4234342455345", "uid": "4056", "hostname": "rest"}

I want to send to elastic some fields (as local and timestamp) "as is" and send the other ones as null.
I want to avoid a lot of "if" conditions: how can i do?
Thanks


(Magnus Bäck) #4

Why pass them on as null—why not just delete the apparently not very interesting fields?


(sampasei) #5

To show that these fields exist


(Magnus Bäck) #6

Okay. I don't think you can set fields to null with the mutate filter so you'll have to use a ruby filter. The example in http://stackoverflow.com/a/30343349/414355 is very close to what you want except that you'll want to change the inner loop statement to this:

event[k] = nil unless wanted_fields.include? k

(system) #7