Replace paths in every field


I've been trying to edit my logstash.config so that all file paths in all fields are changed to "anonym".

input {
    http {
        port => 5044
        codec => json

filter {
    split {
        field => "events"
        target => "event"
        remove_field => "events"
    ruby {
        code => 'event.to_hash.each { |k, v|
            if v == "(?:[\w]\:|\\)(\\[A-Z\a-z_\-\s0-9\.]+)+(\.(txt|gif|pdf|doc|docx|xls|xlsx|log))?"
                event.set(k, "anonym")

output {
    elasticsearch {
        hosts => ["http://localhost:9200"]
        index => "scs-%{+YYYY.MM.dd}"

That's my attempt but it's not working. I've tried something simpler instead of the RegEx but still nothing happens.

Thanks in advance

You should use a grok filter instead.

FYI: I have not tested this config, but I hope you get the general idea.
Use a grok filter to figure out if it matches the pattern.
If it does match (_grokparsefailure is NOT thrown) then replace the event with "anonym".

If _grokparsefailure IS thrown then you know that there is no path in your event field.

Then always remove _grokparsefailure tag.

grok {
	match => {"event" => "%{PATH}"}

if "_grokparsefailure" not in ["tags"] {
	mutate {
		replace => {"event" => "anonym"}
mutate {
	remove_tag ["_grokparsefailure"]

Thanks for your help :slight_smile:
Is there a way to just change the path in the fields to "anonym" and not the whole event?

I thought that the field was called "event". You have to replace the field with whatever you want inside of it, there is no easy way to just replace a path inside of a field. If you want to do that you'll have to do some ruby coding and fancy regex.

Yes, that's what I've tried at the top. I've tested the regex and it should work. But nothing happens at all. I don't know what's wrong with my ruby code.

Did you mean if v =~ ...? You are doing a string literal comparison, not a regexp match.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.