Code review: masking credit card numbers

Elastic Stack Logstash
1

We have a failsafe to mask credit card numbers just in case they appear in a log (they shouldn't but...). We have a Grok pattern for credit card numbers and want to replace any occurrences of that with 16 X's.

# Check for credit card numbers and then replace them with XXXX...
grok {
  patterns_dir => "/etc/logstash/conf.d/patterns"
  match => { "message" => "%{GREEDYDATA:ccPart1}%{CREDITCARDNUMBER:ccNumber}%{GREEDYDATA:ccPart2}" }
  add_tag => [ "ccDetected" ]
  tag_on_failure => []
}
if "ccDetected" in [tags] {
  if ![ccPart1] { mutate { add_field => { "ccPart1" => "" } } }
  mutate {
    replace => [ "message", "%{ccPart1}XXXXXXXXXXXXXXXX%{ccPart2}" ]
    remove_field => [ "ccPart1", "ccPart2", "ccNumber" ]
  }
}

It seems like there should be a more efficient way of doing this (besides creating a custom plugin). We're still on Logstash 1.4.2.

2

Wouldn't the gsub option of a mutate filter work?

3

Ah, good point. Thanks!

4

Ah, there is one thing though. In the original code a tag was added when a credit card number was found. Using Mutate/gsub what's the best way to add that tag now?

5

Something like

if [message] =~ /XXXXXXXXXXXXXXXX/ {
  mutate {
    add_tag => ["ccNumber"]
  }
}

maybe?

6

What's the difference between =~ and in as operators? I think =~ can have regex but in this simple case are these the same?

if [message] =~ /XXXXXXXXXXXXXXXX/ {...}

vs.

if "XXXXXXXXXXXXXXXX" in [message] {...}
7

Oh, right. Yeah, prefer the in operator in this case. It's a simple substring search that's faster than a regexp (but obviously not as expressive).

1 Like