Masking logic is not working

Elastic Stack Logstash
1

Hi,

I want to mask the few fields input is coming from json . Please find the configuration below.
input
{
file
{
path => "xxx/sample.log"
}
}
filter
{
mutate
{
gsub => ["message", "PASSWORD:((?=.[a-z])(?=.[A-Z])(?=#.\d)(?=.[#@!%*?&])[a-zA-Z\d@!%?#&]{8,})", "PASSWORD",
"message", "PIN:(\d{4})", "PIN*****",
"message", "WEIGHT:[0-9]{2,3}", "WEIGHT***" ]
}
json
{
source => "message"
}
}
output
{
elasticsearch
{
hosts => ["localhost:9200"]
index => "patternmasking"
}
}

After running logstash iam getting the output of json without masking
This is my json input
{"PASSWORD":"Qwerty@123","PIN":1234,"WEIGHT":42}

Getting output like this
{
"host" => "host",
"@version" => "1",
"message" => "{"PASSWORD":"Qwerty@123","PIN":1234,"WEIGHT":42}\r",
"WEIGHT" => 42,
"@timestamp" => 2021-01-21T16:42:41.649Z,
"PIN" => 1234,
"PASSWORD" => "Qwerty@123"
}

Can you find guide me how to mask those fields.

Thanks,
Shree

2

I would suggest doing the masking after parsing the JSON, but if you want to do it before the gsub patterns would have to result in valid JSON, so something like

"message", '"PIN": \d{4}', '"PIN": "****"'
3

Hi,

I tried parsing json first then mutate gsub even though it's not masking, resulting valid json output.

input
{
}
filter
{
json
{
source => "message"
}
mutate
{
gsub => [ ```
"message", '"PIN": \d{4}', '"PIN": "****"'

}
}
output
{
}

output:
{
           "PIN" => 1234,
       "message" => "{\"PIN\":1234}\r",
    "@timestamp" => 2021-01-22T07:05:04.730Z,
      "@version" => "1",
          "host" => "host"
}
4

If you are going to parse the JSON before doing the mutates then you need to mutate the parsed fields, not the [message] field.

5

You mean like this.
mutate
{
gsub => [ ```
'"PIN": \d{4}', '"PIN": "****"'

Copy to clipboard

}
6

mutate
{
gsub => [
'"PIN": \d{4}', '"PIN": "****"'

}

7

No, more like

mutate { convert => { "PIN" => "string" } }
mutate { gsub => [ "PIN", "\d{4}", "****" ] }
8

Hi,

I tried the same. But output is not masking.
Getting output like this.
{
"message" => "{"PIN":1234}\r",
"PIN" => 1234,
"@version" => "1",
"@timestamp" => 2021-01-24T15:33:54.686Z,
"host" => "host"
}

9

PIN in an integer there, not a string, so clearly you have not applied the filters I suggested.

10

Hi,

Here is the filter part which i parsed.

mutate
{
gsub => ['"PIN"', '"\d{4}"', "****"]

}

11

Hi,

Any solution?? Waiting for the response.

Thanks

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.