# Anonymization of sensitive data by Logstash

**URL:** <https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585>\
**Category:** Logstash\
**Created:** [August 19, 2020, 10:52am UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585 "2020-08-19T10:52:11Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [August 19, 2020, 10:52am UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/1 "2020-08-19T10:52:11Z")

</div>

Hello,  
can anyone help me how to anonymize sensitive data from **multline** log?  
How to anonymize value of parameter **sensitive data:**

```
sensitive data:more than one line
some part on second
and on third line

```

should be replaced by

```
sensitive data: ANONYMIZED

```

A structure of a log:

```
# modify 1597833688 lorem ipsum lorem ipsum lorem ipsum
lorem: ipsum
lorem: ipsum
lorem: ipsum
lorem: ipsum
-
lorem: ipsum
lorem: ipsum
-
sensitive data:more than one line
some part on second
and on third line

lorem: ipsum

```

Is it possible to do with Logstash?

---

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [August 19, 2020, 11:16am UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/2 "2020-08-19T11:16:21Z")

</div>

I've tried

```
  mutate {
    gsub => ["message", "sensitive data:\s*\S+", "sensitive data: ANONYMIZED"]
  }

```

replacing has been done only on line where _sensitive data:_ was occured

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 19, 2020, 1:09pm UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/3 "2020-08-19T13:09:25Z")

</div>

I think you would have to use a ruby filter and sprintf it.

---

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [August 20, 2020, 8:00am UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/4 "2020-08-20T08:00:43Z")

</div>

Thank you @Badger. Could you please post some simple example? I have no idea how to do using sprint. I found out something like that:

```
filter {
  ruby {
    code=>"       
      event['message'] = event.sprintf('%{message}')
    "
  }
}

```

Is there any documentation for sprintf function?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 20, 2020, 3:51pm UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/5 "2020-08-20T15:51:42Z")

</div>

Sorry, I have no idea why I suggested that. It makes no sense to me today.

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 20, 2020, 4:06pm UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/6 "2020-08-20T16:06:26Z")

</div>

Is it always 3 lines?  
Is there something to indicate when the sensitive data ends?

---

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [August 21, 2020, 8:57am UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/7 "2020-08-21T08:57:51Z")

</div>

Hello @aaron-nimocks,  
the end indicator is dash " **-**" .

```auto
-
sensitive data: more than one line
some part on second
and on third line
and other line
-

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 21, 2020, 3:55pm UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/8 "2020-08-21T15:55:43Z")

</div>

Are you ingesting that log as a single event? If you are you could use

```
mutate { gsub => ["message", "(?m)sensitive data.*^-", "sensitive data: ANONYMIZED"] }

```

If every line is standalone they you could use a ruby filter and maintain state in an @instance variable.

---

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [August 25, 2020, 1:45pm UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/9 "2020-08-25T13:45:24Z")

</div>

@badger Nice solution. Thank you very much.  
I've used this:

```
(?m)sensitive data:.*?^-", "sensitive data: ANONYMIZED"]
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 22, 2020, 1:45pm UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585/10 "2020-09-22T13:45:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.