# Masking the password and handling two client IP's in apache\_access log

**URL:** <https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927>\
**Category:** Logstash\
**Created:** [May 29, 2020, 1:39pm UTC](https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927 "2020-05-29T13:39:50Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![marco2005](https://avatars.discourse-cdn.com/v4/letter/m/b5ac83/32.png) [@marco2005](https://discuss.elastic.co/u/marco2005)\
**Post date:** [May 29, 2020, 1:39pm UTC](https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927/1 "2020-05-29T13:39:51Z")

</div>

Sometimes, I have standard apache logs starting with a single client IP and sometimes with two client IP's like this which my logstash config does not handle. Shall I use a 3rd pattern in Grok ?

> 1.2.3.4,5.6.7.8

Also, How to mask the value of the key secret =ABCDEF to secet=xxxxx ?

I have the below log line.  
`- - - [28/May/2020:21:46:07 +0200] \"GET /dev/log/foo.py?dev_name=aggr_api&aggr_name=wip_aggr_test&output_format=python&_username=_tech_user&_secret=ABCDEF HTTP/1.1\" 200 50 \"-\" \"Python-urllib/2.6"`

Here is my logstash config(I have not implemented for 2 client IP's yet):

```
filter {
     grok {
            match => [
                "message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}",
                "message" , '-*%{SYSLOG5424SD}\s":?%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent}'
            ]
            overwrite => ["message"]
        }
        if ("&" in [request]) {
            grok {
                match => ["request", "%{DATA:script_name}\?%{GREEDYDATA:script_kv}"]
            }
            kv {
                source => "script_kv"
                field_split => "&"
                value_split => "="
                remove_char_key => "_"
                prefix => "query_string_"
            }
            mutate {
                remove_field => ["request"]
                remove_field => ["script_kv"]
            }
            if [query_string_secret] {
                mutate {
                    gsub => ["query_string_secret", ".*$", "X"]
                }
            }
        }
         if "secret" in [message] {
                mutate {
                    gsub => [
                        "message", "_secret=.*\&", "_secret=XXXX&",
                        "message", "_secret=\w+\&", "_secret=XXXX&"
                    ]
                }
            }

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 29, 2020, 1:58pm UTC](https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927/2 "2020-05-29T13:58:40Z")

</div>

I would define a third grok pattern. Use a variant of the [COMBINEDAPACHELOG](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/httpd) pattern that replaces IPORHOST with NOTSPACE for the clientip field.

To mask the secret use mutate+replace. Either replace it with a constant, or, if necessary, replace it with a hash of the secret (created using a fingerprint filter).

---

<div class="post-metadata">

**Author:** ![marco2005](https://avatars.discourse-cdn.com/v4/letter/m/b5ac83/32.png) [@marco2005](https://discuss.elastic.co/u/marco2005)\
**Post date:** [May 29, 2020, 2:03pm UTC](https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927/3 "2020-05-29T14:03:54Z")

</div>

Okay. Thanks. Let me try this. But what I don't get is that why this is not working ?

```
    if "secret" in [message] {
            mutate {
                gsub => [
                    "message", "_secret=.*\&", "_secret=XXXX&",  
                    "message", "_secret=\w+\&", "_secret=XXXX&"
                ]
            }
        }

```

The first pattern in gsub works for the standard apache logs.It does the masking fine. Now, I added the second gsub pattern for the non-standard apahce log without the client IP and the secret masking doesn't ork with either of these patterns. Is this the right way to do it ?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 29, 2020, 2:47pm UTC](https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927/4 "2020-05-29T14:47:45Z")

</div>

In your sample data the value of \_secret is followed by space, not &

---

<div class="post-metadata">

**Author:** ![marco2005](https://avatars.discourse-cdn.com/v4/letter/m/b5ac83/32.png) [@marco2005](https://discuss.elastic.co/u/marco2005)\
**Post date:** [May 29, 2020, 3:44pm UTC](https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927/5 "2020-05-29T15:44:42Z")

</div>

Also, I ended up using DATA instead of NOTSPACE for 2 clientip's because I found spaces after the comma. So, the below works fine:

`"message" , '%{DATA:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent}',`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 26, 2020, 3:44pm UTC](https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927/6 "2020-06-26T15:44:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.