# Logstash filter: find all occurrences and put in array

**URL:** <https://discuss.elastic.co/t/logstash-filter-find-all-occurrences-and-put-in-array/163412>\
**Category:** Logstash\
**Created:** [January 8, 2019, 5:24pm UTC](https://discuss.elastic.co/t/logstash-filter-find-all-occurrences-and-put-in-array/163412 "2019-01-08T17:24:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![agosmaker](https://avatars.discourse-cdn.com/v4/letter/a/edb3f5/32.png) [@agosmaker](https://discuss.elastic.co/u/agosmaker)\
**Post date:** [January 8, 2019, 5:24pm UTC](https://discuss.elastic.co/t/logstash-filter-find-all-occurrences-and-put-in-array/163412/1 "2019-01-08T17:24:21Z")

</div>

Hello,  
For example we have some log string:  
`2019-01-08 01:00:10 INFO Emails were sent from some@domain.com to a@domain.com, b@domain.com. The topic is cool. c@domain.com is ignored.`  
Is there a way to find and put all emails to an array via some filter?  
Expected result is:  
`"emails": ["some@domain.com", "a@domain.com", "b@domain.com", "c@domain.com"]`

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 8, 2019, 6:20pm UTC](https://discuss.elastic.co/t/logstash-filter-find-all-occurrences-and-put-in-array/163412/2 "2019-01-08T18:20:14Z")

</div>

If someone held a gun to my head I would implement this using

```
ruby {
    code => '
        a = event.get("message").split(" ").keep_if { |x| x.include? ("@") }
        unless a.nil?
            a = a.collect { |x| x.gsub(/[,\.]$/, "") }
            event.set("emails", a)
        end
    '
}

```

Error handling left as an exercise for the reader.

---

<div class="post-metadata">

**Author:** ![Chris\_Lyons](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_lyons/32/48107_2.png) [@Chris\_Lyons](https://discuss.elastic.co/u/Chris_Lyons)\
**Post date:** [January 9, 2019, 2:21pm UTC](https://discuss.elastic.co/t/logstash-filter-find-all-occurrences-and-put-in-array/163412/3 "2019-01-09T14:21:28Z")

</div>

We implemented a similar process to this for caputring emails within our helpdesk tickets.

GROK match:  
You can certainly do all this in Ruby if you would like.....  
NOTE: make sure to set `break_on_match => false` to capture all occurrences in the field

> match =\> { "notes" =\> ["%{EMAILADDRESS:email\_list}"] }

Pattern for RFC 5322 Official Standard (99% accurate for us):

> EMAILADDRESS (?:[a-zA-Z0-9!#$%&'_+/=?^\_`{|}~-]+(?:\.[a-zA-Z0-9!#$%&'*+\/=?^_`{|}~-]+)_|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\[\x01-\x09\x0b\x0c\x0e-\x7f])_")@(?:(?:a-zA-Z0-9?.)+a-zA-Z0-9?|[(?:(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9])).){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9])|[a-zA-Z0-9-]_[a-zA-Z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\[\x01-\x09\x0b\x0c\x0e-\x7f])+)])

Ruby Code to remove duplicates:

> ruby {  
> id =\> ""  
> code =\> "  
> emails = event.get('email\_list')  
> # If email\_list is not nil and has more than one email (is Array) then remove duplicate emails  
> if !emails.nil? and emails.kind\_of?(Array) then event.set('email\_list', emails.uniq) end  
> "  
> }

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 6, 2019, 2:21pm UTC](https://discuss.elastic.co/t/logstash-filter-find-all-occurrences-and-put-in-array/163412/4 "2019-02-06T14:21:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.